A possible API for a Terraform provider that wraps Terraform.

README.md

Wrapper.tf

As Terraform is a state machine, there is no reason Terraform couldn't wrap and execute itself.

Problems with Terraform

• Workspaces are shit!
• Configuring backends are a pain!
• You have to embed modules into modules to track cascading changes between them

Problems with Terragrunt

• Noisy directory structure
• Deleting .hcl doesn't destroy that module implementation, so it needs destroying first before deletion.
• Changes still don't cascade fully.
• You need to install another binary to run Terraform

Solution

Create a Terraform provider to wrap terraform.

Benifits

• You can exploit Terraforms own syntax to drive it
• You can run your modules in isolation
• You can wire module outputs to the inputs of another
• Deleting a module from the configuration, can cause terraform to run a destroy operation
• You can easily target a specific module implementation using Terraforms own -target switch.
• You'll get some seriously DRY terraform projects.
• You should no longer need to use terraform_remote_state due to wiring between modules.

So what would a terraform project look like using this method?

Using this method your terraform modules should be extremely DRY, only requiring top-level .tf files for wrapping your module configuration, then a folder structure for your modules

project
│   main.tf # your wrapper terraform file
│
└───modules
    │
    └───aws-vpc
    │       main.tf
    └───aws-alb
            main.tf

Other Potential Benifits

• We could create a similar product for SAM and CloudFormation, allowing us to wire Inputs/Variables/Outputs between them and Terraform removing the need for hacky solutions for resource information.

alternative-api.tf

// provider configures the backend for every instance of tf_module
provider "tf_runner" {
  state {
//  profile = ...
//  region  = ...
    bucket  = "terraform-state-management"
    table   = "terraform-state-management"
  }
}

// only the backend key is set per tf_module rather than the full backend configuration
resource "tf_module" "my_module" {
  key       = "/us-east-1/aws-vpc/terraform.tfstate"
  module    = "modules/aws-vpc"
  variables = {
    region = "us-east-1"
  }
}

complex-example.tf

variable "tags" {
  type    = map(string)
  default = {}
}

variable "module_dir" {
  default = "terraform/modules"
}

variable "key_prefix" {
  default = "live"
}

variable "key_postfix" {
  default = "terraform.tfstate"
}

variable "region" {
  default = "us-east-1"
}

// configure the backend used accross your modules
data "tf_s3_backend" "this" {
  region = var.region
  bucket = "terraform-state-management"
  table  = "terraform-state-management"
}

// implement a module
resource "tf_module" "use1_aws_vpc" {
  // use the predefined backend config
  backend   = data.terraform_s3_backend.config
  // set your own backend key
  key       = "${var.key_prefix}/us-east-1/aws-vpc/${var.key_postfix}"
  // set the module you wish to apply
  module    = "${var.modules}/aws-vpc"
  // set the variables you want on the apply
  variables = {
    region = var.region
    tags   = var.tags
  }
}

// this module implicitely depends on use1_aws_vpc
resource "tf_module" "use1_aws_alb" {
  backend   = data.terraform_s3_backend.config
  key       = "${var.key_prefix}/us-east-1/aws-alb/${var.key_postfix}"
  module    = "${var.modules}/aws-vpc"
  variables = {
    region = var.region
    // here we are wiring the outputs of the above module into this one
    vpc_id = tf_module.use1_aws_vpc.outputs.vpc_id
    tags   = var.tags
  }
}

minimal-example.tf

data "tf_s3_backend" "this" {
  region = "us-east-1"
  bucket = "terraform-state-management"
  table  = "terraform-state-management"
}

resource "tf_module" "my_module" {
  backend   = data.terraform_s3_backend.config
  key       = "/us-east-1/aws-vpc/terraform.tfstate"
  module    = "modules/aws-vpc"
  variables = {
    region = "us-east-1"
  }
}