Terragrunt config to auto-generate provider and backend config so you can apply library modules directly in infrastructure-live without an adapter module in infrastructure-modules

README.md

Using Terragrunt generate for extra DRY Terraform

Terragrunt config to auto-generate provider and backend config so you can apply library modules directly in infrastructure-live without an adapter module in infrastructure-modules.

dev > .terraform-version

0.22.4

dev > .terragrunt-version

0.12.21

dev > ca-central-1 > dev > data-stores > sqs-worker > terragrunt.hcl

# ------------------------------------------------------------------------------
# ENVIRONMENT VARIABLES
# Define these secrets as environment variables
# ------------------------------------------------------------------------------

# AWS_ACCESS_KEY_ID
# AWS_SECRET_ACCESS_KEY

# ------------------------------------------------------------------------------
# TERRAGRUNT CONFIGURATION
# This is the configuration for Terragrunt, a thin wrapper for Terraform that supports locking and enforces best
# practices: https://github.com/gruntwork-io/terragrunt
# ------------------------------------------------------------------------------

# Terragrunt will copy the Terraform configurations specified by the source parameter, along with any files in the
# working directory, into a temporary folder, and execute your Terraform commands in that folder.
terraform {
  source = "git::git@github.com:gruntwork-io/package-messaging.git//modules/sqs?ref=v0.3.1"
}

# Include all settings from the root terragrunt.hcl file
include {
  path = find_in_parent_folders()
}

dependency "kms_master_key" {
  config_path = "../../../_global/kms-master-key"
}

inputs = {
  name = "worker"

  visibility_timeout_seconds = 60
  message_retention_seconds = 86400
  max_message_size = 131072
  delay_seconds = 10
  receive_wait_time_seconds = 20
  dead_letter_queue = true

  kms_master_key_id = dependency.kms_master_key.outputs.remote_state.key_id
}

dev > ca-central-1 > dev > env.yaml

# These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
vpc_name: "dev"

dev > ca-central-1 > region.yaml

# These variables apply to this entire AWS region. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
aws_region: "ca-central-1"

dev > empty.yaml

# This is intentionally empty object. This YAML file is used as a catch all to return an empty map when expected yaml
# vars do not exist. Note that this file can not be completely empty because terragrunt's yamldecode function expects an
# object, and it doesn't treat empty files as an empty object (you get a syntax error).
{}

dev > terragrunt.hcl

# ---------------------------------------------------------------------------------------------------------------------
# TERRAGRUNT CONFIGURATION
# Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules,
# remote state, and locking: https://github.com/gruntwork-io/terragrunt
# ---------------------------------------------------------------------------------------------------------------------

# Configure Terragrunt to automatically store tfstate files in an S3 bucket
remote_state {
  backend = "s3"
  config = {
    encrypt        = true
    bucket         = local.terraform_state_s3_bucket
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = local.aws_region
    dynamodb_table = "terraform-locks"
  }
}

# ---------------------------------------------------------------------------------------------------------------------
# LOCAL PARAMETERS (just within this file)
# ---------------------------------------------------------------------------------------------------------------------
locals {
  aws_region     = "ca-central-1"
  aws_account_id = "000000000000"
  account_alias  = "dev"
  terraform_state_s3_bucket = "mycompany-${local.account_alias}-${local.aws_account_id}-terraform-state"
}

# ---------------------------------------------------------------------------------------------------------------------
# GLOBAL PARAMETERS
# These variables apply to all configurations in this subfolder. These are automatically merged into the child
# `terragrunt.hcl` config via the include block.
# ---------------------------------------------------------------------------------------------------------------------

inputs = merge(
  # Configure Terragrunt to use common vars encoded as yaml to help you keep often-repeated variables (e.g., account ID)
  # DRY. We use yamldecode to merge the maps into the inputs, as opposed to using varfiles due to a restriction in
  # Terraform >=0.12 that all vars must be defined as variable blocks in modules. Terragrunt inputs are not affected by
  # this restriction.
  yamldecode(
    file("${get_terragrunt_dir()}/${find_in_parent_folders("region.yaml", "${path_relative_from_include()}/empty.yaml")}"),
  ),
  yamldecode(
    file("${get_terragrunt_dir()}/${find_in_parent_folders("env.yaml", "${path_relative_from_include()}/empty.yaml")}"),
  ),
  # Additional global inputs to pass to all modules called in this directory tree.
  {
    aws_account_id             = local.aws_account_id
    terraform_state_s3_bucket  = local.terraform_state_s3_bucket
    terraform_state_aws_region = local.aws_region
  },
)

# When using this terragrunt config, terragrunt will generate the file "provider.tf" with the aws provider block before
# calling to terraform. Note that this will overwrite the `provider.tf` file if it already exists.
generate "provider" {
  path      = "provider.tf"
  if_exists = "skip" # Allow modules to override provider settings
  contents = <<EOF
provider "aws" {
  # The AWS region in which all resources will be created
  region = "${local.aws_region}"

  # Only these AWS Account IDs may be operated on by this template
  allowed_account_ids = ["${local.aws_account_id}"]

  version = "~> 2.50.0"
}
provider "external" {
  version = "~> 1.2"
}
provider "null" {
  version = "~> 2.1"
}
provider "template" {
  version = "~> 2.1"
}
EOF
}

generate "backend" {
  path      = "backend.tf"
  if_exists = "overwrite"
  contents = <<EOF
terraform {
  backend "s3" {}
}
EOF
}

FYI: this file is not in YAML format: https://gist.github.com/AlainODea/61ef3da56b27d8cc81490a5e70db2b64#file-dev-ca-central-1-dev-env-yaml

FYI: this file is not in YAML format: https://gist.github.com/AlainODea/61ef3da56b27d8cc81490a5e70db2b64#file-dev-ca-central-1-dev-env-yaml

@jasonwbarnett explain your reasoning? It validates as YAML and works when processed, so I don't understand your assertion here.

More precicely I would expect the value to be:

# These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
vpc_name: "dev"

Diff below:

--- c/env.yaml
+++ i/env.yaml
@@ -1,3 +1,3 @@
 # These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
 # setting in the root terraform.tfvars file's Terragrunt configuration.
-vpc_name = "dev"
+vpc_name: "dev"

More precicely I would expect the value to be:

# These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
# setting in the root terraform.tfvars file's Terragrunt configuration.
vpc_name: "dev"

Diff below:

--- c/env.yaml
+++ i/env.yaml
@@ -1,3 +1,3 @@
 # These variables apply to this entire environment. They are automatically pulled in using the extra_arguments
 # setting in the root terraform.tfvars file's Terragrunt configuration.
-vpc_name = "dev"
+vpc_name: "dev"

@jasonwbarnett it is valid YAML either way. What I provided is in the form Gruntwork provides in examples of env.yaml. Is there a problem created by the format I shared here beyond a code style preference?

EDIT: it is NOT valid YAML. I should have gone to the spec and not trusted Terragrunt's successfully parsing my original garbage input to be an authority on its validity. My bad here.

@AlainODea I'm not sure how else to communicate that YAML format uses colons (:), not equals (=), to assign keys a value.

@AlainODea I'm not sure how else to communicate that YAML format uses colons (:), not equals (=), to assign keys a value.

You are absolutely correct here. Good catch! Sorry for my denseness.

Not sure why Terragrunt's YAML implementation doesn't flag this. I can't find anywhere in the specification that allows the = for key value construction. This is a cursed implementation. I can't unsee this now 😅