Emulating Terraform Module for_each

README.md

Emulating Terraform Module for_each

tl;dr

We want to create n resources, and 1:m relationships to each of these resources. In the end we expect n + n*m resources to be created.

Running the example:

$ terraform apply -auto-approve
module.foreach.null_resource.outer["bbb"]: Creating...
module.foreach.null_resource.outer["aaa"]: Creating...
module.foreach.null_resource.outer["aaa"]: Creation complete after 0s [id=9081550616613728]
module.foreach.null_resource.outer["bbb"]: Creation complete after 0s [id=8573813731028499305]
module.foreach.null_resource.inner["bbb_sss"]: Creating...
module.foreach.null_resource.inner["bbb_rrr"]: Creating...
module.foreach.null_resource.inner["aaa_xxx"]: Creating...
module.foreach.null_resource.inner["aaa_yyy"]: Creating...
module.foreach.null_resource.inner["bbb_rrr"]: Creation complete after 0s [id=2978539577358065537]
module.foreach.null_resource.inner["bbb_sss"]: Creation complete after 0s [id=5412028042115444627]
module.foreach.null_resource.inner["aaa_yyy"]: Creation complete after 0s [id=1814355603638411593]
module.foreach.null_resource.inner["aaa_xxx"]: Creation complete after 0s [id=1161253475644873094]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Examining the resources that got created:

$ terraform state list

module.foreach.null_resource.outer["aaa"]
module.foreach.null_resource.outer["bbb"]
module.foreach.module.inner.null_resource.inner["aaa_xxx"]
module.foreach.module.inner.null_resource.inner["aaa_yyy"]
module.foreach.module.inner.null_resource.inner["bbb_rrr"]
module.foreach.module.inner.null_resource.inner["bbb_sss"]

The dirty hacky workaround until we have module for_each

The trick in this workaround is twofold:

First, each module needs to do the for_each iteration within. Second, we need to concatenate the resource keys the deeper we nest for_each.

First, the module needs to get a map as an input (items).

module outer {
  source = "./modules/outer"
  items = {
    aaa = {
      spec = "something"
      inners = {
        xxx = "x",
        yyy = "y"
      }
    },
    bbb = {
      spec = "whatever"
      inners = {
        rrr = "r",
        sss = "s"
      }
    }
  }
}

we have to for_each each resource in the outer module:

resource null_resource outer {
  for_each = var.items
  triggers = {
    item = each.value.spec
  }
}

Then we need to concatenate the outer and inner keys and key a new map with that.

locals {
  items = {
    for value in flatten([
      for outer_name, outer in var.items : [
        for inner_name, inner in outer.inners : {
          outer_name = outer_name,
          inner_name = inner_name,
          outer      = outer,
          inner      = inner
        }
      ]
    ]) :
    "${value.outer_name}_${value.inner_name}" => {
      outer_name = value.outer_name,
      inner      = value.inner
    }
  }
}

We then pass that to the inner module.

module inner {
  source = "../../modules/inner"
  items = local.items
  outer_resources = null_resource.outer
}

Then the next for_each is to be put within the inner module:

resource null_resource inner {
  for_each = var.items
  triggers = {
    item = each.value.inner
    outer_resource_id = var.outer_resources[each.value.outer_name].id
  }
}

The ideal scenario: module for_each

What we actually want is declaring a module n times (for_each), and each of those module instances declare another module m times (for_each again).

Basically you would declare a module:

module outer {
  source = "./modules/outer"
  for_each = {
    aaa = {
      spec = "something"
      inners = {
        xxx = "x",
        yyy = "y"
      }
    },
    bbb = {
      spec = "whatever"
      inners = {
        rrr = "r",
        sss = "s"
      }
    }
  }

  item = each.value
}

The outer module would look something like this:

resource null_resource outer {
  triggers = {
    item = var.item.spec
  }
}

module inner {
  source = "../../modules/inner"
  for_each = var.item.inners
  item = each.value
  outer_item_id = null_resource.outer.id
}

The inner module would look something like this:

resource null_resource inner {
  triggers = {
    item = var.item
    outer = var.outer_item_id
  }
}

The inner module only needs to concern itself with a single resource, for_each handles multiple declarations for you outside the module definition at the module users side.

The outer module only needs to concern itself with a single resource, and declaring the child module with for_each. Multiple instances of the outer resources are outside the scope of the module.

The outermost declaration of the outer module is then the one who declares the second for_each for you.

In an ideal world this is how I would use Terraform Modules with for_each support.

Lucky for us Module for_each support is constantly being worked on by the Terraform core team: hashicorp/terraform#10462 (comment)

gistlog.yml

published: true