Recently, one of my customers requested that I create some Terraform-based automation to help them deploy several (AWS) VPC Endpoint definitions into their accounts. This customer maintains a set of cloud-hosted environments for development, integration, testing and production activities. This meant that I was going to have to write things in a way that was going to be flexible enough to be repeatably-excecuted across all their environments.
Further, when I asked the customer for a specific list of services they wanted endpoints created for, the response I got back was basically, "all of them?". Previously, the customer had been creating per-service automation as they needed it. So, they already had redundant code that could have been refactored to use iterations rather than repeated chunks of all-but-identical code. Given that there's, like, four dozen VPC endpoint services that can be configured, I especially didn't want to go down their prior "copy the existing code to a new block and edit one string" method-path.
I had a general idea of how I wanted to approach their problem. Not wanting to waste a ton of time reinventing a wheel that might be sitting in some dusty corner, I first reached out to a teammate to see if we'd done anything similar for other customers. My teammate pointed me to our terraform-aws-tardigrade-vpc-endpoints project. I looked over the code to see how much I could "steal" and apply to the project at hand.
Ultimately, I cut away all but the specific iteration-bits I needed and tested the code out. It worked well enough for what I wanted to accomplish for my customer. Unfortunately, the borrowed code was designed more for facilitating the setting up of a smaller list of VPC endpoints than seemed to be eh case for my customer. The borrowed code was more useful as a framework or reference for "how to make terraform iteratively create a list of VPC endpoints" than, "how to created an exhaustive list of endpoints across accounts, regions and/or VPCs". The former was easily used with an auto.tfvars type of file, but would have meant telling my current customer's Ops teams, "you need to create per account/region/VPC auto.tfvars files to be able to use this automation". I knew that wasn't going to go over well, so I opted to sort out how to read the service-definition content from a reusable template-file rather than an object-list variable. This way, I could have a file with contents similar to:
{
name = "com.amazonaws.${region}.ssm"
type = "Interface"
},
[…elided…]
And allow terraform to customize the behavior based on the region being executed within – Terraform doesn't like you to try to embed variable-references within the definition of another variable. Where the previous code looked like:
data "aws_vpc_endpoint_service" "this" {
for_each = { for service in var.vpc_endpoint_services : "${service.name}:${service.type}" => service }
service_name = length(regexall(data.aws_region.selected.name, each.value.name)) == 1 ? each.value.name : "com.amazonaws.${data.aws_region.selected.name}.${each.value.name}"
service_type = title(each.value.type)
}
The modification to use a template file – instead of the vpc_endpoint_services object-list variable – looked like:
data "aws_vpc_endpoint_service" "this" {
for_each = {
for service in jsondecode(
templatefile(
"./endpoint_services.tpl.hcl",
{
endpoint_region = var.region
}
)
) : "${service.name}:${service.type}" => service
}
service_name = length(
regexall(
var.region,
each.value.name
)
) == 1 ? each.value.name : "com.amazonaws.${var.region}.${each.value.name}"
service_type = title(each.value.type)
Terraform's templatefile() function is used to substitute the execution-region's value – supplied via other logic's setting of the region variable ‐ into each region-reference in the template file (e.g., for "s3" – a "Gateway" endpoint-type):
{
name = "com.amazonaws.${endpoint_region}.s3"
type = "Gateway"
},
or (e.g., for "ec2" – an "Interface" endpoint-type):
{
name = "com.amazonaws.${endpoint_region}.ec2"
type = "Interface"
},
It's worth noting that the template file is HCL-formatted. The execution logic doesn't really understand HCL – it prefers to parse JSON or YAML – requiring a small data-type conversion shim. The modified code-section's part of the shim is the use of Terraform's jsondecode() function. To convert the HCL to decodable JSON, the template file needs to encapsulate the HCL within the jsonencode() function. This requires the template file to look like:
${jsonencode(
[
{
name = "com.amazonaws.${endpoint_region}.access-analyzer"
type = "Interface"
},
[…elided…]
{
name = "com.amazonaws.${endpoint_region}.translate"
type = "Interface"
}
]
)}