As your Terraform codebase grows in size and reach, you might find yourself in one of these common scenarios:
- enabling contributions from specific teams (e.g. NetOps to create new subnets or firewall rules, or the Logging and Monitoring team to configure new alerts) can be difficult since Terraform knowledge is not widespread across teams
- the repetitive creation of resources of a given kind (e.g. firewall rules, subnets, projects) is scattered throughout your codebase, making it complex to get a clear picture of their structure
- sticking to a monolithic, end-to-end approach for a large infrastructure (i.e. describing your whole infrastructure in a large terraform module/state) makes for a slower, less maintainable codebase, which doesn’t mirror the different speeds at which your infra evolves (e.g. core infrastructure vs firewall rules)
This article proposes a configuration-based approach that enables parceling out the repetitive creation of specific resources in distinct repositories and enabling non-Terraform-enabled teams to contribute to your IaC.
Let's see an example of how we would handle the creation of multiple resources of the same kind using solutions within the traditional spectrum.
Naive definition using the provider’s built-in resources. Doesn’t solve the challenge of enabling different teams to contribute to the codebase, can creep into long and hardly readable code walls when resource count gets into the tens or the hundreds and is generally unsuitable for production usage.
Wrap logic in a Terraform module and leverage Terraform variables. This approach allows for the separation of code and configuration, allowing non-core teams to contribute code to a specific repository, and providing them a much easier interface to work with (configuration vs Terraform code).
However as resource count grows, your variables rapidly become harder to parse and make sense of, and a single typo can lead to hard-to-troubleshoot mistakes.
Wrap logic in a Terraform resource factory, and leverage YaML/JSON configurations, one per resource. This is a battle-tested approach we are using on large and complex codebases.
A resource factory is an opinionated, purpose-built module that
- implements specific requirements and best practices (e.g. “always enable PGA for GCP subnets”, or “only allow using regions europe-west1 and europe-west3”)
- codifies business logics and policies (e.g. labels and naming conventions)
- standardizes, automates and centralizes the repetitive creation of resources of a given kind
Our approach is based on modules implementing the factory logic using Terraform code, and a set of directories having a well-defined, semantic structure, holding the configuration for the resources in YaML syntax.
Terraform natively supports YaML, JSON and CSV parsing - a few observations:
- JSON and CSV can’t include comments, which can be used to document configurations
- JSON and CSV are often useful to bridge from other systems in automated pipelines
- JSON is more verbose (reads: longer) and harder to parse for a human
- CSV isn’t often expressive enough (e.g. doesn’t allow for nested structures)
Tl;dr, if you’re editing files by hand, YaML is probably your best choice, but you can also consume JSON or CSV files coming from your automations
With that in mind, let's see an example of this approach using the subnet module available in the Cloud Foundation Fabric Resource Factories modules
The module is conceived so that it can be used by simply pointing it to our configuration folder
module "subnets" {
source = "./modules/resource-factories/subnet"
config_folder = "./subnets"
}... it uses a fixed data directory structure
└── subnets # Configuration folder entry point
├── project-ada # Project ID the VPC belongs to
│ ├── vpc-alpha # VPC name
│ │ ├── subnet-a.yaml # Subnet name
│ │ └── subnet-b.yaml
│ └── vpc-beta
│ └── subnet-c.yaml
└── project-bob
└── vpc-gamma
└── subnet-d.yaml...and YaML files for the actual configuration, one file per subnet
# subnets/project-ada/vpc-alpha/subnet-a.yaml
region: europe-west1
description: Frontend
ip_cidr_range: 10.0.0.0/24
secondary_ip_ranges:
- secondary-range-a: 192.168.0.0/24# subnets/project-ada/vpc-alpha/subnet-b.yaml
region: europe-west1
description: Backend
ip_cidr_range: 10.0.1.0/24
# Assign roles/compute.networkUser
iam_groups: ["backend-admins@example.com"] The folder structure supplies many of the required attributes for each subnet (project id, VPC name, subnet name), reducing the potential for mistakes, and making the network structure explicit and easy to navigate even from a repository browser view, while the YaML file structure is essential and easy to understand and maintain for non-terraform-savvy operators, enabling them to contribute to your IaC codebase.
GitOps strategies can then be implemented to review every suitable commit for correctness (e.g. YaML syntax, terraform plan outputs) and compliance (e.g. allow only certain users/groups to commit under given folders), and to finally terraform apply the resulting infrastructure updates.
This is just an example of how resource factories can be handy when managing large and complex infrastructures. Besides subnets, our code repository has a growing number of other factories (hierarchical firewall policies, vpc firewall rules) you can take inspiration from, or directly grab and adapt to your needs.