Let's talk Terraform

genesis.md

I have used terraform almost daily for the past 5 years and during this time I have developed some ideas on how to use it.

What to use.
What to not use.
Why.

This is a series of posts meant to share how I'm using terraform in 2021.

Note: while some of the content is directly taken from my experience, some of the examples are just created to deliver the point of the post. They might look exaggerated and sometimes they are. Feel free to reach out to me @lucalanziani if you would like to talk about this topic. ;).

Premise

The first time I looked at terraform, 5 years ago now, it all looked quite straightforward

1. you write some .tf files with the resources you want to provision:

resource "aws_instance" "iac_in_action" {
  ami               = var.ami_id
  instance_type     = var.instance_type
  availability_zone = var.availability_zone

  // dynamically retrieve SSH Key Name
  key_name = aws_key_pair.iac_in_action.key_name

  // dynamically set Security Group ID (firewall)
  vpc_security_group_ids = [aws_security_group.iac_in_action.id]

  tags = {
    Name = "Terraform-managed EC2 Instance for IaC in Action"
  }
}

2. You move into the folder containing the files
3. You run a couple of commands

terraform plan
terraform apply

4. ZAP your environment is ready...

or so I thought 😂...

Obviously as usual that couldn't be further from the truth. What you get presented, even nowadays, from the terraform.io website is an oversimplification of what's needed to manage your infrastructure with Terraform.

Don't get me wrong I liked Terraform back then and I still like it today, especially given all the improvements Hashicorp has pushed and still pushing.

But, what I want to do with this series of blog posts is to share what I learned about terraform during the years and maybe leave you with something useful.

The idea is to address 4 main steps (plus a special one if I get around it):

• Manage Infrastructure growth
• Reduce duplication
• Module's dilemma
• Divide and conquer

Setting the stage

It all started with a simple Jenkins server, it always does, doesn't it?

Easy peasy, I said:

An ec2 instance here
An ELB there
Ah, I need a VPC, right...
And networking...
And NACL and SGs...
Do I need multi AZs? How I forgot about the EBS volume?
Wait volume_attachement? ok 🤷
AH! The aim role... obviously
And the S3 bucket
And the provider, with the correct region
Remote state... I need first an S3 bucket for that state

I suppose you get my point right? Writing terraform is (was?) not the fun part.

After some nice headaches at least I had my stack ready and I was able to deploy and destroy reliably multiple times, that's If terraform wasn't failing in the middle because of some provider bug or I wasn't corrupting the state with a CTRL^C at the wrong time 😂.

But hey we are talking about 5 years ago, terraform 0.7.x is what we were using and we felt pretty happy about it.

Then the second request came in, let's build another Jenkins!!!

And the path started...

If you want to talk about the content of this post or if you want to know when the next post is coming out, you can find me on Twitter @lucalanziani.

See you in the next post.

Multi_environment_multi_instance.md

title	date	category	tags	email
Let's talk terraform	2021-04-26 21:39:00 +0200	general, terraform, devops	devops terraform	luca@lanziani.com

I have used terraform almost daily for the past 5 years and during this time I have developed some ideas on how to use it.

What to use.
What to not use.
Why.

This is a series of posts meant to share how I'm using terraform in 2021.

Note: while some of the content is directly taken from my experience, some of the examples are just created to deliver the point of the post. They might look exaggerated and sometimes they are. Feel free to reach out to me @lucalanziani if you would like to talk about this topic. ;).

Multiple instances of the same stack in multiple environments

It looks obvious to me that you would want to use the same terraform stack on multiple environments, that's because you want reproducibility and consistency, right?

When I first started using terraform I don't believe there was anything provided by terraform to solve this problem, the solution that my team found was to write a "simple" bash script that would build a terraform command like this:

terraform plan \
    -var-file="${config_dir}/common.tfvars.json" \
    -var-file="${config_dir}/${env}/common.tfvars.json" \
    -var-file="${config_dir}/${env}/${region}/common.tfvars.json" \
    -var-file="${config_dir}/${env}/${region}/${component}/common.tfvars.json" \
    -var-file="${config_dir}/${env}/${region}/${component}/${instance}/config.tfvars.json" \
    ...

Every new component and every instance of the component would have to follow this pattern when defining variables so we could use the same pattern everywhere.

It soon became evident that this method wasn't going to scale, we were building a significant sized infrastructure and it wasn't so much the number of components to be a problem, it was more the instances of the same components.

56 directories, 57 files

terraform-configs > tree
.
├── common.tfvars.json
├── dev
│   ├── ap-southeast-2
│   │   ├── application
│   │   │   ├── common.tfvars.json
│   │   │   ├── test1
│   │   │   │   └── config.tfvars.json
│   │   │   ├── test2
│   │   │   │   └── config.tfvars.json
│   │   ├── common.tfvars.json
│   │   ├── logging
│   │   │   ├── common.tfvars.json
│   │   │   └── default
│   │   │       └── config.tfvars.json
│   │   └── monitoring
│   │       ├── common.tfvars.json
│   │       └── default
│   │           └── config.tfvars.json
│   ├── common.tfvars.json
│   ├── eu-west-1
│   │   ├── application
│   │   │   ├── common.tfvars.json
│   │   │   ├── test1
│   │   │   │   └── config.tfvars.json
│   │   │   ├── test2
│   │   ├── common.tfvars.json
│   │   ├── logging
│   │   │   ├── common.tfvars.json
│   │   │   └── default
│   │   │       └── config.tfvars.json
│   │   └── monitoring
│   │       ├── common.tfvars.json
│   │       └── default
│   │           └── config.tfvars.json
│   └── us-east-1
│       ├── application
...
│       ├── logging
...
│       └── monitoring
└── prd
    ├── ap-southeast-2
    │   ├── application
    │   │   ├── common.tfvars.json
    │   │   ├── test1
    │   │   │   └── config.tfvars.json
    │   │   ├── test2
    │   │   │   └── config.tfvars.json
    │   │   └── test3
    │   │       └── config.tfvars.json
    │   ├── common.tfvars.json
    │   ├── logging
    │   │   ├── common.tfvars.json
    │   │   └── default
    │   │       └── config.tfvars.json
    │   └── monitoring
    │       ├── common.tfvars.json
    │       └── default
    │           └── config.tfvars.json
    ├── common.tfvars.json
    ├── eu-west-1
    │   ├── application
...
    │   ├── logging
...
    │   └── monitoring
...
    └── us-east-1
...

Imagine having to maintain all similar files in sync and having to propagate changes per environment in case of:

• variable renaming
• variable addition or removal
• component split

Of course, these weren't the only issues that this model was posing:

1. how to init the remote state for each stack using a different S3 path
2. how to know which version of the stack was was compatible with variables in each stack/environment
3. how to know how many components and how many instances of the same were present
4. how to know if a stack was dependant on another
5. support of a custom bash script
6. which version of terraform was ok to run on each stack/environment

Enter Terragrunt

Thanks Gruntwork.io 🙇

Looking at the first two terragrunt commits this is what you get:

commit 493637bc77831a119f3083b9a45f6275d5792fd7
Author: Yevgeniy Brikman <brikis98@gmail.com>
Date:   Tue May 24 00:18:06 2016 +0200

    Initial commit

commit 9e266bebe67d7f659fce5702d0005cec4c1feb77
Author: Yevgeniy Brikman <brikis98@gmail.com>
Date:   Wed May 25 17:01:02 2016 +0200

    RDD: Create Readme to describe how terragrunt will work

We probably started using terragrunt the year after that, I vaguely remember using terraform 0.8.x at the time and that would place it around Feb 2017.

The tool brought a breath of fresh air to the team, we could finally use a more appropriate configuration to manage everything, replacing the instance variables with a terragrunt.hcl file.

56 directories, 57 files

terragrunt-config > tree
.
├── common.tfvars.json
├── dev
│   ├── ap-southeast-2
│   │   ├── application
│   │   │   ├── common.tfvars
│   │   │   ├── test1
│   │   │   │   └── terragrunt.hcl
│   │   │   ├── test2
│   │   │   │   └── terragrunt.hcl
│   │   │   └── test3
│   │   │       └── terragrunt.hcl
│   │   ├── common.tfvars
│   │   ├── logging
│   │   │   ├── common.tfvars
│   │   │   └── default
│   │   │       └── terragrunt.hcl
│   │   └── monitoring
│   │       ├── common.tfvars
│   │       └── default
│   │           └── terragrunt.hcl
│   ├── eu-west-1
│   │   ├── application
│   │   │   ├── common.tfvars
│   │   │   ├── test1
│   │   │   │   └── terragrunt.hcl
...
│   │   ├── logging
...
│   │   └── monitoring
...
│   ├── terragrunt.hcl
│   └── us-east-1
...
└── prd
    ├── ap-southeast-2
    │   ├── application
    │   │   ├── common.tfvars
    │   │   ├── test1
    │   │   │   └── terragrunt.hcl
    │   │   ├── test2
    │   │   │   └── terragrunt.hcl
    │   │   └── test3
    │   │       └── terragrunt.hcl
    │   ├── common.tfvars
    │   ├── logging
    │   │   ├── common.tfvars
    │   │   └── default
    │   │       └── terragrunt.hcl
    │   └── monitoring
    │       ├── common.tfvars
    │       └── default
    │           └── terragrunt.hcl
    ├── eu-west-1
...
    └── us-east-1
...

The environment /<env>/terragrunt.hcl file would contain something like:

terragrunt = {  
  remote_state {
    backend = "s3"
    config {
      encrypt = "true"
      bucket = "my-bucket"
      key = "${path_relative_to_include()}/terraform.tfstate"
      region = "eu-west-1"
    }
  }
}

And each instance of each component will have something like:

terragrunt = {
  terraform {
    source = "git::git@github.com:<org>/<repo>//<stack-folder>?ref=<git-tag>"
  }

  include {
    path = find_in_parent_folders()
  }

  dependencies {
    paths = ["../dep1", "../dep2", "../dep3"]
  }
}

This setup solved the problems 1, 2, 4 and 5. Variables were imported from relative paths and we could get rid of the bash tool.

We still had the issue of maintaining the files in sync and propagate changes between environments.

This is exactly the problem we are going to solve in the next post of the "Let's talk terraform" series, be sure not to miss it ;)

If you want to talk about the content of this post or if you want to know when the next post is coming out, you can find me on Twitter @lucalanziani