This post/message in response to https://twitter.com/trusted47/status/1523816335221956609
I was required to use Terragrunt for a while by a well-meaning company policy. I did put a lot of time and effort into trying to make it work well for us though and definitely agree that it solves some problems, but in my experience it introduced more complexity than it solved.
I'll start by saying that my usage of Terraform has been at larger companies where dev teams own their services end-to-end and manage most of their own infrastructure (Some companies using single/few AWS accounts might centrally manage VPCs/DNS, but teams with their own AWS accounts might manage everything other than audit logging/security checks). In any case, I can't talk to a central operations team managing infrastructure for everyone. As we both noted, the actual use-case and requirements make a big difference for what is going to work for any single team/company.
-
Strongly prefer using human readable variable values with data resources for dependencies. TF remote state and TG dependencies both caused more problems than they solved in my experience.
-
Not sure I understand this question - are you asking about how we keep different Terraform projects across the company similar? I've used a standard Terraform repo template that everyone uses at some places, but at others it hasn't really been an issue for teams to use their own standards (within reason - secuity checks and infra requirments were handled in other ways in those scenarios).
-
No global providers; Providers are on a project by project basis and determined by the team managing that infra. Environment specific provider settings like the region are configured as a variable and managed through tfvars.
-
Done this a few different ways - GitLab has a really great Terraform state management solution where every repo gets it's own state storage and you can dynamically generate new states for every environment you want to support. Terraform Enterprise is also really nice for managing Terraform state and provides great authorization management for determining who is allowed to do what. Also have used straight s3 state storage with a required CI template that overrode/determined the state location based on the project to keep state locations unique (no local deployments to non-dev environments).
-
Some global configs based on Terraform repo templates that everyone starts with, but haven't really dealt with many global configs.
-
Local handling of -var-file flags with Makefile commands. For non-dev environments Terraform Enterprise allows you to set these varaibles at the workspace level which is really nice. If using CI for deployment, then -var-file flags can be handled with standard CI template.
A few of my issues with TG:
- I think my biggest issue with TG is the suggested repo structure (environment/{module-name}/terragrunt.hcl) ends up with a lot of duplication between those terragrunt.hcl files and you have to remember to keep any changes in sync across all environments. There isn't any requirement to structure TG that way, so you can use a single terragrunt.hcl file and pass in tfvar files just like with TF, but at that point you just have a large complex repo where a lot of that infrastructure probably isn't changing regularly (vpcs, dns, sgs, dbs...). I know one goal for TG is to keep your modules and the blast radius for those modules small, but I find that taking that a step further to the actual Terraform repos is even more helpful. Why touch a repo where the VPCs exist when you might only want to deploy a new version of your serverless app.
- It feels more difficult to do state moves/refactoring with TG especially since the state location is usually based on the file structure of the repo.
- With TG you end up deciding whether you want to make your CI a lot more complex to handle TG runs for certain directories only when they change or if you want to keep your CI simpler, but do TG runs over a larger group of modules on every CI run.