TWL-Lab49/Learning IaC - Relative Path in Terragrunt

lac-relative-path-terragrunt.md

One of the gotchas mentioned in Terragrunt doc is about Relative File Path. To summarize, this works:

terraform {
  source = "git::git@github.com:foo/modules.git//frontend-app?ref=v0.0.3"

  extra_arguments "custom_vars" {
    commands = [
      "apply",
      "plan",
      "import",
      "push",
      "refresh"
    ]

    # With the get_terragrunt_dir() function, you can use relative paths!
    arguments = [
      "-var-file=${get_terragrunt_dir()}/../common.tfvars"
    ]
  }
}

But this doesn't:

terraform {
  source = "git::git@github.com:foo/modules.git//frontend-app?ref=v0.0.3"

  extra_arguments "custom_vars" {
    commands = [
      "apply",
      "plan",
      "import",
      "push",
      "refresh"
    ]

    arguments = [
      "-var-file=../common.tfvars" # Note: This relative path will NOT work correctly!
    ]
  }
}

Terragrunt doc explains very well why relative path doesn't work:

Note how the source parameter is set, so Terragrunt will download the frontend-app code from the modules repo into a temporary folder and run terraform in that temporary folder. Note also that there is an extra_arguments block that is trying to allow the frontend-app to read some shared variables from a common.tfvars file. Unfortunately, the relative path (../common.tfvars) won’t work, as it will be relative to the temporary folder! Moreover, you can’t use an absolute path, or the code won’t work on any of your teammates’ computers.

However, it does not stress why relative path works in following code:

include "root" {
  path = find_in_parent_folders()
}

terraform {
  source = "github.com/<org>/modules.git//app?ref=v0.1.0"
}

dependency "vpc" {
  config_path = "../vpc"
}

dependency "mysql" {
  config_path = "../mysql"
}

inputs = {
  env            = "qa"
  basename       = "example-app"
  vpc_id         = dependency.vpc.outputs.vpc_id
  subnet_ids     = dependency.vpc.outputs.subnet_ids
  mysql_endpoint = dependency.mysql.outputs.endpoint
}

Notice the config_path "../vpc" and "../mysql"; they are relative paths but they are working. Why? I can't find clear documentation on how Terragrunt runs under the hood, nor can I google any useful information from other sources but after reading through all the use cases, I can share my understanding (Please point out where I'm wrong :)).

Essentially, Terragrunt is a thin layer that generates configurations (e.g., inputs, remote state etc.) for Terraform modules. It breaks down into two stages. First stage is generate all the necessary .tf files or merge all the includes (e.g. find from parents) into one final .hcl file or wait for all dependencies to resolve - we can think of this stage as "compilation"; second stage is copy all the generated or downloaded .tf files into a folder directory and execute whatever Terraform command we specified in that temporary folder - we can think of this stage as "runtime".

This becomes clear now. dependency happens in compilation which is run by Terragrunt in the source folder which is why relative path works. But anything used by running Terraform command at runtime is based on the temporary folder which is why it must call get_terragrunt_dir to get the absolute path to the source file during compilation time.

Again, this is just my understanding which might be wrong, and we have to resort to Terragrunt source code to get how it works exactly under the hood which is a long shot.

Happy learning:)