Demo: Terraform external data soruce

README.md

Demo: Terraform external data source

Prerequisite: Nix or install Terraform by hand.

$ nix run nixpkgs#terraform init
[...]

$ nix run nixpkgs#terraform plan
[...]
Changes to Outputs:
  + foo = {
      + bar = {}
      + foo = [
          + 1,
          + 2,
          + 3,
        ]
    }

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

main.tf

data "external" "foo" {
  # Outputs `Dict[str, Path]` on stdout. E.g. `{"stuff": "processed-data.json"}`
  # You can construct `processed-data.json` however you like: from HTTP APIs,
  # databases, or by processing some other configuration file.
  #
  # This works around a Terraform limitation: external data sources cannot output
  # arbitrary JSON. But if they output a file path, we can use `jsondecode()`.
  program = ["${path.module}/process.py"]
}

output "foo" {
  value = jsondecode(file(data.external.foo.result["stuff"]))
}

process.py

#!/usr/bin/env python3
import json
from pathlib import Path


# Can also be in a tempdir, etc.
outfile = Path("processed-data.json")
outfile.write_text(json.dumps({"foo": [1, 2, 3], "bar": {}}))

print(json.dumps({"stuff": str(outfile)}))