emteeoh

Terraform supports several backends.

1. The default is a local file, which, of course, doesn't scale very well.
2. It also supports using AWS's S3 to store the state. This scales much better, but its rather finicky if you're not and AWS user, and instead using an S3 compatible object store such as DigitalOcean's Spaces. In fact, right now, I think it's utterly broken: I have configs that used to work that do not work with TF 1.6.
3. A third option is the HTTP backend. In short, this backend uses HTTP(S) to GET, POST, LOCK, and UNLOCK the state on a webserver. A single URL can be used, or you can use several URLs and exchange LOCK and UNLOCK for POST, or possibly other http request types. It seems very simple, but there's actually very sparse documentation about what is expected from the HTTP server in this case. This gist aims to fill in the details as I learn them writing my own HTTP server for this backend.

But first, why write my own server? There are many implementaitons of this backend out there. Why

emteeoh

My desktop runs Bullseye, and I haven't had the opportunity to do a proper backup before updating to Bookworm. As a result, I regularly run into tools I want to run that want a more recent version of this or that. Open-Interpreter, a CLI tool for ChatGPT, is one such tool. You can install it with python's pip, but it wants Python 3.10 or newer and Bullseye runs 3.9.

So first you have to download, build, and install python-3.10 or newer:

wget https://www.python.org/ftp/python/3.11.6/Python-3.11.6.tgz
tar zxvf Python-3.11.6.tgz
cd Python-3.11.6/
make -j $(grep -c processor /proc/cpuinfo) //run one make process per CPU thread

emteeoh

Don't put secrets in your Terraform files to be checked into GitHub!

I'm using Terraform with Digital Ocean from a linux desktop, but the same problem exists with other service providers: you need to provide some kind of secret for authentication, but you don't want the secret to be in your terraform files where they can accidentally be checked into revision control, thus shared with a thousand of your closest friends and hackers, not to mention search engines.

Put secrets into environment variables

Most Terraform tutorials and such suggest you put the secret into an environment variable, and then run terraform with something like "terraform plan -var "do_pat=${DO_PAT}" " That works, but now you need to add a parameter to terraform every time you run it. Probably not a big deal when you're using automated CI/CD-type stuff, but if you're running things manually, it's a pain.

Name your environment variables correctly so that they're automatically found by Terraform

If you dig just a little deeper