trusted-broker.md

Demo: CI/CD with TFC + Vault AppRole

Recommended Pattern for Vault AppRole Use | Vault - HashiCorp Learn Terraform Cloud Secrets Engine | Vault - HashiCorp Learn https://registry.terraform.io/providers/hashicorp/vault/latest/docs/data-sources/generic_secret

This demo aims to demonstrate how a CI/CD tool like GitLab or Jenkins could be used to broker trust for Vault by providing role IDs and wrapped secret IDs for the "build job" to consume. You can find the described pattern in the documentation .

Create AppRoles

Vault needs to be configured to create the AppRoles needed.

Create a policy for the CI Controller:

cat <<EOF > ci-control-policy.hcl
# Allow CI to read from the secret KV store
path "secret/*" {
  capabilities = ["read", "list"]
}
# Allow CI to create wrapped secret-ids
path "auth/approle/role/app1/*" {
  capabilities = [ "create", "read", "update", "delete", "sudo" ]
}
EOF

vault policy write ci-control ci-control-policy.hcl

Configure the CI Controller AppRole:

vault auth enable approle
vault write auth/approle/role/ci-control \
    token_num_uses=0 \
    token_ttl=0 \
    token_max_ttl=0 \
    secret_id_num_uses=0 \
    token_policies="ci-control"

# Fetch the RoleId of the AppRole
vault read auth/approle/role/ci-control/role-id \
    --format=json | jq -r .data.role_id > ./ci-control-approle-role-id

# Get a SecretID issued against the AppRole
vault write -f auth/approle/role/ci-control/secret-id \
    --format=json | jq -r .data.secret_id > ./ci-control-approle-secret-id

Create a policy for the CI App1 build job:

cat <<EOF > app1_policy.hcl
# Allow app1 to retrieve secret from the secret KV store
path "secret/data/dev" {
  capabilities = ["read"]
}
path "terraform/creds/app1" {
  capabilities = [ "read" ]
}
EOF

vault policy write app1 app1_policy.hcl

Configure the CI App1 build job AppRole:

# Write some test data at secret/dev that the wrapped secretID will allow access to
vault secrets enable -path=secret kv-v2
vault kv put secret/dev username="webapp" password="my-long-password"

# Create AppRole for app1_policy
vault write auth/approle/role/app1 \
    secret_id_ttl=10m \
    token_num_uses=2 \
    token_ttl=20m \
    token_max_ttl=30m \
    secret_id_num_uses=1 \
    token_policies="app1"

TFC Secrets Engine

export TF_TOKEN=1bsIT..
vault secrets enable terraform
vault write terraform/config token=$TF_TOKEN

curl -s \
    --header "Authorization: Bearer $TF_TOKEN" \
    --header "Content-Type: application/vnd.api+json" \
    --request GET \
    https://app.terraform.io/api/v2/account/details | jq -r ".data.id"

USER_ID=$(!!)
vault write terraform/role/app1 user_id=$USER_ID ttl=60m

Pipeline Steps

CI Controller authenticates to Vault and Vault returns a token:

vault write auth/approle/login \
    role_id=$(cat ci-control-approle-role-id) \
    secret_id=$(cat ci-control-approle-secret-id) \
    --format=json | jq -r .auth.client_token > ./ci-control-token

Controller uses token to retrieve the RoleID of the CI job it will spawn:

export VAULT_TOKEN=$(cat ci-control-token)
vault read auth/approle/role/app1/role-id \
    --format=json | jq -r .data.role_id > ./app1-approle-role-id

Controller uses token to retrieve a wrapped SecretID:

export VAULT_TOKEN=$(cat ci-control-token)
vault write -wrap-ttl=60m -f auth/approle/role/app1/secret-id \
    --format=json | jq -r .wrap_info.token > ./wrapping-token

Controller spawns the CI build job and passes the wrapped SecretID as a variable to the job and the build job requests to unwrap the SecretID:

# Attempting to unwrap secret ID from wrapping token'
export VAULT_ADDR=http://127.0.0.1:8200
export VAULT_TOKEN=$(cat wrapping-token)
vault unwrap \
    --format=json | jq -r .data.secret_id > ./app1-approle-secret-id

Build job uses RoleID and SecretID to authenticate to Vault and Vault returns a token with policies that allow read of the required secrets:

vault write auth/approle/login \
    role_id=$(cat app1-approle-role-id) \
    secret_id=$(cat app1-approle-secret-id) \
    --format=json | jq -r .auth.client_token > ./app1-approle-token

The build job uses the token to get secrets from Vault:

VAULT_TOKEN=$(cat app1-approle-token) vault read secret/data/dev

The build job creates dynamic credentials to authenticate to TFC:

VAULT_TOKEN=$(cat app1-approle-token) vault read terraform/creds/app1

The build job is using the TF generic data source to retrieve other data from Vault https://registry.terraform.io/providers/hashicorp/vault/latest/docs/data-sources/generic_secret