IAM auth to Hashicorp Vault server from an ECS container

get_vault_secret.py

#!/usr/bin/env python3
import base64
import json

import requests
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth

"""
This code will connect from an ECS container to a remote Hashicorp Vault server
and authenticate using the 'iam' auth_type for the AWS auth backend.

To authenticate to the Vault server, we have to have a signed request to STS
GetCallerIdentity. We take advantage of BotoAWSRequestsAuth to avoid having to
do this ourselves. This will use the ECS container's credentials, which botocore
can find because ECS sets the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment
variable for it. These credentials correspond to the ECS task role.

We send Vault all the necessary information (including signed headers) to
reconstruct the request, which it sends. It then verifies that the returned
caller identity corresponds to the bound_iam_principal_arn in the target Vault
role.

Required Python packages:
  * requests (http://docs.python-requests.org/)
  * aws-requests-auth (https://github.com/DavidMuller/aws-requests-auth)

Other useful docs if you want details:
  * https://www.vaultproject.io/docs/auth/aws.html#iam-authentication-method
  * http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
"""

VAULT_BASE_URL = 'https://vault.mycompany.com:8200/v1/'
AWS_REGION = 'us-east-1'

# You could get the region by stripping the last character (AZ) from
# the output of http://169.254.169.254/latest/meta-data/placement/availability-zone


def _encode_for_payload(bytestring):
    """base64 encode supplied string and ensure it's JSON-serializable."""
    return base64.b64encode(bytestring).decode()


print('Retrieving authentication information from Vault...')
# Construct, but do not send, signed request for STS GetCallerIdentity
# We need the signed headers to use IAM auth to Vault
# Boto knows how to get credentials from the ECS host; these correspond to your ECS task IAM role
auth = BotoAWSRequestsAuth(
    aws_host='sts.amazonaws.com',
    aws_region=AWS_REGION,
    aws_service='sts')
headers = {
    'Accept-Encoding': 'identity',
    'Content-Type': 'application/x-www-form-urlencoded'
}
req = requests.Request('POST', 'https://sts.amazonaws.com/', headers=headers, auth=auth,
                       data='Action=GetCallerIdentity&Version=2011-06-15').prepare()

# Now we can use the headers from this prepared request for our Vault auth
# The Vault role should have the ECS task role set as its bound_iam_principal_arn
payload = {
    'role': 'my_vault_role',
    'iam_http_request_method': 'POST',
    'iam_request_url': _encode_for_payload(b'https://sts.amazonaws.com'),
    'iam_request_body': _encode_for_payload(b'Action=GetCallerIdentity&Version=2011-06-15'),
    'iam_request_headers': _encode_for_payload(json.dumps(dict(req.headers)).encode('ascii'))
}
# Get Vault auth token
r = requests.post(VAULT_BASE_URL+'auth/aws/login', json=payload)
try:
    r.raise_for_status()
except requests.HTTPError as e:
    print("Vault login failed: {}".format(r.text))
    raise

headers = {'X-Vault-Token': r.json()['auth']['client_token']}
secretpath = '/secret/path/in/vault'
r = requests.get(VAULT_BASE_URL+secretpath, headers=headers)

try:
    r.raise_for_status()
except requests.HTTPError:
    print("Failed to read secrets from Vault. {}".format(r.text))
    raise
finally:
    requests.post(VAULT_BASE_URL+'auth/token/revoke-self', headers=headers)

secrets = r.json()['data']  # K/V pairs as stored in Vault
# Do something with secrets here

print('Successfully retrieved from Vault.')

Unfortunately aws-requests-auth library doesn't support customization of SignedHeaders, thus iam_server_id_header_value as well

Use this code instead - https://gist.github.com/joelthompson/378cbe449d541debf771f5a6a171c5ed#file-vault_aws_auth_py3-py