k8s sidecar vault workflow

$ Vault k8s sidecar

- [article](https://learn.hashicorp.com/vault/identity-access-management/vault-agent-k8s)
- [code](https://github.com/hashicorp/vault-guides/tree/master/identity/vault-agent-k8s-demo)
- [RFC vault agent template](https://docs.google.com/document/d/1TBE5TuzgXpTBq2gGaJLd9gjWd1KW1MfXm2AUEIvFJtY/edit)
- [RFC Vault Kubernetes Admissions Webhook](https://docs.google.com/document/d/1nEaJiH_WO3SaHU178-zHRvz1Ic4m5q6ofbJJYxOV0X4/edit) mutate pod specs to add sidecar which will auth/auto renew and write secrets to a shared in-memory volume. Will live in a new binary named vault-k8s similar to consul-k8s.
- Above is using [Kubernetes Admission Webhooks available in 1.9](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) configured using annotations.

# Example Scripting

    cd /Users/sebbraun/in/code/vault/k8s

# mirror example code

    git clone https://github.com/hashicorp/vault-guides.git
    cd 
    cd vault-guides/identity/vault-agent-k8s-demo/
    
# workflow

1. create service account
2. associate required RBAC policy
3. create Vault policy to allow secret access
4. mount and create a kv secret v1
5. create userpass user with above policy
6. test user can read secret
7. set env variables & configure k8s auth method
8. create a role to map k8s service account to policies
9. test using vault image

# Create and update a service account, 'vault-auth'

    kubectl create serviceaccount vault-auth
    kubectl apply --filename vault-auth-service-account.yml

# Create a policy  

    # Create a policy file, myapp-kv-ro.hcl
    $ tee myapp-kv-ro.hcl <<EOF
    # If working with K/V v1
    path "secret/myapp/*" {
        capabilities = ["read", "list"]
    }

    # If working with K/V v2
    path "secret/data/myapp/*" {
        capabilities = ["read", "list"]
    }
    EOF

# Create some secret 

    vault kv put secret/myapp/config username='appuser' \
        password='suP3rsec(et!' \
        ttl='30s'

In my case

    vault kv put kv/k8s-secret/config username='appuser' \
        password='suP3rsec(et!' \
        ttl='30s'

# Configure Kubernetes auth method

    export VAULT_SA_NAME=$(kubectl get sa vault-auth -o jsonpath="{.secrets[*]['name']}")
    export SA_JWT_TOKEN=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data.token}" | base64 --decode; echo)
    export SA_CA_CRT=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo)

How to communicate with k8s cluster

    vault write auth/kubernetes/config \
        token_reviewer_jwt="$SA_JWT_TOKEN" \
        kubernetes_host="https://<FQDN_K8s_API>:8443" \
        kubernetes_ca_cert="$SA_CA_CRT"

Create a role which map Service Account to Vault policies and TTL

    vault write auth/kubernetes/role/example \
        bound_service_account_names=vault-auth \
        bound_service_account_namespaces=default \
        policies=myapp-kv-ro \
        ttl=24h

# Testing k8s auth.

    kubectl run test --rm -i --tty \
    --env="VAULT_ADDR=https://v1.prod.yet.org:8200" \
    --image alpine:3.7 -- /bin/sh
    
    apk update
    apk add curl jq
    curl -s $VAULT_ADDR/v1/sys/health | jq
    JWT=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
    curl --request POST \
        --data '{"jwt": "'"$JWT"'", "role": "k8s-role"}' \
        $VAULT_ADDR/v1/auth/kubernetes/login | jq

# Create example

    cd ~/in/code/hashicorp/vault-guides/identity/vault-agent-k8s-demo/

    kubectl create configmap example-vault-agent-config-gcp --from-file=./configs-k8s/
    kubectl get configmap example-vault-agent-config-gcp -o yaml

Update Vault API URL in `example-k8s-spec.yml` and provision the pod

    kubectl apply -f example-k8s-spec-gcp.yml

# Debug

uncomment debug in manifest

    args:
        [
          "agent",
          "-config=/etc/vault/vault-agent-config-gcp.hcl",
          "-log-level=debug",
        ]

    kubectl logs vault-agent-example consul-template
    kubectl get pod vault-agent-example --template '{{.status.initContainerStatuses}}'
    kubectl logs vault-agent-example -c vault-agent-auth

# Check

    kubectl port-forward pod/vault-agent-example 8080:80

Access http://localhost:8080