Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
SSH with Vault

Managing SSH authentication with Vault

Managing SSH keys with Vault requires 3 steps:

  1. Setting up Vault
  2. Setting up the host
  3. Setting up the client / using the signed client keys

For a full documentation, see this HashiCorp Blog Post

(0) Challenges with ssh public key authentication

For a good article on this topic, read this blog post from Uber

  • basic idea: use public/private key pair
    • keep private part on client's machine
    • make public part known to host
  • challenges:
    • all public keys must be managed
      • hard to keep track on all of them
      • removal might lead to security risks (ex-employee still having access to machines...)
    • ideally, keys should expire after a certain amount of time
      • ssh keys do not expire
      • manually invalidating them is brittle
    • mitigation: 2-factor authentication, e.g. with
      • problem: inconvenient, users tend to carelessly handle 2FA if annoyed by it
  • solution:
    • use certification authority for SSH keys
      • inventory public keys
      • enable automatic expiration of SSH keys
      • improve host authentication

(1) Set up Vault

  1. mount a ssh secrets engine

    vault secrets enable -path=ssh ssh
  2. configure Vault with a CA for signing client keys using the /config/ca endpoint. If you do not have an internal CA, Vault can generate a keypair for you

    vault write ssh-client-signer/config/ca generate_signing_key=true
  3. create role for signing client keys

    vault write ssh/roles/my-role -<<"EOH"
    {
        "allow_user_certificates": true,
        "allowed_users": "*",
        "valid_principals": "vagrant",
        "default_extensions": [
            {
                "permit-pty": ""
            }
        ],
        "key_type": "ca",
        "default_user": "vagrant",
        "ttl": "30m0s"
    }
    EOH

(2) Set up the host

  1. (on the machine running Vault) Obtain the public key for the ssh key signing

    curl -o /etc/ssh/trusted-user-ca-keys.pem http://127.0.0.1:8200/v1/ssh-client-signer/public_key
  2. store the public key to a proper location on the target host (e.g. /etc/ssh/trusted-user-ca-keys.pem)

  3. add the following line to configure sshd to use the public key (vi /etc/ssh/sshd_config)

    TrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem
    
  4. restart sshd

    service sshd restart

(3) Using the signed keys

  1. Signing a key with Vault

    vault write -field=signed_key ssh/sign/my-role public_key=@$HOME/.ssh/id_rsa.pub > signed-cert.pub
  2. Log in to the remote host via

    ssh -i signed-cert.pub -i ~/.ssh/id_rsa <USER>@<HOST>

Convenience wrapper for ssh with signed keys

  1. add the following function to your ~/.bashrc

    sshv () {
       vault write -field=signed_key ssh/sign/my-role public_key=@$HOME/.ssh/id_rsa.pub > /tmp/${1}-signed-cert.pub
       ssh -i /tmp/${1}-signed-cert.pub -i ~/.ssh/id_rsa ${1}
    }
  2. ssh to your host via

    sshv <USER>@<HOST>

Debugging

SSH issues

  • on the host, check the ssh logs via

    tail -f /var/log/auth.log

    on Centos, the logs are in /var/log/secure

  • on the client, add -vvv to your ssh command

    ssh -i signed-cert.pub -i ~/.ssh/id_rsa <USER>@<HOST> -vvv
  • common errors

    Dec 15 11:03:42 ipa sshd[2144]: error: Certificate invalid: name is not a listed principal
    Dec 15 11:17:01 ipa CRON[2154]: pam_unix(cron:session): session opened for user root by (uid=0)
    Dec 15 11:17:01 ipa CRON[2154]: pam_unix(cron:session): session closed for user root
    Dec 15 11:29:59 ipa sshd[2165]: error: Certificate invalid: not yet valid
    Dec 15 11:30:13 ipa sshd[2165]: Connection closed by authenticating user vagrant 192.168.33.1 port 51154 [preauth]
    Dec 15 11:30:14 ipa sshd[2167]: error: Certificate invalid: not yet valid
    
  • error: Certificate invalid: name is not a listed principal: add allowed_principals to Vagrant role (check above)

  • error: Certificate invalid: not yet valid: fix date with

    sudo date --set "14 Dec 2018 12:35:00"

Check contents of signed key

ssh-keygen -Lf /tmp/ipa-dev-signed-cert.pub

Check Vault roles

vault read ssh/roles/<ROLE NAME>

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.