- Cloud deployment problem: Where do we store our secrets (read: database username and password) and how to give them to the server when we deploy?
- Secret sharing and auditing problem: If everybody in our team uses the same token, how can we audit when something wrong happend? Who is the last person logged in using the shared key?
- Revocation problem: our ex-sysadmin have quitted, how we managed to remove his public key from all 50 production servers? How about our database username and password, AWS token? Are we sure our production servers don't use the same key so we can remove it without bombing production environment.
- Break glass procedure: We know that our production database key is public somewhere in the internet right now, what should we do? Should we turn off the production database now? Maybe someone hacks our teammate computer, maybe the hacker knows every secrets we have, should we turn off everything? Oh my god!!! What should we do now??? :crying:
This is where Vault comes up to save us all!
Vault is a secret management tool which secures, stores, and tightly controls access all kind of secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing.
- Vault needs storage backend to store encrypted data.
- Vault has multiple secret backend to store or generate dynamic backend.
- Vault has multiple authentication backend to allow human and machine authenticate with Vault
- Each authenticated member has a set of policy and they can only request token in their policy
- Vault can store our existing secret, dynamic generate new secrets to control access to third-party resources or provide time-limited credentials for our infrastructure
- All data is encrypted and stores on a different storage backend outside of Vault.
- All dynamic generated secret has a lease time, when lease time expires, the key will be revoked
- Vault stores a detailed audit log of all authenticated client interaction: authentication, token creation, secret access, secret revocation, and more. Paired with Vault's strict leasing policies, we can easily trace the lifetime and origin of any secret.
- Response wrapping: instead of return the real secret for us, Vault can return a one time token that we can use to get the real secret. This token has default lease time about 15 second. So the attacker only have a very small windows to steal it, and if he/she manage to steal it, we will know because the production server can not use that token anymore.
- Vault can generate one time token or even allows machine to authenticate with Vault and get the token it needs. This makes sure that if someone try to steal the key somewhere between our CI and production server, we can know about it.
- Vault can generate dynamic token so every team member will have different key to access the same resource. Different production server will have different keys as well.
- If we need to we can revoke one specific token or remove membership of one person, then all their keys will be revoked as well.
- Every action to Vault will be recorded to audit log so if something go wrong, we can always track it back, so build automated system to detect suspicious action.
- Last but not least, if something go wrong (unauthorized access to Vault and we don't have time to track audit log now), we can revoke suspicious token, or revoke everything to minimize the damage and seal Vaul. After that, no one can access Vault again until we unseal it. Now audit log will be our best friend to see what's going wrong.
Vault can not protect us if:
- Someone tries to destroy the storage backend, no secret is leaked but all data is lost :troll:.
- Someone has access to Vault server and is able to inspect the memory of Vault.
- Someone has access to production server and try to see the secret on that server (in a file or memory, wherever the server store secret after retrieving it from Vault).
Most of the pros has covered above but because of security nature, there are some technical issues need to be covered when using Vault, some of you may consider it as cons. We will setup a Vault demo next week and see some of technical decisions we need to make before using Vault.