[DRAFT] Secret Management with Vault (Part 2).md

Lesson from part 1

Vault can protect us against:

• Eavesdropping on any Vault communication
• Tampering with data at rest or in transit
• Access to data or controls without authentication or authorization
• Access to data or controls without accountability
• Confidentiality of stored secrets
• Availability of secret material in the face of failure

Vault can't protect us against:

• Protecting against arbitrary control of the storage backend
• Protecting against the leakage of the existence of secret material
• Protecting against memory analysis of a running Vault

Quick Demo

https://www.vaultproject.io/intro/getting-started/install.html

Real world problem

Let's say we use a job scheduler (Nomad, Marathon, K8s, swarm?) to schedule and run our application, so our stack has job scheduler, the application, the database and Vault.

In order to use Vault in real application, there is a critical question that we need to answer: Should our application know about Vault?

My application know about Vault

Great, because your application know about Vault, it can query Vault and get whatever it wants. All it needs now it login certification. We can use app-id or app-role, which is the authentication methods for machine to login. We can hard code (:omg:) the app id in the application and get the token at runtime (maybe MAC address of the machien).

I don't want my application know about Vault

This is a much better approach, because we decouple our application with Vault. Now who will get the secrets for our application?

Because our application doesn't know about Vault, the job scheduler must know about it. But if we allow the scheduler get all the secrets for our application and pass it through file system or environment variable, the whole purpose of using Vault is defeated.

That's a bad idea. Let's try something else. How about having a small tool wrap your application, let's the job scheduler run that tool, it will communicate with Vault, get all tokens and pass it to your application (Picture needed!!!). This looks like a nice idea, the application doesn't need to know about Vault, all tokens is query in the place using it.

Drawbacks

Using two approaches above, we can integrate Vault into our current application but it comes with a few drawbacks.

1. Even if the application (or the wrapping script) knows about Vault, we still need to hard code the app id (role id) in order to let it login. This is a easy to solve problem, we can use Response Wrapping and let the scheduler pass it to the application. But the response wrapping can be only used once, so if we run more than one instance of the application, the second instance can not use it. So in order to use it, the job scheduler must give each instance a different response wrapping app id, which I think that all job schedulers can't do right now.

2. If we use second approve, beside the hard code app id, the wrapper tool itself is a problem.

I have a dream

To solve this problem completely, I think we need a more powerful job scheduler. After decided where the job will be running on, it will run a script to get Vault token, then pass it as a variable to the scheduler client on the node that will run our application. The scheduler client that unwrap the token, query all the secrets and pass it to the application. Everything is secured, our application doesn't need to know about Vault, no custom tool, no hardcode value.