Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

Lab 1: Lab Setup

Duration: 10 minutes

Each student should have received the lab workstation log in information from the instructor. This lab ensures that everyone can connect to the workstation, and verify that a Vault server is running so that vault commands can run against it.

  • Task 1: Connect to the Student Workstation
  • Task 2: Getting Help
  • Task 3: Enable Audit Logging
  • Task 4: Access Vault UI

Task 1: Connect to the Student Workstation

Step 1.1.1

SSH into your workstation using the provided credentials.

$ ssh <username>@<workstation_IP_address>
password: <password>

NOTE: Depending on your machine setting, you may need to explicitly set the PubKeyAuthentication to false:

$ ssh -o PubKeyAuthentication=false -l training <workstation_public_IP_address>

When you are prompted, enter "yes" to continue connecting.

On a Windows, use SSH client such as PuTTY. On a Linux or Mac, use the Terminal to SSH into your workstation.

Alternatively, launch a web browser and enter:

http://<workstation_IP_address>/wetty/ssh/<username>

When a security error is presented, accept and proceed. Depending on the web browser, this page has a slightly different navigation. The below is an example of Google Chrome:

wetty

When you are prompted, enter the password provided by your instructor.

Setup

Step 1.1.2

Run the following command to check the Vault server status:

$ vault status

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.0.2
Cluster Name    vault-cluster-875c9adb
Cluster ID      8917ca81-e460-49e5-b85d-db02a34d2720
HA Enabled      false

Notice that the server has been unsealed.

Sealed          false

The server has been started in dev mode. When you start a Vault server in dev mode, it automatically unseals the server.

Step 1.1.3

Authenticate with Vault using the root token:

$ vault login root

Expected output:

Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                root
token_accessor       6urXl1sr1zQJRUHD95jUzC4P
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

NOTE: For the purpose of training, we will start slightly insecure and login using the root token. Also, the Vault server is running in dev mode.

Task 2: Getting Help

Step 1.2.1

Execute the following command to display available commands:

$ vault help

Or, you can use short-hand:

$ vault -h

Step 1.2.2

Get help on vault server commands:

$ vault server -h

The help message explains how to start a server and its available options.

As you verified at Step 1.1.2, the Vault server is already running. The server was started using the command described in the help message: vault server -dev -dev-root-token-id="root"

Step 1.2.3

Get help on the read command:

$ vault read -h

Notice that there are HTTP Options that you can pass since the CLI is invoking the Vault API underneath. Also, there are Output Options.

Step 1.2.4

To get help on the API, the help command becomes path-help instead:

$ vault path-help sys/policy

Task 3: Enable Audit Logging

Audit backend keeps a detailed log of all requests and responses to Vault. Sensitive information is obfuscated by default (HMAC). Prioritizes safety over availability.

Step 1.3.1

Change directory into /workstation/vault101

$ cd /workstation/vault101

Step 1.3.2

Get help on the audit enable command:

$ vault audit enable -h

Step 1.3.3

Let's write audit log in current working directly so that you can inspected as you go through other labs.

Execute the following command to enable audit logging:

$ vault audit enable file \
        file_path=/workstation/vault101/audit.log

Expected output:

Success! Enabled the file audit device at: file/

Step 1.3.4

You can verify that the audit log file is generated:

$ sudo cat audit.log

If prompted for password, enter the student workstation password.

However, at this point, its content is hard to read. You can pipe the output with jq tool.

$ sudo cat audit.log | jq
...
  "request": {
    "id": "0f2fb5fd-6a74-f425-9537-2c6d4283b7b8",
    "operation": "read",
    "client_token": "hmac-sha256:85a4130cf4527b8bc5...",
    "client_token_accessor": "hmac-sha256:7dcfaabb1c...",
    "path": "secret/company",
    "data": null,
    "policy_override": false,
}
...

Sensitive information such as client token is obfuscated by default (HMAC).

NOTE: To disable the audit log, execute the following command.

$ vault audit disable file

Optional

Often times, the logged information can help you understand what is going on with each command during the development. Invoke the following command to generate a raw log:

$ vault audit enable -path=file-raw file \
        file_path=/workstation/vault101/audit-raw.log log_raw=true

If you want to tail the log as you go through hands-on labs, you can open another terminal, and run the following command:

$ cd /workstation/vault101

$ sudo tail -f audit-raw.log | jq

Task 4: Access Vault UI

Vault UI is another useful client interface to interact with Vault.

Step 1.4.1

Open a web browser and enter the following address to launch Vault UI: http://<workstation_ip>:8200/ui/vault

Step 1.4.2

Enter root in the Token field, and click Sign in.

Login


End of Lab 1

Lab 2: Static Secrets

Duration: 20 minutes

This lab demonstrates both CLI commands and API to interact with key/value and cubbyhole secret engines.

  • Task 1: Write Key/Value Secrets using CLI
  • Task 2: List Secret Keys using CLI
  • Task 3: Delete Secrets using CLI
  • Task 4: Working with Key/Value Secret Engine using API
  • Task 5: Explorer UI only feature
  • Challenge: Protect secrets from unintentional overwrite

Task 1: Write Key/Value Secrets using CLI

First, write your very first secrets in the key/value secret engine.

Step 2.1.1

First, check the current version of the key/value secret engine. Execute the following command:

$ vault secrets list -detailed

In the output, locate "secret/" and check its version under Options.

Path          Type         Accessor              ...  Options  
----          ----         --------              ...  -------  
cubbyhole/    cubbyhole    cubbyhole_8f752112    ...  map[]    
identity/     identity     identity_8fb35fba     ...  map[]    
secret/       kv           kv_00c670a4           ...  map[version:2]
...

Step 2.1.2

Execute the following command to read secrets at secret/training path:

$ vault kv get secret/training

Expected output:

====== Metadata ======
Key              Value
---              -----
created_time     2019-03-01T18:37:15.170521722Z
deletion_time    n/a
destroyed        false
version          1

==== Data ====
Key      Value
---      -----
value    Hello!

Step 2.1.3

Write a secret into secret/training path:

$ vault kv put secret/training username="student01" password="pAssw0rd"

Expected output:

Key              Value
---              -----
created_time     2019-03-01T23:28:30.587223947Z
deletion_time    n/a
destroyed        false
version          2

Step 2.1.4

Now, read the secrets in secret/training path.

$ vault kv get secret/training

Expected output:

====== Metadata ======
Key              Value
---              -----
created_time     2019-03-01T23:28:30.587223947Z
deletion_time    n/a
destroyed        false
version          2

====== Data ======
Key         Value
---         -----
password    pAssw0rd
username    student01

Step 2.1.5

Retrieve only the username value from secret/training.

$ vault kv get -field=username secret/training

Expected output:

student01

Question

What will happen to the contents of the secret when you execute the following command:

$ vault kv put secret/training password="another-password"

Answer

Creates another version of the secret.

Key              Value
---              -----
created_time     2019-03-01T23:29:12.580401169Z
deletion_time    n/a
destroyed        false
version          3

When you read back the data, username no longer exists!

$ vault kv get secret/training

====== Metadata ======
Key              Value
---              -----
created_time     2019-03-01T23:29:12.580401169Z
deletion_time    n/a
destroyed        false
version          3

====== Data ======
Key         Value
---         -----
password    another-password

This is very important to understand. The key/value secret engine does NOT merge or add values. If you want to add/update a key, you must specify all the existing keys as well; otherwise, data loss can occur!

Step 2.1.6

If you wish to partially update the value, use patch:

$ vault kv patch secret/training course="Vault 101"

This time, you should see that the course value is added to the existing key.

$ vault kv get secret/training
...
====== Data ======
Key         Value
---         -----
course      Vault 101
password    another-password

Step 2.1.7

Review a file named, data.json in the /workstation/vault101 directory:

$ cat data.json
{
  "organization": "hashicorp",
  "region": "US-West",
  "zip_code": "94105"
}

Step 2.1.8

Now, let's upload the data from data.json:

$ vault kv put secret/company @data.json

Read the secret in the secret/company path:

$ vault kv get secret/company

====== Metadata ======
Key              Value
---              -----
created_time     2019-03-01T23:30:35.24991211Z
deletion_time    n/a
destroyed        false
version          1

======== Data ========
Key             Value
---             -----
organization    hashicorp
region          US-West
zip_code        94105

Task 2: List Secret Keys using CLI

Step 2.2.1

Get help on the list command:

$ vault kv list -h

This command can be used to list keys in a given secret engine.

Step 2.2.2

List all the secret keys stored in the key/value secret backend.

$ vault kv list secret

Expected output:

Keys
----
company
training

The output displays only the keys and not the values.

Task 3: Delete Secrets using CLI

Step 2.3.1

Get help on the delete command:

$ vault kv delete -h

This command deletes secrets and configuration from Vault at the given path.

Step 2.3.2

Delete secret/company:

$ vault kv delete secret/company

Step 2.3.3

Try reading the secret/company path again. (Hint: Step 2.1.8)

Expected output includes the deletion_time:

====== Metadata ======
Key              Value
---              -----
created_time     2019-03-01T23:30:35.24991211Z
deletion_time    2019-03-01T23:31:17.090938047Z
destroyed        false
version          1

NOTE: To permanently delete secret/company, use vault kv destroy or vault kv metadata delete commands instead.

Task 4: Working with Key/Value Secret Engine API

In this task, you are going to write, read, and delete secrets in key/value secret engine via API.

Step 2.4.1

Check the vault address on your student workstation:

$ echo $VAULT_ADDR

Expected output:

http://127.0.0.1:8200

Step 2.4.2

You have learned how to store secrets under secret/ using CLI. To find out the equivalent API, you can use "-output-curl-string" flag:

$ vault kv put -output-curl-string secret/apikey/google apikey="my-api-key"

curl -X PUT -H "X-Vault-Token: $(vault print token)" -d '{"data":{"apikey":"my-api-key"},"options":{}}' http://127.0.0.1:8200/v1/secret/data/apikey/google

You can copy and paste the output to invoke the API using cURL.

Use the jq tool to parse the output for readability as follow:

$ curl --header "X-Vault-Token: root" --request POST \
        --data '{"data": {"apikey": "my-api-key"} }' \
        $VAULT_ADDR/v1/secret/data/apikey/google | jq

NOTE: If you are tailing the audit.log (optional step in Lab 1), you should see the trace log of this API call.

Step 2.4.3

Read the data in secret/apikey/google path:

$ curl --header "X-Vault-Token: root"  \
       $VAULT_ADDR/v1/secret/data/apikey/google | jq

Expected output:

{
  "request_id": "dda623da-ff4f-7417-f354-4dcfa68cff5e",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "data": {
      "apikey": "my-api-key"
    },
    "metadata": {
      "created_time": "2018-05-02T18:59:24.293039655Z",
      "deletion_time": "",
      "destroyed": false,
      "version": 1
    }
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

Step 2.4.4

To retrieve the apikey value alone:

$ curl -s --header "X-Vault-Token: root" \
       $VAULT_ADDR/v1/secret/data/apikey/google | jq ".data.data.apikey"

Step 2.4.5

Delete the latest version of secret/apikey/google using API.

$ curl --header "X-Vault-Token: root" \
       --request DELETE \
       $VAULT_ADDR/v1/secret/data/apikey/google

Step 2.4.6

Launch the Vault UI if it's not already running: http://<workstation_ip>:8200/ui/vault (Note: Enter root in the Token field, and click Sign in.)

Select secret from the Secrets Engines list, and then apikey > google. It should show that version 1 of this secret has been deleted if your API invocation was successful.

Vault UI

Task 5: Explorer UI only feature

Step 2.5.1

In the UI, return to the secrets root.

Vault UI

Step 2.5.2

Select training and then History > View version history.

Vault UI

Step 2.5.3

From Version 2 menu, select Create new version from 2.

Vault UI

Step 2.5.4

Modify the secrets by adding course key and its value, Vault. Vault UI

Click Save.

This creates version 5 of the data.

NOTE: The patch command enabled you to merge new values into the latest version of the key/value secret. Only from UI, you can create a new version based on any of the versions. This is particularly useful when you unintentionally modified the data and wish to recover.


Challenge: Protect secrets from unintentional overwrite

How can an organization protect the secrets in secret/data/certificates from being unintentionally overwritten?

Hint:

Lab 2: Static Secrets - Challenge Solution

Challenge: Protect secrets from unintentional overwrite

You have a couple of options:

  • Option 1: Enable check-and-set at the secret/data/certificates level
  • Option 2: Remind everyone to pass the -cas flag with every write operation

Option 1

Enable check-and-set at the secret/data/certificates:

$ vault kv metadata put -cas-required secret/certificates

This ensures that every write operation must pass the -cas flag.

Example:

$ vault kv put secret/certificates root="certificate.pem"

Error writing data to secret/data/certificates: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/secret/data/certificates
Code: 400. Errors:

* check-and-set parameter required for this call

In absence of the -cas flag, the write operation fails.

$ vault kv put -cas=0 secret/certificates root="certificate.pem"
Key              Value
---              -----
created_time     2018-06-11T21:59:06.055765168Z
deletion_time    n/a
destroyed        false
version          1

If you re-run the same command:

$ vault kv put -cas=0 secret/certificates root="certificate.pem"

Error writing data to secret/data/certificates: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/secret/data/certificates
Code: 400. Errors:

* check-and-set parameter did not match the current version

Since -cas=0 allows the write operation only if there is no secret already exists at secret/certificates.

Option 2

Make sure that everyone to pass the -cas flag with every write operation":

$ vault kv put -cas=1 secret/certificates root="certificate.pem"

The down side of this is that there will be no warning if one forgets to pass the -cas flag.

To learn more about the versioned key/value secret engine, refer to the Versioned Key/Value Secret Engine guide at https://www.vaultproject.io/guides/secret-mgmt/versioned-kv.html.


### End of Lab 2 # Lab 3: Cubbyhole Secret Engine

Duration: 20 minutes

This lab demonstrates both CLI commands and API to interact with key/value and cubbyhole secret engines.

  • Task 1: Test the Cubbyhole Secret Engine using CLI
  • Task 2: Trigger a response wrapping
  • Task 3: Unwrap the Wrapped Secret
  • Task 4: Response Wrapping via UI

Task 1: Test the Cubbyhole Secret Engine using CLI

Step 3.1.1

To better demonstrate the cubbyhole secret engine, first create a non-privileged token.

$ vault token create -policy=default

Expected output look similar to:

Key                  Value
---                  -----
token                s.FECyDayl7PS0g8I4JSuJwYdj
token_accessor       3n0ODvf1106H6OAkGMTvJNTx
token_duration       768h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Step 3.1.2

Log into Vault using the newly generated token:

$ vault login <token>

Example:

$ vault login s.FECyDayl7PS0g8I4JSuJwYdj

Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.FECyDayl7PS0g8I4JSuJwYdj
token_accessor       3n0ODvf1106H6OAkGMTvJNTx
token_duration       767h59m48s
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Step 3.1.3

Execute the following command to write secret in the cubbyhole/private path:

$ vault write cubbyhole/private mobile="123-456-7890"

Step 3.1.4

Read back the secret you just wrote. It should return the secret.

$ vault read cubbyhole/private

Key       Value
---       -----
mobile    123-456-7890

Step 3.1.5

Login with root token:

$ vault login root

Step 3.1.6

Now, try to read the cubbyhole/private path.

$ vault read cubbyhole/private

What response did you receive?

Cubbyhole secret backend provide an isolated secrete storage area for an individual token where no other token can violate.

Cubbyhole Wrapping Token

Think of a scenario where a user does not have a permission to read secrets from the secret/data/training path. As a privileged user (admin), you have a permission to read the secret in secret/data/training.

You can use response wrapping to pass the secret to the non-privileged user.

  1. Admin user reads the secret in secret/data/training with response wrapping enabled
  2. Vault creates a temporal token (wrapping token) and place the requested secret in the wrapping token's cubbyhole
  3. Vault returns the wrapping token to the admin
  4. Admin delivers the wrapping token to the non-privileged user
  5. User uses the wrapping token to read the secret placed in its cubbyhole

description

Remember that cubbyhole is tied to its token that even the root cannot read it if the cubbyhole does not belong to the root token.

NOTE: A common usage of the response wrapping is to wrap an initial token for a trusted entity to use. For example, an admin generates a Vault token for a Jenkins server to use. Instead of transmitting the token value over the wire, response wrap the token, and let the Jenkins server to unwrap it.

Task 2: Trigger Response Wrapping

Step 3.2.1

Execute the following commands to read secrets using response wrapping with TTL of 360 seconds.

$ vault kv get -wrap-ttl=360 secret/training

Output should look similar to:

Key                              Value
---                              -----
wrapping_token:                  s.mieZRgn1hcupCqmXJEdfhnY3
wrapping_accessor:               F5BjzIl8j2rHjlciA5f4nxDN
wrapping_token_ttl:              6m
wrapping_token_creation_time:    2019-02-11 21:28:44.188122677 +0000 UTC
wrapping_token_creation_path:    secret/data/training

The response is the wrapping token; therefore, the admin user does not even see the secrets.

Make a note of this wrapping_token. You will use it later to unwrap the secret.

Task 3: Unwrap the Wrapped Secret

Since you are currently logged in as a root, you are going to perform the following to demonstrate the apps operations:

  1. Create a token with default policy (non-privileged token)
  2. Authenticate with Vault using this default token
  3. Unwrap the secret to obtain the apps token
  4. Verify that you can read secret/data/dev using the apps token

Step 3.3.1

Generate a token with default policy:

$ vault token create -policy=default

Key                  Value
---                  -----
token                s.daroDM01N0NCglvwGwlYIjtS
token_accessor       aClzy3fhMO6PLcPQkrD8gGVm
token_duration       768h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Step 3.3.2

Login with the generated token.

Example:

$ vault login s.daroDM01N0NCglvwGwlYIjtS

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.daroDM01N0NCglvwGwlYIjtS
token_accessor       aClzy3fhMO6PLcPQkrD8gGVm
token_duration       767h59m23s
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Step 3.3.3

Test to make sure that you cannot read the secret/data/training path with default token.

$ vault kv get secret/training

Error making API request.

URL: GET http://127.0.0.1:8200/v1/sys/internal/ui/mounts/secret/training
Code: 403. Errors:

* Preflight capability check returned 403, please ensure client's policies grant access to path "secret/training/"

Step 3.3.4

Now, execute the following commands to unwrap the secret.

$ vault unwrap <WRAPPING_TOKEN>

Where <WRAPPING_TOKEN> is the wrapping_token obtained at Step 3.2.1.

For example:

$ vault unwrap s.mieZRgn1hcupCqmXJEdfhnY3

Key         Value
---         -----
data        map[course:Vault 101 password:another-password]
metadata    map[created_time:2019-02-11T21:08:42.098087533Z deletion_time: destroyed:false version:3]

Since the wrapping token is a single-use token, you will receive an error if you re-run the command.

Step 3.3.5

Log back in as root:

$ vault login root

Question

What happens to the token if no one unwrap its containing secrets within 360 seconds?

Answer

To test this, generate a new token with short TTL (e.g. 15 seconds):

$ vault token create -wrap-ttl=15 -format=json \
        | jq -r ".wrap_info.token" > wrapping-token.txt

The above command stores the generated wrapping_token in a file.

Wait for 15 seconds and try to unwrap the containing secret:

$ vault unwrap $(cat wrapping-token.txt)

Error unwrapping: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/sys/wrapping/unwrap
Code: 400. Errors:

* wrapping token is not valid or does not exist

NOTE: The TTL of the wrapping token is separate from the wrapped secret's TTL (in this case, a new token you generated).

Task 4: Response Wrapping via UI

Launch the Vault UI if it's not already running: http://<workstation_ip>:8200/ui/vault (Note: Enter root in the Token field, and click Sign in.)

Step 3.4.1

Under Secrets, select secret > training, and then select Copy Secret > Wrap Secret.

Vault UI

Step 3.4.2

Copy the wrapping token value.

Vault UI


Paste the value into your prefer text editor. You will use this later.

Step 3.4.3

Sign out of the UI.

Vault UI

Step 3.4.4

Now, sign in using one of the non-root token you generated earlier. (Hint: Step 3.3.1) Notice that this non-root token does not have a permission to view secret/training path.

Step 3.4.5

Select Tools tab, and then Unwrap. Paste in the wrapping token value you copied earlier.

Vault UI

Step 3.4.6

Click Unwrap Data.

Vault UI

Now, non-root token can see the secrets along with its metadata.

Summary: The Cubbyhole response wrapping allows privileged token to wrap a secret so that non-privilege token can read the secrets once. This does not require any policy change on the non-privilege token. Since you are sending the wrapping token which is a reference to the wrapped secrets and not the actual secrets over the wire, it is more secure.

Additional Exercises

To learn more about Cubbyhole secret engine, try additional hands-on exercises:


End of Lab 3

Lab 4: Secrets as a Service - Dynamic Secrets

Duration: 25 minutes

This lab demonstrates how Vault generates dynamic credentials for database on-demand.

  • Task 1: Enable and Configure a Database Secret Engine
  • Task 2: Generate Readonly PostgreSQL Credentials
  • Task 3: Revoke Leases
  • Challenge: Setup Database Secret Engine via API

The scenario is:

DB Dynamic Secrets

A privileged user (e.g. admin, security team) enables and configures the database secret engine. Also, creates a role which defines what kind of users to generate credentials for. Once the secret engine is set up, the Vault clients (apps, machine, etc.) can request a new set of database credentials. Since the clients don't need the database access for a prolonged time, you are going to set its expiration time as well.

Task 1: Enable and Configure a Database Secret Engine

For a production environment, this task is performed by a privileged user.

Step 4.1.1

Most secret engines must be enabled and configured before use. Execute the following command to enable database secret engine:

$ vault secrets enable database

NOTE: By default, this mounts the database secret engine at database/ path. If you wish the mounting path to be different, you can pass -path to set desired path.

Expected output:

Success! Enabled the database secrets engine at: database/

Step 4.1.2

Now that you have mounted the database secret engine, you can ask for help to configure it. Use the path-help command to display the help message.

$ vault path-help database/

Also, refer to the online API document: https://www.vaultproject.io/api/secret/databases/index.html.

Step 4.1.3

In this lab scenario, you are going to configure a database secret engine for PostgreSQL.

Execute the following command to configure the database secret engine:

$ vault write database/config/postgresql \
        plugin_name=postgresql-database-plugin \
        allowed_roles=readonly \
        connection_url=postgresql://postgres@localhost/myapp

NOTE: For the purpose of training, PostgreSQL has been installed and a database name, myapp, has been created on each student workstation. It is very common to give Vault the root credentials and let Vault manage the auditing and lifecycle credentials instead of having one person manage it manually.

Step 4.1.4

Notice that you set the allowed_roles to be readonly in previous step. Since Vault does not know what kind of PostgreSQL users you want to create. So, you supply the rule with the SQL to run and create the users.

Since this is not a SQL course, we've added the SQL on the student workstation. You can see the script:

$ cat readonly.sql
CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';
REVOKE ALL ON SCHEMA public FROM public, "{{name}}";
GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";

The values between the {{ }} will be filled in by Vault. Notice that we are using the VALID UNTIL clause. This tells PostgreSQL to revoke the credentials even if Vault is offline or unable to communicate with it.

Step 4.1.5

Next step is to configure a role. A role is a logical name that maps to a policy used to generate credentials. Here, we are defining a readonly role.

$ vault write database/roles/readonly db_name=postgresql \
        creation_statements=@readonly.sql \
        default_ttl=1h max_ttl=24h

NOTE: This command creates a role named, readonly which has a default TTL of 1 hour, and max TTL is 24 hours. The credentials for readonly role expires after 1 hour, but can be renewed multiple times within 24 hours of its creation. This is an example of restricting how long the database credentials should be valid.

Task 2: Generate Read-only PostgreSQL Credentials

As described earlier, privileged users (admin, security team, etc.) enable and configure the database secret engine. Therefore, Task 1 is a task that needs to be completed by the privileged users.

Now that the database secret engine has been enabled and configured, applications (Vault clients) can request a set of PostgreSQL credentials to read from the database.

Step 4.2.1

Execute the following command to generate a new set of credentials:

$ vault read database/creds/readonly

The output should look similar to:

Key                Value
---                -----
lease_id           database/creds/readonly/86a2109c-780c...
lease_duration     1h
lease_renewable    true
password           A1a-u443zy2w14245784
username           v-token-readonly-x271s0zv6x42wsqx...

To generate new credentials, you simply read from the role endpoint.

Step 4.2.2

Copy the lease_id. You will use it later.

Step 4.2.3

Let's check that the newly generated username exists by logging in as the postgres user and list all accounts.

$ psql -U postgres

At the postgres command prompt, enter \du to list all accounts.

postgres > \du

postgres output

The username generated at Step 4.2.1 should be listed.

Notice that the Attributes for your username has "password valid until" clause.
This means that even if an attacker is able to DDoS Vault or take it offline, PostgreSQL will still revoke the credential. When backends support this expiration, Vault will take advantage of it.

Step 4.2.4

Enter \q to exit.

Step 4.2.5

Now, let's renew the lease for this credential.

$ vault lease renew <lease_id>

While <lease_id> is what you copied at Step 4.2.2.

Expected output:

Key                Value
---                -----
lease_id           database/creds/readonly/86a2109c-780c...
lease_duration     1h
lease_renewable    true

The lease duration for this credential is now reset.

For the clients to be able to read credentials and renew its lease, its policy must grants the following:

# Get credentials from the database backend
path "database/creds/readonly" {
   capabilities = [ "read" ]
}

# Renew the lease
path "/sys/leases/renew" {
   capabilities = [ "update" ]
}

Step 4.2.6

You can renew and increment the TTL of the lease:

$ vault lease renew -increment=2h <lease_id>

Expected output:

Key                Value
---                -----
lease_id           database/creds/readonly/86a2109c-780c...
lease_duration     2h
lease_renewable    true

Task 3: Revoke Leases

Under a certain circumstances, the privileged users may need to revoke the existing database credentials.

Step 4.3.1

When the database credentials are no longer in use, or need to be disabled, run the following command:

$ vault lease revoke <lease_id>

While <lease_id> is what you copied at Step 4.2.2.

Expected output:

All revocation operations queued successfully!

Step 4.3.2

You can verify that the username no longer exists by logging in as postgres user and list all accounts as you did in Step 4.2.3.

postgres output

Step 4.3.3

Let's read a few more credentials from the postgres secret engine. Here, you will simulate a scenario where multiple applications have requested readonly database access.

$ vault read database/creds/readonly
Key                Value
---                -----
lease_id           database/creds/readonly/563e5e58-aa31-564c-4637-70804cc63fe1
lease_duration     1h
lease_renewable    true
password           A1a-zr9q5t79391w569z
username           v-token-readonly-0306y039q232wvr2y59p-1517945642


$ vault read database/creds/readonly
Key                Value
---                -----
lease_id           database/creds/readonly/67fdf769-c28c-eba7-0ac4-ac9a52f13e4c
lease_duration     1h
lease_renewable    true
password           A1a-89q59vqz83z892xs
username           v-token-readonly-74551qs2us5zzqwsqw56-1517945647


$ vault read database/creds/readonly
Key                Value
---                -----
lease_id           database/creds/readonly/b422c54b-2664-e0b4-1b6e-74badbd7ab1c
lease_duration     1h
lease_renewable    true
password           A1a-0wsw97r2x6s49qv9
username           v-token-readonly-838uu0r2vvzyw0p34qw4-1517945648

Now, you have multiple sets of credentials. 

Step 4.3.4

Imagine a scenario where you need to revoke all these secrets. Maybe you detected an anomaly in the postgres logs or the vault logs indicates that there may be a breach!

If you know exactly where the root of the problem, you can revoke the specific leases as you performed in Step 4.3.1. But what if you don't know!?

Execute the following command to revoke all readonly credentials.

$ vault lease revoke -prefix database/creds/readonly

Expected output:

All revocation operations queued successfully!

If you want to revoke all database credentials, run:

$ vault lease revoke -prefix database/creds

Challenge: Setup Database Secret Engine with API

Perform the same tasks using API.

  1. Enable database secret engine at a different path (e.g. postgres-db/)
  2. Configure the secret engine using the same parameters in Task 1
    • plugin_name: postgresql-database-plugin
    • allowed_roles: readonly
    • connection_url: postgresql://postgres@localhost/myapp
  3. Create a new role named, readonly
    • db_name: postgresql
    • creation_statements: readonly.sql
    • default_ttl: 1h
    • max_ttl: 24h
  4. Generate a new set of credentials for readonly role

Hint & Tips:

Lab 4: Dynamic Secrets - Challenge Solution

Challenge: Setup Database Secret Engine with API - Sample Solution

# Enable database secret engine at 'postgres-db'
$ curl --header "X-Vault-Token: root" --request POST \
       --data '{"type": "database"}' \
       $VAULT_ADDR/v1/sys/mounts/postgres-db

# Request message to configure the secret engine
$ tee payload.json <<EOF
{
   "plugin_name": "postgresql-database-plugin",
   "allowed_roles": "readonly",
   "connection_url": "postgresql://postgres@localhost/myapp"
}
EOF

# API call to configure the database secret engine
$ curl --header "X-Vault-Token: root" --request POST \
       --data @payload.json \
       $VAULT_ADDR/v1/postgres-db/config/postgresql

# Request message for creating a role
$ tee payload2.json <<EOF
{
    "db_name": "postgresql",
    "creation_statements": ["CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; REVOKE ALL ON SCHEMA public FROM public, \"{{name}}\"; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";"],
    "default_ttl": "1h",
    "max_ttl": "24h"
}
EOF

# API call to create a role named 'readonly'
$ curl --header "X-Vault-Token: root" --request POST \
       --data @payload2.json \
       $VAULT_ADDR/v1/postgres-db/roles/readonly

# API call to get a new set of credentials
$ curl --header "X-Vault-Token: root" --request GET \
      $VAULT_ADDR/v1/postgres-db/creds/readonly | jq

End of Lab 4

Lab 5: Encryption as a Service - Transit Secrets Engine

Duration: 25 minutes

The transit secrets engine enables security teams to fortify data during transit and at rest. So even if an intrusion occurs, your data is encrypted with AES 256-bit CBC encryption (TLS in transit). Even if an attacker were able to access the raw data, they would only have encrypted bits. This means attackers would need to compromise multiple systems before exfiltrating data.

This lab demonstrates how Vault provides cryptographic service:

  • Task 1: Configure Transit Secret Engine
  • Task 2: Encrypt Secrets
  • Task 3: Decrypt a cipher-text
  • Task 4: Rotate the Encryption Key
  • Task 5: Update the Key Configuration
  • Task 6: Encrypt data via UI
  • Challenge: Sign and Validate Data

Encryption as a Service

Task 1: Configure Transit Secret Engine

The transit secrets engine must be configured before it can perform its operations. This step is usually done by an operator or a configuration management tool.

Step 5.1.1

Enable the transit secret engine by executing the following command:

$ vault secrets enable transit

Step 5.1.2

Now, create an encryption key ring named, cards by executing the following command:

$ vault write -f transit/keys/cards

Task 2: Encrypt Secrets

Once the transit secrets engine has been configured, any client with a valid token with proper permission can send data to encrypt.

Here, you are going to encrypt a plaintext, "credit-card-number".

NOTE: You can pass non-text binary file such as a PDF or image. When you encrypt a plaintext, it must be base64-encoded.

Step 5.2.1

To encrypt your secret, use the transit/encrypt endpoint. Execute the following command to encrypt a plaintext:

$ vault write transit/encrypt/cards plaintext=$(base64 <<< "credit-card-number")

Key           Value
---           -----
ciphertext    vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U=

Vault does NOT store any of this data. The output you received is the ciphertext. You can store this ciphertext at the desired location (e.g. MySQL database) or pass it to another application.

Task 3: Decrypt a cipher-text

Any client with a valid token with proper permission can decrypt the ciphertext generated by Vault. To decrypt the ciphertext, invoke the transit/decrypt endpoint.

Step 5.3.1

Execute the following command to decrypt the ciphertext resulted in Step 5.2.1.

Example:

$ vault write transit/decrypt/cards \
        ciphertext="vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPv..."

Key          Value
---          -----
plaintext    Y3JlZGl0LWNhcmQtbnVtYmVyCg==

Step 5.3.2

The resulting data is base64-encoded. To reveal the original plaintext, run the following command:

$ base64 --decode <<< "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
credit-card-number

Task 4: Rotate the Encryption Key

One of the benefits of using the Vault transit secrets engine is its ability to easily rotate the encryption keys. Keys can be rotated manually by a human, or an automated process which invokes the key rotation API endpoint through cron, a CI pipeline, a periodic Nomad batch job, Kubernetes Job, etc.

Vault maintains the versioned keyring and the operator can decide the minimum version allowed for decryption operations. When data is encrypted using Vault, the resulting ciphertext is prepended with the version of the key used to encrypt it.

Step 5.4.1

To rotate the encryption key, invoke the transit/keys/<key_ring_name>/rotate endpoint.

$ vault write -f transit/keys/cards/rotate

Step 5.4.2

Let's encrypt another data:

$ vault write transit/encrypt/cards plaintext=$(base64 <<< "visa-card-number")
Key           Value
---           -----
ciphertext    vault:v2:45f9zW6cglbrzCjI0yCyC6DBYtSBSxnMgUn9B5aHcGE...

Step 5.4.3

Compare the ciphertexts:

ciphertext    vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy...

Notice that the first ciphertext starts with "vault:v1:". After rotating the encryption key, the ciphertext starts with "vault:v2:". This indicates that the data gets encrypted using the latest version of the key after the rotation.

Step 5.4.4

Execute the following command to rewrap your ciphertext from Step 5.2.1 with the latest version of the encryption key:

$ vault write transit/rewrap/cards \
        ciphertext="vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZP..."

Key           Value
---           -----
ciphertext    vault:v2:kChHZ9w4ILRfw+DzO53IZ8m5PyB2yp2/tKbub34...

Notice that the resulting ciphertext now starts with "vault:v2:".

This operation does not reveal the plaintext data. But Vault will decrypt the value using the appropriate key in the keyring and then encrypted the resulting plaintext with the newest key in the keyring.

Task 5: Update Key Configuration

The operators can update the encryption key configuration to specify the minimum version of ciphertext allowed to be decrypted, the minimum version of the key that can be used to encrypt the plaintext, the key is allowed to be deleted, etc.

This helps further tightening the data encryption rule.

Step 5.5.1

Execute the key rotation command a few times to generate multiple versions of the key:

$ vault write -f transit/keys/cards/rotate

Step 5.5.2

Now, read the cards key information:

$ vault read transit/keys/cards

Key                       Value
---                       -----
...
keys                      map[6:1531439714 1:1531439594 2:1531439667 3:1531439714 4:1531439714 5:1531439714]
latest_version            6
min_decryption_version    1
min_encryption_version    0
...

In the example, the current version of the key is 6. However, there is no restriction about the minimum encryption key version, and any of the key versions can decrypt the data (min_decryption_version).

Step 5.5.3

Run the following command to enforce the use of the encryption key at version 5 or later to decrypt the data.

$ vault write transit/keys/cards/config min_decryption_version=5

Step 5.5.4

Now, verify the cards key configuration:

$ vault read transit/keys/cards

Key                       Value
---                       -----
allow_plaintext_backup    false
deletion_allowed          false
derived                   false
exportable                false
keys                      map[5:1531811719 6:1531811721]
latest_version            6
min_decryption_version    5
min_encryption_version    0
...

Task 6: Encrypt data via UI

Launch the Vault UI if it's not already running: http://<workstation_ip>:8200/ui/vault (Note: Enter root in the Token field, and click Sign in.)

Step 5.6.1

Under Secrets, select transit > cards, and then select Key actions.

Vault UI

Step 5.6.2

With Encrypt selected, enter "my-master-card-number" in the Plaintext field.

Vault UI

Clicke Encode to base64.

Step 5.6.3

Click Encrypt.

Step 5.6.4

Click Copy to copy the ciphertext.

Step 5.6.5

Now, select Decrypt. (The ciphertext field should be already populated. If not, paste in the ciphertext you copied in Step 5.6.4.)

Finally, click Decode from base64 to reveal the original text.

Challenge: Sign and Validate Data

Consider a scenario where you want to ensure that the data came from a trusted source. You don't care who can read the data but you care about the source of the data. In such a case, you use data signing instead of encrypting.

During the lecture, it was mentioned that the transit secrets engine supports a number of key types and some support signing and signature verification. You can use Vault CLI or Web UI, and perform the following tasks:

  1. Create a key named, newsletter which uses rsa-4096 as its key type
  2. Sign some data using the newsletter key
  3. Verify the signature using the newsletter key

Hint:

Lab 5: Encryption as a Service - Transit Secrets Engine

Challenge: Sign and Validate Data - Sample Solution

CLI

# Create a key
$ vault write transit/keys/newsletter type="rsa-4096"

# Sign some test data
$ vault write transit/sign/newsletter \
          input=$(base64 <<< "HashiCorp Vault is awesome")

Key          Value
---          -----
signature    vault:v1:NCkg6X0AMdLSt4G+R4k8OGcaeVjSN5ZKmpXGFqxFYS8utV+aIahTv5vDCv26...

# Verify the sign data
$ vault write transit/verify/newsletter \
         input=$(base64 <<< "HashiCorp Vault is awesome") \
         signature="vault:v1:NCkg6X0AMdLSt4G+R4k8OGcaeVjSN5ZKmpXGFqxFYS8utV+aIahTv5vDCv26..."

Key      Value
---      -----
valid    true

**Web UI**
  1. Launch the Vault UI: http://<workstation_ip>:8200/ui

  2. Enter root in the Token field and then click Sign In

  3. Select transit from the Secrets Engine list

  4. Select Create encryption key

  5. Enter newsletter in the Name field, select rsa-4096 from the Type drop-down list

  6. Click Create encryption key

  7. Select Key actions and then Sign

  8. Enter "HashiCorp Vault is awesome" in the Plaintext field, and then click Encode to base64

  9. Click Encrypt

  10. Click Copy

  11. Now, select Verify. The Input and Signature fields should be pre-populated for you

  12. Click Verify. If it's successful, "The input is valid for the given signature" message should display

End of Lab 5

Lab 6: Authentication and Tokens

Duration: 20 minutes

Almost all operations in Vault requires a token; therefore, it is important to understand the token lifecycle as well as different token parameters that affects the token's lifecycle. This lab demonstrates various token parameters. In addition, you are going to enable userpass auth method and test it.

  • Task 1: Create a Short-Lived Tokens
  • Task 2: Token Renewal
  • Task 3: Create Tokens with Use Limit
  • Task 4: Create a Token Role and Periodic Token
  • Task 5: Create an Orphan Token
  • Task 6: Enable Username and Password Auth Method
  • Challenge: Generate batch tokens

Task 1: Create Short-Lived Tokens

When you have a scenario where an app talks to Vault only to retrieve a secret (e.g. API key), and never again. If the interaction between Vault and its client takes only a few seconds, there is no need to keep the token alive for longer than necessary. Let's create a token which is only valid for 30 seconds.

Step 6.1.1

Review the help message on token creation:

$ vault token create -h

Expected output:

Usage: vault token create [options]

Creates a new token that can be used for authentication. This token will be created as a child of the currently authenticated token. The generated token will inherit all policies and permissions of the currently authenticated  token unless you explicitly define a subset list policies to assign to the token.
...

Step 6.1.2

Execute the following command to create a token whose TTL is 30 seconds:

$ vault token create -ttl=90

Output should look similar to:

Key                  Value
---                  -----
token                s.5cR9lQmyMfiOGQb8znnq3mDT
token_accessor       4T9AqnCAF8PH9Dm9NOKB9PN8
token_duration       30s
token_renewable      true
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Notice that the generated token inherits the parent token's policy. For the training, you are logged in with root token. When you create a new token, it inherits the parent token's policy unless you specify with -policy parameter.

Copy the token value.

Step 6.1.3

Now, test the token:

$ vault token lookup <token>

Where <token> is the generated token from Step 6.1.2.

Example:

$ vault token lookup s.5cR9lQmyMfiOGQb8znnq3mDT

Key                 Value
---                 -----
accessor            4T9AqnCAF8PH9Dm9NOKB9PN8
creation_time       1544643551
creation_ttl        30s
display_name        token
entity_id           n/a
expire_time         2018-12-12T19:39:41.802136869Z
explicit_max_ttl    0s
id                  s.5cR9lQmyMfiOGQb8znnq3mDT
issue_time          2018-12-12T19:39:11.802135892Z
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [root]
renewable           true
ttl                 22s
type                service

Notice that the token type is set to service. And this token has 22 seconds TTL left before it expires.

Step 6.1.4

Use the upper-arrow key, and then re-run the same command again.

Expected output:

Error looking up token: Error making API request.

URL: POST http://127.0.0.1:8200/v1/auth/token/lookup
Code: 403. Errors:

* bad token

After 90 seconds, the token gets revoked automatically, and you can no longer make any request with this token.

Task 2: Token Renewal

Step 6.2.1

Review the help message on token creation:

$ vault token renew -h

Expected output:

Usage: vault token renew [options] [TOKEN]

Renews a token's lease, extending the amount of time it can be used. If a TOKEN is not provided, the locally authenticated token is used. Lease renewal will fail if the token is not renewable, the token has already been revoked,
or if the token has already reached its maximum TTL.
...

Command Options:

  -increment=<duration>
      Request a specific increment for renewal. Vault is not
      required to honor this request. If not supplied, Vault
      will use the default TTL. This is specified as a
      numeric string with suffix like "30s" or "5m". This is
      aliased as "-i".

Step 6.2.2

Let's create another token with default policy and TTL of 120 seconds:

$ vault token create -ttl=120 -policy="default"

Output should look similar to:

Key                  Value
---                  -----
token                s.1SYfy0LBzGDzUK0kCKaMq6aY
token_accessor       86Rp1mgqUydD0Ns10NbYPgq4
token_duration       2m
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Step 6.2.3

Let's take a look at the token details:

$ vault token lookup <token>

While <token> is the token from Step 6.2.2.

Output should look similar to:

Key                  Value
---                  -----
accessor             86Rp1mgqUydD0Ns10NbYPgq4
creation_time        1544643665
creation_ttl         2m
display_name         token
entity_id            n/a
expire_time          2018-12-12T19:43:23.7754948Z
explicit_max_ttl     0s
id                   s.1SYfy0LBzGDzUK0kCKaMq6aY
issue_time           2018-12-12T19:41:05.531310836Z
last_renewal         2018-12-12T19:41:23.775495601Z
last_renewal_time    1544643683
meta                 <nil>
num_uses             0
orphan               false
path                 auth/token/create
policies             [default]
renewable            true
ttl                  32s
type                 service

Step 6.2.4

Renew the token and double its TTL:

$ vault token renew -increment=180 <token>

While <token> is the token from Step 5.2.1.

Output should look similar to:

Key                  Value
---                  -----
token                s.1SYfy0LBzGDzUK0kCKaMq6aY
token_accessor       86Rp1mgqUydD0Ns10NbYPgq4
token_duration       3m
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Now the token duration is extended to 3 minutes.

Step 6.2.5

Look up the token details again to verify that is TTL has been updated.

$ vault token lookup <token>

Output should look similar to:

Key                  Value
---                  -----
accessor             WUQBrqOHy8coeWlB75ubypjI
creation_time        1552699378
creation_ttl         2m
display_name         token
entity_id            n/a
expire_time          2019-03-15T18:26:31.1699-07:00
explicit_max_ttl     0s
id                   s.YYu0mojeAtd9ytpDC5n1RpvS
...
ttl                  2m42s
type                 service

Task 3: Create Tokens with Use Limit

Step 6.3.1

Create a token with use limit of 2.

$ vault token create -use-limit=2

Output should look similar to:

Key                  Value
---                  -----
token                s.1MXcFZsHMQdniRV4RS9kQfAW
token_accessor       20y827S8hQC3fRpQFHsZ6Ymu
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Step 6.3.2

Test the token with use limit.

Example:

$ VAULT_TOKEN=<token> vault token lookup
Key                 Value
---                 -----
accessor            20y827S8hQC3fRpQFHsZ6Ymu
...
num_uses            1
...


$ VAULT_TOKEN=<token> vault write cubbyhole/test name="student01"
Success! Data written to: cubbyhole/test

$ VAULT_TOKEN=<token> vault read cubbyhole/test
Error making API request.

URL: GET http://127.0.0.1:8200/v1/sys/internal/ui/mounts/cubbyhole/test
Code: 403. Errors:

* permission denied

Task 4: Create an Orphan Token

Step 6.4.1

Create a token with TTL of 90 seconds.

$ vault token create -ttl=90

Output should look similar to:

Key                  Value
---                  -----
token                d3f9538e-32d4-bcb1-c982-c335af532d66
token_accessor       cdf7ab42-9b7d-cec5-bae1-fafab6aa9593
token_duration       1m30s
token_renewable      true
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Step 6.4.2

Using the generated token, create a child token with longer TTL, 180 seconds.

$ VAULT_TOKEN=<token> vault token create -ttl=180

Output should look similar to:

Key                  Value
---                  -----
token                89e11854-8fd3-f86b-3862-34157ecf34c7
token_accessor       2671d179-a8a8-bf30-a300-d19aeb5ece50
token_duration       3m
token_renewable      true
token_policies       ["root"]
identity_policies    []
policies             ["root"]

In this example, the token hierarchy is:

root
  |__ d3f9538e-32d4-bcb1-c982-c335af532d66 (TTL = 90 seconds)
          |__ 89e11854-8fd3-f86b-3862-34157ecf34c7 (TTL = 180 seconds)

Step 6.4.3

After 90 seconds, the token from Step 6.5.1 would expire!

This automatically revokes its child token. If you try to look up the child token, you should receive bad token error since the token was revoked when its parent expired.

$ vault token lookup <child_token>

Now, if this behavior is undesirable, you can create an orphan token instead.

Step 6.4.4

Now, repeat the exercise with -orphan flag.

$ vault token create -ttl=90

$ VAULT_TOKEN=<token> vault token create -ttl=180 -orphan

Now, manually revoke the parent token instead of waiting for it to expire.

$ vault token revoke <token>

# Verify that the orphan token still exists
$ vault token lookup <orphan_token>

Task 5: Create a Token Role and Periodic Token

A common use case of periodic token is long-running processes where generation of a new token can interrupt the entire system flow. This task demonstrates the creation of a role and periodic token for such long-running process.

Step 6.5.1

Get help on auth/token path:

$ vault path-help auth/token
...
## PATHS
    ...
   ^roles/(?P<role_name>\w(([\w-.]+)?\w)?)$
        This endpoint allows creating, reading, and deleting
        roles.
    ...

The API endpoint to create a token role is auth/token/roles.

Step 6.5.2

First, create a token role named, monitor. This role has default policy and token renewal period of 24 hours.

$ vault write auth/token/roles/monitor \
           allowed_policies="default" period="24h"

Expected output:

Success! Data written to: auth/token/roles/monitor

Step 6.5.3

Now, create a token for role, monitor:

$ vault token create -role="monitor"

Output should look similar to:

Key                  Value
---                  -----
token                s.r5pAUxAxo1bO1kle0h7m0Rl0
token_accessor       1gpzNPcBpcAHJ6sYyO7rg9Ry
token_duration       24h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

This token can be renewed multiple times indefinitely as long as it gets renewed before it expires.

Task 6: Enable Username & Password Auth Method

Now, shift a gear and you are going to enable userpass auth method.

Step 6.6.1

Execute the following command to list which authentication methods have been enabled:

$ vault auth list

Expected output:

Path      Type     Description
----      ----     -----------
token/    token    token based credentials

Step 6.6.2

Userpass auth method allows users to login with username and password. Execute the CLI command to enable the userpass auth method.

$ vault auth enable userpass

Now, when you list the enabled auth methods, you should see userpass.

Step 6.6.3

Let's create your first user.

$ vault write auth/userpass/users/<user_name> \
        password="training" policies="default"

While <user_name> is your desired user name.

Notice that the username is a part of the path and the two parameters are password (in plain-text) and the list of policies as comma-separated value.

Example:

$ vault write auth/userpass/users/student01 \
         password="training" policies="default"
Success! Data written to: auth/userpass/users/student01

Step 6.6.4

You can test it.

$ vault login -method=userpass username=<user_name> \
        password="training"

When you successfully authenticate with Vault using your username and password, Vault returns a token. From then on, you can use this token to make API calls and/or run CLI commands.

Example:

$ vault login -method=userpass username="student01" \
        password="training"
Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token.

Key                    Value
---                    -----
token                  s.1mhSiEOEQs21uW546ItFWm9m
token_accessor         6MliAbcbZgzX2lQTShBr3PSf
token_duration         768h
token_renewable        true
token_policies         ["default"]
identity_policies      []
policies               ["default"]
token_meta_username    student01

Step 6.6.5

Log back in with the root token.

$ vault login root

Challenge: Generate batch tokens

Now, you should be familiar with vault token commands. Perform the following tasks.

Task 1: Create a token of type batch with default policy attached, and its TTL is set to 360 seconds.

Task 2: Enable another userpass auth method at userpass-batch path which generates a batch token upon a successful user authentication. Be sure to test and verify.

Hint: Run vault auth enable -h to see the available parameters.

Lab 6: Authentication and Tokens - Challenge Solution

Task 1: Solution

Create a batch token with default policy attached, and its TTL is set to 360 seconds.

$ vault token create -type=batch -policy=default -ttl=360

Key                  Value
---                  -----
token                b.AAAAAQKbcLlgc7zZ57FcrH123AcHnprewUsV75CIck0PqLQ18nmXHv...
token_accessor       n/a
token_duration       6m
token_renewable      false
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Task 2: Solution

Enable another userpass auth method at userpass-batch path which generates a batch token upon a successful user authentication. Be sure to test and verify.

# Enable userpass at 'userpass-batch' which generates batch tokens
$ vault auth enable -path="userpass-batch" -token-type=batch userpass

# Create a user for testing
$ vault write auth/userpass-batch/users/john \
         password="training" policies="default"

# Authenticate as 'john' to verify its generate token type
# The token should starts with 'b.'
$ vault login -method=userpass -path="userpass-batch" \
        username="john" password="training"
Key                    Value
---                    -----
token                  b.AAAAAQIEQsY0RjktFBMD3U_8_w0_S0qCBreHJTW3tBwvXLxRk-in...
token_accessor         n/a
token_duration         768h
token_renewable        false
token_policies         ["default"]
identity_policies      []
policies               ["default"]
token_meta_username    john

End of Lab 6

Lab 7: Direct Application Integration

Duration: 35 minutes

This lab demonstrates the use of Consul Template and Envconsul tools. To understand the difference between the two tools, you are going to retrieve the same information from Vault.

  • Task 1: Run Vault Agent
  • Task 2: Use Consul Template to populate DB credentials
  • Task 3: Use Envconsul to populate DB credentials
  • Challenge: Read Key/Value secrets using Consul Template

Resources:

Task 1: Run Vault Agent

Vault Agent runs on the client side to automate leases and tokens lifecycle management.

Since each student has only one workstation assigned, you are going to run the Vault Agent on the same machine as where the Vault server is running. The only difference between this lab and the real world scenario is that you set VAULT_ADDR to a remote Vault server address.

Step 7.1.1

First, review the /workstation/vault101/setup-approle.sh script to examine what it performs.

$ cat setup-approle.sh

This script creates a new policy called, db_readonly. (This assumes that you have completed Lab 4.) It enables approle auth method, generates a role ID and stores it in a file named, "roleID". Also, it generates a secret ID and stores it in the "secretID" file.

AppRole Workflow

The approle auth method allows machines or apps to authenticate with Vault using Vault-defined roles. The role ID is equivalent to username, and the secret ID is equivalent to a password.

Refer to the AppRole Pull Authentication guide (https://learn.hashicorp.com/vault/identity-access-management/iam-authentication) as well as AppRole with Terraform & Chef guide (https://learn.hashicorp.com/vault/identity-access-management/iam-approle-trusted-entities) to learn more.

Step 7.1.2

Execute the setup-approle.sh script.

$ ./setup-approle.sh

Step 7.1.3

Examine the Vault Agent configuration file, /workstation/vault101/agent-config.hcl.

$ cat agent-config.hcl

exit_after_auth = false
pid_file = "./pidfile"

auto_auth {
   method "approle" {
       mount_path = "auth/approle"
       config = {
           role_id_file_path = "roleID"
           secret_id_file_path = "secretID"
           remove_secret_id_file_after_reading = false
       }
   }

   sink "file" {
       config = {
           path = "/workstation/vault101/approleToken"
       }
   }
}

cache {
   use_auto_auth_token = true
}

listener "tcp" {
   address = "127.0.0.1:8007"
   tls_disable = true
}

vault {
   address = "http://127.0.0.1:8200"
}

The auto_auth block points to the approle auth method which setup-approle.sh script configured. The acquired token gets stored in /workstation/vault101/approleToken (this is the sink location).

The cache block specifies the agent to listen port 8007.

Step 7.1.4

NOTE: If you want to run Vault Agent against your neighbor's Vault server instead, edit the vault block so that it points to the correct Vault server address. Needless to say, your neighbor has to provide you the roleID and secretID to successfully authenticate.

Execute the following command to start the Vault Agent with debug logs.

$ vault agent -config=agent-config.hcl -log-level=debug

==> Vault server started! Log data will stream in below:

==> Vault agent configuration:

           Api Address 1: http://127.0.0.1:8007
                     Cgo: disabled
               Log Level: debug
                 Version: Vault v1.1.0
             Version Sha: 36aa8c8dd1936e10ebd7a4c1d412ae0e6f7900bd
...

Step 7.1.5

Open another terminal, and then SSH into your student workstation. Be sure to change the working directory to /workstation/vault101. Place the two terminals side-by-side if possible so that you can examine the logs as you execute commands.

Terminals

Step 7.1.6

Vault Agent successfully authenticated with Vault using the roleID and secretID, and stored the acquired token in the approleToken file.

$ more approleToken
s.DL0ToAJKVjOSXXZdfzAKPWLY

Notice the following entires in the agent log in the first terminal:

[INFO]  sink.file: creating file sink
[INFO]  sink.file: file sink configured: path=/workstation/vault101/approleToken
[DEBUG] cache: auto-auth token is allowed to be used; configuring inmem sink

Step 7.1.7

Set the VAULT_AGENT_ADDR environment variable.

$ export VAULT_AGENT_ADDR="http://127.0.0.1:8007"

Step 7.1.8

Create a short-lived token and see how agent manages its lifecycle:

$ VAULT_TOKEN=$(cat approleToken) vault token create -ttl=30s -explicit-max-ttl=2m

Key                  Value
---                  -----
token                s.qaPOodPTUdtbj5REak2ICuyg
token_accessor       Bov810fwIPlp48bENCuW8xv9
token_duration       30s
token_renewable      true
token_policies       ["db_readonly" "default"]
identity_policies    []
policies             ["db_readonly" "default"]

Examine the agent log:

...
[INFO]  cache: received request: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: forwarding request: path=/v1/auth/token/create method=POST
[INFO]  cache.apiproxy: forwarding request: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: processing auth response: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: setting parent context: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: storing response into the cache: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: initiating renewal: path=/v1/auth/token/create method=POST
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create

The request was first sent to VAULT_AGENT_ADDR (agent proxy) and then forwarded to the Vault server (VAULT_ADDR). You should find an entry in the log indicating that the returned token was stored in the cache.

Step 7.1.9

The token's TTL was intentionally set to very short (30 seconds). Examine the agent log to see how it manages the token's lifecycle.

...
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create
[DEBUG] cache.leasecache: secret renewed: path=/v1/auth/token/create
[DEBUG] cache.leasecache: renewal halted; evicting from cache: path=/v1/auth/token/create
[DEBUG] cache.leasecache: evicting index from cache: id=1f9d3e6d037d18f1e91b70be9918f95009433bf585252134de6a41a187e873ee path=/v1/auth/token/create method=POST

Vault Agent renews the token before its TTL was reached until the token reaches its maximum TTL (2 minutes). When the token renewal failed, the agent automatically evicts the token from the cache.

Step 7.1.10

Now, request database credentials for role, "readonly" which was configured in Lab 4.

$ VAULT_TOKEN=$(cat approleToken) vault read database/creds/readonly
Key                Value
---                -----
lease_id           database/creds/readonly/2TW9uVXkMB5oBw1DhtgzQZZb
lease_duration     1h
lease_renewable    true
password           A1a-5ZqdiR8AD5N46Mk6
username           v-approle-readonly-vFmdbjZ1HGXsKsKPTzpa-1552079424

You should find the following entires in the agent log:

...
[INFO]  cache: received request: path=/v1/database/creds/readonly method=GET
[DEBUG] cache.leasecache: forwarding request: path=/v1/database/creds/readonly method=GET
[INFO]  cache.apiproxy: forwarding request: path=/v1/database/creds/readonly method=GET
[DEBUG] cache.leasecache: processing lease response: path=/v1/database/creds/readonly method=GET
[DEBUG] cache.leasecache: storing response into the cache: path=/v1/database/creds/readonly method=GET
...

Step 7.1.11

Re-run the same command and examine the behavior.

$ VAULT_TOKEN=$(cat approleToken) vault read database/creds/readonly

Key                Value
---                -----
lease_id           database/creds/readonly/2TW9uVXkMB5oBw1DhtgzQZZb
lease_duration     1h
lease_renewable    true
password           A1a-5ZqdiR8AD5N46Mk6
username           v-approle-readonly-vFmdbjZ1HGXsKsKPTzpa-1552079424

Exactly the same set of database credentials are returned. The lease_id should be identical as well.

In the agent log, you find the following:

...
[INFO]  cache: received request: path=/v1/database/creds/readonly method=GET
[DEBUG] cache.leasecache: returning cached response: path=/v1/database/creds/readonly

Step 7.1.12

Press Ctrl + C in the first terminal to stop the Vault Agent.

Task 2: Use Consul Template to populate DB credentials

In the Secrets as a Service - Dynamic Secrets lab, you enabled and configured a database secret engine. Assuming that you have an application that needs database credentials, use Consul Template to properly update the application file.

Step 7.2.1

Create a token with db_readonly policy and TTL of 24 hours:

$ vault token create -policy="db_readonly" -ttl=24h

Output should look similar to:

Key                  Value
---                  -----
token                s.uDl7B9jmx4zJwDiEwRIg841D
token_accessor       I7UHnyu2wb8G1TB9YCwGk69M
token_duration       24h
token_renewable      true
token_policies       ["db_readonly" "default"]
identity_policies    []
policies             ["db_readonly" "default"]

This is the token that Consul Template uses to talk to Vault. Copy the token.

NOTE: If you modified VAULT_ADDR in Step 7.1.4, be sure to re-set it back to your local Vault server (export VAULT_ADDR=http://127.0.0.1:8200).

Step 7.2.2

Review the config.yml.tpl file exists on the /workstation/vault101 directory:

$ cat config.yml.tpl
---
{{- with secret "database/creds/readonly" }}
username: "{{ .Data.username }}"
password: "{{ .Data.password }}"
database: "myapp"
{{- end }}

Step 7.2.3

In this case, the input file is the config.yml.tpl. Specify the output file name to be config.yml:

$ VAULT_TOKEN=<token> consul-template -template="config.yml.tpl:config.yml" -once

While <token> is the token you generated at Step 7.2.1.

The -once flag tells Consul Template not to run this process as a daemon, and just run it once.

Step 7.2.4

Open the generated config.yml file to verify its content. It should look similar to:

$ cat config.yml
---
username: "v-token-readonly-tu17xrtz345uz643980r-1527630039"
password: "A1a-7s0z9y223x2rp6v9"
database: "myapp"

The username and password were retrieved from Vault and populated the config.yml file.

Task 3: Use Envconsul to populate DB credentials

Now, use Envconsul to populate the username and password for your application.

Step 7.3.1

View the app.sh file exists on the /workstation/vault101 directory:

$ cat app.sh

#!/usr/bin/env bash

cat <<EOT
My connection info is:

username: "${DATABASE_CREDS_READONLY_USERNAME}"
password: "${DATABASE_CREDS_READONLY_PASSWORD}"
database: "my-app"
EOT

The main difference here is that the app.sh is reading the environment variables to retrieve username and password.

Step 7.3.2

Run the Envconsul tool using the Vault token generated at Step 7.2.1.

$ VAULT_TOKEN=<token> envconsul -upcase -secret database/creds/readonly ./app.sh

My connection info is:

username: "v-token-readonly-ww1tq33s7z5uprpxxy68-1527631219"
password: "A1a-u54wut0v605qwz95"
database: "my-app"

The output should display the username and password generated by Vault.

The -upcase flag tells Envconsul to convert all environment variable keys to uppercase. The default is lowercase (e.g. database_creds_readonly_username).

Step 7.3.3

Run the following command to see the environment variables created by the Envconsul:

$ VAULT_TOKEN=<token> envconsul -upcase -secret database/creds/readonly \
        env | grep DATABASE

DATABASE_CREDS_READONLY_PASSWORD=A1a-6808qrp9t64utw3t
DATABASE_CREDS_READONLY_USERNAME=v-token-readonly-31sq7t64pp2379r55s49-1527631800

If your application is designed to read values from environment variables, Envconsul provides the easiest app integration with Vault.

Challenge: Read Key/Value secrets using Consul Template

You have an application which must retrieve the course, username and password from secret/training path and populate a file such as follow:

student-file.txt

Course: <course>
Username: <username>
Password: <password>
  • Create a template file using Consul Template syntax to read data from secret/training (student-file.tpl)
  • Create a policy, readonly so that Consul Template can read secret/training
  • Create a token with the readonly policy attached
  • Using the generated token, run the consul-template command to read version 5 of the secret/training secret, and populate the student file (student-file.txt)

Hint & Tips

The readonly policy file (readonly.hcl) should look like:

path "secret/data/training" {
   capabilities = [ "read" ]
}

NOTE: Remember from Module 2 that the path for key/value v2 secrets engine is <mount_path>/data/.

Refer back to Step 7.2.1 to create a readonly policy.

Consul Template README provides detailed information: https://github.com/hashicorp/consul-template

Lab 7: Direct App Integration - Challenge Solution

Challenge: Read Key/Value secrets using Consul Template

Step 1: Create a template file

Your template file should look like:

student-file.tpl

{{ with secret "secret/data/training?version=5" }}
Course: {{ .Data.data.course }}
Username: {{ .Data.data.username }}
Password: {{ .Data.data.password }}
{{ end }}

NOTE: If you skipped any of the steps in Lab 2, the secret/training may not have version 5. If that's the case, adjust your template file accordingly.

For example:

{{ with secret "secret/data/training?version=1" }}
Value: {{ .Data.data.value }}
{{ end }}

Step 2: Create a readonly policy

Create a policy file named readonly.hcl as follow:

path "secret/data/training" {
   capabilities = [ "read" ]
}

Execute the following command to create readonly policy:

$ vault policy write readonly ./readonly.hcl

Step 3: Generate a token

Execute the following command to generate a token with readonly policy attached:

$ vault token create -policy=readonly

Step 4: Run Consul Template

Execute the following command to run Consul Template:

$ VAULT_TOKEN=<readonly_token> consul-template \
             -template=student-file.tpl:student-file.txt -once

Verify that the student-file.txt was populated as expected:

$ cat student-file.txt

Course: Vault
Username: student01
Password: pAssw0rd

End of Lab 7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.