vault-on-aws-with-s3

# HOWTO: Installing Vault On AWS with S3 backend

This is a HOWTO on installing [Vault](https://www.vaultproject.io/https://github.com/hashicorp/vault) on AWS with S3.  

Components used:
* [Vault](https://www.vaultproject.io/https://github.com/hashicorp/vault)
* [AWS S3](https://aws.amazon.com/s3/)
* [AWS IAM](https://aws.amazon.com/iam/)
* [AWS EC2](https://aws.amazon.com/ec2/)
* [OpenSSL](https://www.openssl.org/)
* [Supervisord](http://supervisord.org/)


## Setting up S3
We'll setup an S3 bucket to use as a Vault backend:

1. From the AWS Mangement Console, go to the S3 console.

2. Click on the `Create Bucket` button

3. Name it


## IAM 
Next, create an IAM Policy with full access to the S3 bucket.

1. From the AWS Management Console, go the IAM console.

2. Click on **Policies**

3. Click on **Create Policy**

4. Select the **Policy Generator** option.

5. Select **Amazon S3** from the *AWS Service* dropdown

6. Select **All Actions (*)** from the *Actions* dropdown

7. Enter the **Amazon Resource Name**: `arn:aws:s3:::<your_bucket_name>`

8. Click **Add Statement**

9. Next, repeat steps 5-8, except use the following **ARN**: `arn:aws:s3:::<your_bucket_name>/*` *(this is required to let vault manage all keys within the bucket)*

10. Click **Next Step**

11. Give the policy a name: `s3-vault-full-access`

12. Click `Add Policy`

Next, we create an **IAM Role** and attach our policy to it. We will use this role as the **EC2** instance role later on.
1. Click on **Roles** in the side nav

2. Click **Create New Role**

3. Give the role a name: `vault-ec2`

4. Click `Next Step`

5. Under **Select Role Type**, select **Amazon EC2** from the *AWS Service Roles* section

6. Attach our newly created `s3-vault-full-access` policy to the role and click **Next Step**

7. Lastly, review our role, and click **Create Role**

Lastly, due to a bug in the current version of vault (v4.1), we create a new user and assign the policy to it. We will then generate access keys for this user to use when initializing vault.
1. Click on **Users** from the side nav

2. Click on **Create New Users**

3. Enter a username: `vault` and click **Create**

4. Save and download the security credentials on the next screen

5. Back to the **Users** screen, and click on our newly created user

6. Under the **Permissions** tab, click **Attach Policy**

7. Select the `s3-vault-full-access` policy and attach it.


## Launch EC2
Ok, now it's time to launch an ec2 that will act as our Vault server.
1. From the AWS Management Console, go to the EC2 console.

2. Click **Launch Instance**

3. Select the most recent **Amazon Linux AMI**, usually at the top

4. Select an appropriate size, for this tutorial, I'll use a t2.nano

5. Click **Next: Configure Instance Details**

6. Under **IAM role**, select the **IAM Role** we created earlier (`vault-ec2`)

7. Click **Next: Add Storage**

8. The default storage is fine, so click **Next: Tag Instance**

9. Give your instance a name tag: `vault`

10. Click **Next: Configure Security Group**

11. Give your security group a name: `vault`

12. Give your security group a description: `vault server security group`

13. Click **Add Rule**

14. Select **Custom TCP Rule** and define a port range: `8200`

15. Under source, for the purposes of this tutorial, select **My IP**. However, in production, you should restrict this port to the security groups of the servers that require access to vault.

16. Click **Review and Launch**

17. Click **Launch**

18. If you have an existing key-pair, you can use it, or create a new one and download it

19. Lastly, click **Launch Instance** and then **View Instances**


## Generating self-signed certificate for vault
Now, we're going to generate a self-signed certificate to use with Vault.

1. Create a directory to hold the ssl stuff
```bash
mkdir .ssl && cd .ssl
```

2. Generate a private key, remember the password you use
```bash
openssl genrsa -aes256 -out server.key 2048
```

3. Generate a CSR (Certificate Signing Request). This will prompt you to enter some details, go ahead and skip the challenge password part by pressing `enter`.
```bash
openssl req -new -key server.key -out server.csr
```

4. Remove passhprase from key
```bash
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
```

5. Generate a Self-Signed Certificate
```bash
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
```

6. Minor cleanup, discard the temporary key file
```bash
rm server.key.org && cd
```


## Installing Vault
Once the instance has finished initializing, it's time to download the Vault binary and unpack it.
1. ssh into the ec2 instance
```bash
ssh -i <path/to/key.pem> ec2-user@<ec2-dns>
```

  *note:* if this is a new key, you may receive a permission denied error, in which case, modify the key permissions and try again.
  ```bash
  chmod 0700 <path/to/key.pem>
  ```
  
2. update the instance
```bash
sudo yum update
```

3. install Vault *(find the latest binary on the vault project page)*
```bash
wget https://releases.hashicorp.com/vault/0.6.0/vault_0.6.0_linux_amd64.zip
```

4. unzip it
```bash
unzip vault_0.6.0_linux_amd64.zip
```

5. move the binary
```bash
sudo mv vault /usr/local/bin/vault
```

6. verify that Vault is ready to go
```bash
vault version
```

7. create the vault configuration file
```bash
touch vault-config.hcl
```

8. edit the file
```bash
vim vault-config.hcl
```

9. define the vault configuration like so
```bash
listener "tcp" {
    address = "0.0.0.0:8200"
    tls_cert_file="/home/ec2-user/.ssl/server.crt"
    tls_key_file="/home/ec2-user/.ssl/server.key"
}

backend "s3" {
  bucket = "foldername"
  access_key = "xxxxxxxxxxxx"
  secret_key= "xxxxxxxxxxxxxxxxxxxxxxx"
}

disable_mlock=true
```

10. Save the file


## Installing supervisord
Next, we install [supervisord](http://supervisord.org/), to simplify running Vault as a service.

1. Install `supervisor`
```bash
apt-get install supervisor -y
```

2. Edit the supervisord configuration file
```bash
vim /etc/supervisor/supervisord.conf 
```

3. modify the configuration file
    - under `[unix_http_server]`
        - change `;chmod=0700` to `chmod=0766`
    - add the following: `[program:vault]`
        - add `command=vault server -config=/home/ec2-user/vault-config.hcl`
        - add `user=ec2-user`
        - add `environment=AWS_ACCESS_KEY_ID="<your_access_key_id",AWS_SECRET_ACCESS_KEY="<your_secret_access_key>"`, where `<your_access_key_id>` and `<your_secret_access_key>` are the credentials you downloaded/wrote down when we created the `vault` user.


## Configuring supervisord
Lastly, we need to configure `supervisord` to start on init.

1. create a new init script
```bash
sudo touch /etc/init.d/supervisord
```

2. edit the file contents:
```bash
#!/bin/sh
# Amazon Linux AMI startup script for a supervisor instance
#
# chkconfig: 2345 80 20
# description: Autostarts supervisord.

# Source function library.
. /etc/rc.d/init.d/functions

supervisorctl="/usr/bin/supervisorctl"
supervisord="/usr/bin/supervisord"
name="supervisor-python"

[ -f $supervisord ] || exit 1
[ -f $supervisorctl ] || exit 1

RETVAL=0

start() {
     echo -n "Starting $name: "
     $supervisord -c /home/ec2-user/supervisord.conf
     RETVAL=$?
     echo
     return $RETVAL
}

stop() {
     echo -n "Stopping $name: "
    $supervisorctl shutdown
     RETVAL=$?
     echo
     return $RETVAL
}

case "$1" in
         start)
             start
             ;;

         stop)
             stop
             ;;

         restart)
             stop
             start
             ;;
esac

exit $REVAL
```

3. make the init script executable
```bash
sudo chmod +x /etc/init.d/supervisord
```

3. add the supervisor init script to chkconfig services
```bash
sudo chkconfig --add supervisord
```

4. start the supervisord service
```bash
sudo service supervisord start
supervisorctl
```

yum and apt-get at the same server? really?