mikegreen

The purpose of this document is to provide consideration to security, operational and support tasks and conditions of a production-ready / mission-critical Vault deployment.
This is a living document, please feel free to suggest changes and have someone take a review and approve. You might also want to see this doc from Julia that was done for pre-renewal health checking. 

Infrastructure Security
Are servers provisioned via a build/codified pipeline?
Can staff login (SSH/Console/etc) to individual servers?
Is all traffic in/out of server encrypted?
Is the cluster subnet firewalled from other network resources?
If a server is destroyed/lost, are logs and events available post-mortem?
Is root token creation restricted? Monitored? 

mikegreen

$ vault secrets list  --detailed
Path                  Plugin       Accessor              Default TTL    Max TTL    Force No Cache    Replication    Seal Wrap    External Entropy Access    Options                                                            Description                                                    UUID
----                  ------       --------              -----------    -------    --------------    -----------    ---------    -----------------------    -------                                                            -----------                                                    ----
cubbyhole/            cubbyhole    cubbyhole_fcbf5e6d    n/a            n/a        false             local          false        false                      map[]                                                              per-token private secret storage                               11a09df9-8ef6-bf26-2cc1-d4c6424c4780
database/             database     database_c7158e73     system         system     fal

mikegreen

-- sample row
SELECT FIRST CONFNUM, [INTERN NUM], RESERVESTATUS, CHECKIN, CHECKOUT, NIGHTS, BEDTYPE,
                                        [RATE REQ], GROUPCODE, EMAIL, PREFIX, LASTNAME, FIRSTNAME, STREET, CITY, STATE,
                                        ZIPCODE, COUNTRY, PHONE1, ADULTS, CHILD, ROOMRATE, ENDRATE, VIPNUMBER,
                                        HOUSENOTE, NOC, MADEBY, ENTEREDBY, ENTRYDATE, ENTRYTIME,
                                        CAST(CAST(ENTRYDATE AS VARCHAR(20)) + ' ' + CAST(ENTRYTIME AS VARCHAR(20)) AS DATETIME) as ENTRYTIMESTAMP,
                                        CHANGEDATE, CHANGETIME,
                                        CAST(CAST(CHANGEDATE AS VARCHAR(20)) + ' ' + CAST(CHANGETIME AS VARCHAR(20)) AS DATETIME) as CHANGETIMESTAMP,
                                        CXLNUM,
                                        CASE

mikegreen

# drafty custom-vault.py 
# needs a corresponding /etc/datadog-agent/conf.d/custom-vault.yaml with `instances: [{}]` only
# see https://datadoghq.dev/integrations-core/base/api/#datadog_checks.base.checks.base.AgentCheck.count
# https://github.com/DataDog/integrations-core/tree/master/vault/datadog_checks/vault 

import requests

# the following try/except block will make the custom check compatible with any Agent version
try:
    # first, try to import the base class from new versions of the Agent...

mikegreen

# This should be run as root
# this creates a self-signed certificate with the IP addresses as IP SANS
# Todo - see if we want to roll this into https://github.com/hashicorp/linux-packaging/blob/master/products/vault/scripts/postinst

# dump interfaces to file to work with
ifconfig > ifconfig.txt

# Get number of IPs returned, excluding the ip6 loopback for now
grep "inet" ifconfig.txt | grep -v "::1" | awk -F' ' '{print $2}' | wc -l

mikegreen

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

provider "aws" {

mikegreen

# started/using https://github.com/mbround18/valheim-docker

:/opt/docker-configs/valheim-server$ cat docker-compose.yml 
version: "3"
services:
  valheim:
    image: mbround18/valheim:latest
    restart: unless-stopped
    ports:
      - 2456:2456/udp

mikegreen

# 2020-06-23 
# this shows creating a Vault instance running integrated storage/raft, 
# then adding a KV and taking a snapshot
# then kill the raft DB files to simulate a storage failure
# repeat new Vault instance, restore snapshot, unseal and auth with orig keys
# and read some data to show how backup/restore works

# not meant to be a live script to run!
# this uses the vault_config.hcl from https://gist.github.com/mikegreen/c2df5eea2283f0dbc5f3a5d3650536fd 

mikegreen

### Keybase proof

I hereby claim:

  * I am mikegreen on github.
  * I am mikegreen13 (https://keybase.io/mikegreen13) on keybase.
  * I have a public key ASCaP9tqP5U_C0bqZ3xj1G2koYY0uMvuxojlIMyEbQOI2wo

To claim this, I am signing this object:

mikegreen

{
  "data": {
    "id": "polchk-K7hoj4fHrp17TZw9",
    "type": "policy-checks",
    "attributes": {
      "result": {
        "result": true,
        "passed": 2,
        "total-failed": 0,
        "hard-failed": 0,