- Table of Contents
- Setup Jenkins Locally in Docker
- Jenkins Job
- CLI Configuration
- Trigger Parametrized Jenkins Build
- Installing Vault
- Using Vault in Jenkins
- References
run
docker run -d -p 49001:8080 -v $PWD/jenkins:/var/jenkins_home:z -t jenkins/jenkins:lts
Jenkins will be accessible after a while on http://localhost:49001/
Sample Jenkins Job:
properties([
parameters([
string(name: 'cfuser', defaultValue: 'Jenkins', description: "CF user"),
password(name: 'cfpassword', description: "CF password"),
booleanParam(name: 'deploy', description: 'Really deploy?')
])
])
node {
stage('Echo Stage') {
echo params.cfpassword.getPlainText()
echo params.cfuser
echo params.deploy.toString()
}
}
-
Given, you have Jenkins running in Docker, download CLI from http://localhost:49001/nlpJars/jenkins-cli.jar
-
Copy the
jenkins-cli.jar
into~/jenkins/
-
Create auth token in http://localhost:49001/user/admin/configure
-
Create a shell alias:
alias jc="java -jar ~/jenkins/jenkins-cli.jar -s http://localhost:49001 -auth admin:11278ecdcddb512c7da2a91f2683a541e2"
-
Check whether everything is working via
jc help
The jenkins cli running over http requires buffering turned off in the nginx reverse proxy (see https://issues.jenkins-ci.org/browse/JENKINS-43666):
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
# try_files $uri $uri/ =404;
include /etc/nginx/proxy_params;
proxy_pass http://localhost:8080;
proxy_read_timeout 90s;
# Fix potential "It appears that your reverse proxy set up is broken" error.
proxy_redirect http://localhost:8080 https://jenkins.tld.com;
# Required for new HTTP-based CLI
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
}
Run
jc build 'cli-test' -p cfuser=lalalauser -p cfpassword=lalapassword -p deploy=true -s -v
to trigger the before-mentioned job with the two parameters.
Run
brew install https://raw.githubusercontent.com/petems/homebrew-vault-prebuilt/master/Formula/vault.rb
vault -autocomplete-install
# restart your shell
we use https://github.com/petems/homebrew-vault-prebuilt since the default homebrew vault
offering does not contain the UI.
Create a config file ~/vault/vault.config
with
ui = true
storage "file" {
path = "/Users/USERNAME/vault/data"
}
listener "tcp" {
address = "127.0.0.1:8200"
tls_disable = 1
}
Start the vault server with the given configuration via
vault server -config ~/vault/vault.config
You can now access the vault UI via http://localhost:8200/ui
Run
export VAULT_ADDR='http://127.0.0.1:8200'
check Vault status
vault status
See https://www.monterail.com/blog/2017/lets-encrypt-vault-free-ssl-tls-certificate
Since we do not have a public domain for Vault, we cannot use letsencrypt
- instead create your own self-signed certificate for vault.localhost
:
openssl req -x509 -out vault.localhost.crt -keyout vault.localhost.key \
-newkey rsa:2048 -nodes -days 712 -sha256 \
-subj '/CN=vault.localhost' -extensions EXT -config <( \
printf "[dn]\nCN=vault.localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:vault.localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
Move vault.localhost.crt
and vault.localhost.key
to ~/vault/
mv vault.localhost.* ~/vault/
Add the certificate to the system's keystore:
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/vault/vault.localhost.crt
Add a hosts entry in /etc/hosts
127.0.0.1 vault.localhost
Configure vault server to use the certificates - within ~/vault/vault.config
:
listener "tcp" {
address = "vault.localhost:8200"
tls_cert_file = "/Users/USER/vault/vault.localhost.crt"
tls_key_file = "/Users/USER/vault/vault.localhost.key"
}
Restart vault, via CTRL+C in the terminal window running vault server and then running
vault server -config ~/vault/vault.config
You can now access your Vault UI via https://vault.localhost:8200
See https://blog.alanthatcher.io/fun-and-profit-with-vault-part-2/
Within /Library/LaunchDaemons/
create a new file com.hashicorp.vault.plist
:
vi /Library/LaunchDaemons/com.hashicorp.vault.plist
Add the following content:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key> <string>com.hashicorp.vault</string>
<key>Disabled</key> <false/>
<key>RunAtLoad</key> <true/>
<key>KeepAlive</key> <false/>
<key>LaunchOnlyOnce</key> <true/>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/vault</string>
<string>server</string>
<string>-config</string>
<string>/Users/USER/vault/vault.config</string>
</array>
</dict>
</plist>
You can test your configuration via
sudo launchctl load -w /Library/LaunchDaemons/com.hashicorp.vault.plist
Write a vault config script
vi /Users/USERS/vault/user-settings.sh
with the following content
export VAULT_ADDR=https://vault.localhost:8200
source it in your shell startup script (e.g. ~/.zshrc
):
. ~/vault/user-settings.sh
Use the following command to re-generate the SSL certificates:
openssl req -x509 -out vault.localhost.crt -keyout vault.localhost.key \
-newkey rsa:2048 -nodes -sha256 \
-subj '/CN=vault.localhost' -extensions EXT -config <( \
printf "[dn]\nCN=vault.localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:vault.localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
Copy the certificates to ~/vault/
via mv vault.localhost.* ~/vault/
. Restart Vault.
-
Stop Vault via
sudo launchctl unload -w /Library/LaunchDaemons/com.hashicorp.vault.plist
-
Start Vault via
sudo launchctl load -w /Library/LaunchDaemons/com.hashicorp.vault.plist
See https://www.reddit.com/r/1Password/comments/8zq79d/how_to_get_password_for_use_in_applescript/
Here is a hack for unsealing Vault with a password that was previously stored in 1Password with the name Vault Unseal (Master Key)
#!/usr/bin/env bash
vault_unseal_name='Vault Unseal (Master Key)'
osascript <<-END
on run
tell application "System Events" to tell process "1Password mini"
open location "onepassword://extension/search/${vault_unseal_name}"
delay 0.2
keystroke "C" using {command down, shift down}
set unseal_key to the clipboard as text
end tell
do shell script "vault operator unseal " & unseal_key
end run
END
For enabling Jenkins to access Vault we will use an app role.
A description of what needs to be done to setup the app role can be found here.
-
Create a new KV store in Vault for our secrets:
vault mount -path=jenkins-secrets kv
You can check whether this worked with
vault mounts
-
For testing, put a secret in that newly created store:
vault write jenkins-secrets/test-secret value='my-secret-value'
You can check whether this worked with
vault kv get jenkins-secrets/test-secret
-
Generate a policy
jenkins-ro
(for read-only) for the jenkins role in Vault:echo 'path "jenkins-secrets/*" { capabilities = ["read", "list"] }' | vault policy-write jenkins-ro -
You can check whether this worked with
vault read /sys/policy/jenkins-ro
-
Create a Vault token with the afore-created policy assigned:
vault token create -policy="jenkins-ro" Key Value --- ----- token d9a343cf-a559-ebfc-a702-60c57faf4e7c token_accessor 3aa9c348-2797-89d7-18cf-ab2f019e1540 token_duration 768h token_renewable true token_policies ["default" "jenkins-ro"] identity_policies [] policies ["default" "jenkins-ro"]
Next, try to authenticate with the token created before:
vault auth d9a343cf-a559-ebfc-a702-60c57faf4e7c
And read the
jenkins-secret
with the applied role, this should work:vault read jenkins-secrets/test-secret Key Value --- ----- refresh_interval 768h value2 another-secret-value
-
Re-authenticate with an Vault admin user
vault auth <ADMIN TOKEN>
-
Enable the
app-role
as authentication backend:vault auth-enable approle
-
Now create an app role for jenkins:
vault write auth/approle/role/jenkins-ro secret_id_ttl=1m secret_id_num_uses=1 token_num_uses=3 token_ttl=10m token_max_ttl=30m policies=jenkins-ro
You can check whether everything works as expected via
vault read auth/approle/role/jenkins-ro
-
Get the internal ID of the app role:
export ROLE_ID=$(vault read -field role_id auth/approle/role/jenkins-ro/role-id)
-
Now generate a secret ID for the app role:
export SECRET_ID=$(vault write -field secret_id -f auth/approle/role/jenkins-ro/secret-id)
-
Login using the app role and afore created
ROLE_ID
andSECRET_ID
(you have thesecret_id_ttl
time to do so after you created theSECRET_ID
):
export APP_ROLE_TOKEN=$(vault write -field token auth/approle/login role_id=${ROLE_ID} secret_id=${SECRET_ID})
- You can now check the generated token via:
vault login $APP_ROLE_TOKEN
vault read jenkins-secrets/test-secret
Key Value
--- -----
refresh_interval 768h
value my-secret-value
The tricky thing to understand is that we use the app role to generate a token that we then use for authentication. We do not directly authenticate to Vault using the app role's secret id.
Here is an example of a Jenkinsfile
that reads secrets from Vault:
node {
stage('Reading from Vault') {
sh 'curl -o vault.zip https://releases.hashicorp.com/vault/0.11.3/vault_0.11.3_linux_amd64.zip ; yes | unzip vault.zip'
sh '''
# set to +x later on to avoid log output containing sensitive data
set -ex
export VAULT_ADDR='https://docker.for.mac.localhost:8200'
export VAULT_SKIP_VERIFY=true
## needs to be the token generated for being able to create the secret_id
## we want to query this via a input field...
export VAULT_TOKEN='e0c4dc53-369b-8d3e-09db-cf34a4610fc4'
## the id of the 'jenkins-ro' role
export ROLE_ID='949be848-5368-ffac-b4fb-c61288ca9d6b'
export SECRET_ID=$(./vault write -field=secret_id -f auth/approle/role/jenkins-ro/secret-id)
export VAULT_TOKEN=$(./vault write -field=token auth/approle/login role_id=${ROLE_ID} secret_id=${SECRET_ID})
export TEST_VAR=$(./vault kv get -field=-value jenkins-secrets/test-secret)
#echo $TEST_VAR
'''
}
}