Skip to content

Instantly share code, notes, and snippets.

@kawsark
Last active June 29, 2023 14:39
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save kawsark/5379eada694915e15d84547169a62aa1 to your computer and use it in GitHub Desktop.
Save kawsark/5379eada694915e15d84547169a62aa1 to your computer and use it in GitHub Desktop.
Example Jenkins integration for Vault using AppRole and curl

Example Jenkins integration for Vault

This snippet provides an example Jenkinsfile that performs an AppRole authentication using curl utility. The objective is to allow Jenkins to Authenticate to Vault, then use a temporary token to retrieve a secret. It does not rely on a plugin and therefore offers more flexibility.

AppRole authentication relies on a ROLE_ID and SECRET_ID to login and retrieve a Vault token. There are two ways to provide the SECRET_ID to Jenkins. Both of these are expanded upon below.

  1. Pre-created SECRET_ID as a Jenkins secret. An out-of-band workflow will need to refresh the SECRET_ID periodically so Jenkins continues to perform AppRole logins successfully.
  2. Alternative AppRole design: Give Jenkins the ability to refresh the SECRET_ID by itself.

1. Pre-created Secret ID

Vault setup

Please use commands below to create the AppRole Auth method, define an App role, and retrieve the Role ID and Secret ID.

  • In this example, the SECRET_ID is limited to a TTL of 24 hours (secret_id_ttl) and a limit of 40 uses (secret_id_num_uses). Hence the value will need to be updated periodically, otherwise, you will get an error message: "invalid secret id".
  • The resulting Vault token can be used 5 times (token_num_uses=5). This will need to be adjusted if you have additional stages in the Pipeline where you will continue to use the same token.
vault secrets enable -path=secrets kv
vault write secrets/creds/dev username=dev password=legos
cat <<EOF > jenkins-policy.hcl
path "secrets/creds/dev" {
 capabilities = ["read"]
}
EOF
vault policy write jenkins jenkins-policy.hcl
vault auth enable approle
vault write auth/approle/role/jenkins-role \
    secret_id_ttl=24h \
    token_num_uses=5 \
    token_ttl=20m \
    token_max_ttl=30m \
    secret_id_num_uses=40 \
    policies="jenkins"

# Use .data.role_id in role.json file as the ROLE_ID for Jenkins setup
vault read -format=json auth/approle/role/jenkins-role/role-id > role.json

# Use .data.secret_id in secretid.json file as the SECRET_ID for Jenkins credential
vault write -format=json -f auth/approle/role/jenkins-role/secret-id > secretid.json

Jenkins setup

We will create a new Jenkins pipeline project to demonstrate Vault interaction. Note: you will need curl and jq utilities installed on your Jenkins server/worker node.

  • Please adjust the values of VAULT_ADDR to your Vault server.
  • Adjust the ROLE_ID from the output of your vault write auth/approle/role/jenkins-role ... command.
  • Please import the Jenkinsfile snippets below into a new or existing Pipeline stage.
  • Define a SECRET_ID in Jenkins credentials of type "Secret Text," The value will come from Vault Setup above.
  • Adjust other variables appropriately: VAULT_ADDR,ROLE_ID and SECRETS_PATH.
  • Run the Pipeline project
pipeline {
agent any
    environment {
        VAULT_ADDR="http://35.194.95.200:80"
        ROLE_ID="c4cec819-eae2-ca98-b312-46fcfe322c7c"
        SECRET_ID=credentials("SECRET_ID")
        SECRETS_PATH="secrets/creds/dev"
    }

    stages {     
      stage('Stage 0') {
          steps {
            sh """
            export PATH=/usr/local/bin:${PATH}
            # AppRole Auth request
            curl --request POST \
              --data "{ \"role_id\": \"$ROLE_ID\", \"secret_id\": \"$SECRET_ID\" }" \
              ${VAULT_ADDR}/v1/auth/approle/login > login.json

            VAULT_TOKEN=$(cat login.json | jq -r .auth.client_token)
            # Secret read request
            curl  --header "X-Vault-Token: $VAULT_TOKEN" \
              ${VAULT_ADDR}/v1/${SECRETS_PATH} | jq .
            """
          }
      }
    }
}

Testing

Upon running the pipeline successfully you should see an output such as below.

{
  "request_id": "38e4ff71-fea3-837e-b4fd-9d19050826d7",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 2764800,
  "data": {
    "password": "legos",
    "username": "dev"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

You have now setup Jenkins server to Authenticate with Vault and retrieve a secret.

2. Alternative AppRole design

This snippet provides an example set of commands that performs an AppRole authentication using vault CLI. In this workflow, the client process is given the permission to refresh SECRET_ID by itself. The objective is to allow automatic generation of SECRET_ID.

Initial Vault setup by Admin

vault login <admin-or-root-token>

vault policy write jenkins -<<EOH
path "auth/approle/role/jenkins/secret-id" { capabilities = ["create","update"] }
path "ssh/sign/ansible" { capabilities = ["read","create","update"] }
EOH

# Note the token_bound_cidrs and secret_id_num_uses is limited for Jenkins
vault write auth/approle/role/jenkins \
    secret_id_ttl=24h \
    token_num_uses=10 \
    token_ttl=48h \
    token_max_ttl=48h \
    secret_id_num_uses=1 \
    policies=jenkins
    token_bound_cidrs="1.1.1.1/32"

vault read auth/approle/role/jenkins/role-id
vault write -f auth/approle/role/jenkins/secret-id

Jenkins pipeline job that runs on a 24 hours schedule

# 1. Read secret id
vault write auth/approle/login \
    role_id=<role-id> \
    secret_id=<secret-id>

# Alert on Login failure!
# 2. Inject resulting Vault_token to Ansible

# 3. Generate new secret_id
vault write -f auth/approle/role/jenkins/secret-id
vault write auth/approle/login \
    role_id=<role-id> \
    secret_id=<new-secret-id>

# 4. Persist new secret id
@Alex-N-16
Copy link

Hi Kawsar,

I followed the Jenkins integration from your code but for some reason it looks like Jenkins is not interpreting curl correctly.

stage('Stage 0') {
steps {
sh """
# AppRole Auth request
curl --request POST
--data "{ "role_id": "$ROLE_ID", "secret_id": "$SECRET_ID" }"
${VAULT_ADDR}/v1/auth/approle/login > login.json
cat login.json

{"errors":["failed to parse JSON input: invalid character 'r' looking for beginning of object key string"]}

During your pipeline development have you encountered this issue? It looks like groovy interpolation and double quote escaping is not working properly and I'm using the latest Jenkins version.

Best regards,
Alex

@kawsark
Copy link
Author

kawsark commented Mar 15, 2021

Hello @Alex-N-16,
Interesting, it could be due to a Jenkins version difference. I posted that Gist some time ago so its possible the newer Jenkins versions interpreting the syntax differently. I will need some time to try it out on a newer version. Could you please reference this repo for syntax? Its definitely more recent: https://github.com/dcanadillas/jenkinsfile-vaultplugin/blob/master/Jenkinsfile

@kawsark
Copy link
Author

kawsark commented Mar 15, 2021

Hello @Alex-N-16, Please also see this Jenkinsfile which uses the native groovy syntax as opposed to bash: https://github.com/assareh/terraform-jenkins-pipeline/blob/master/scripts/Jenkinsfile

@saharshjain
Copy link

Hey I also faced the parsing issue. Finally got it working after sometime. Please use below code as an example

pipeline {
    agent any
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
        disableConcurrentBuilds()
    }
    stages{   
        stage("Write config"){
            environment{
                VAULT_URL=credentials('VAULT_URL')
                VAULT_ROLE_ID=credentials('VAULT_ROLE_ID')
                VAULT_SECRET_ID=credentials('VAULT_SECRET_ID')
                SECRET_PATH='<your path to secret>'
                LOGIN_PATH='<you url for login>'
            }
            steps{
                sh '''
                export PATH=/usr/local/bin:$PATH
                LOGIN_PATH=$VAULT_URL$LOGIN_PATH
                SECRET_PATH=$VAULT_URL$SECRET_PATH
                curl --request POST --data '{ "role_id": "\'$VAULT_ROLE_ID\'", "secret_id": "\'$VAULT_SECRET_ID\'" }' $LOGIN_PATH > login.json
                VAULT_TOKEN=$(cat login.json | jq -r .auth.client_token)
                curl  --request GET --header "X-Vault-Token: $VAULT_TOKEN" $SECRET_PATH > envVariables.json
                ENV_VARS=$(cat envVariables.json | jq -r .data.data)
                echo $ENV_VARS
                '''
            }
        }
    }
}

Hope it helped!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment