A way to automatically populate one or more GitHub Organizations on a Jenkins instance

scan-organizations.adoc

A way to automatically populate one or more GitHub Organizations on a Jenkins instance

This document contains the way that I solved a specific issue. The issue was to populate one or more GitHub Organizations on a Jenkins instance upon initial start up.

Back story

To ensure compliance with audit, security, maintenance, and general sanity, my team is tasked with ensuring our 200+ developer have access to clean and up to date Jenkins instances.

The way we solve this is by having a locally developed portal that acts as an order form. Basically any developer can order an instance by supplying the request name of the instance and a list of one or more GitHub Organizations (from our internal GitHub Enterprise) they wish to use on the instance.

Thanks to the wonders of containerization the requested instance is delivered, ready to run within a few minutes.

To ensure some kind of integrity, we do not allow regular users any magical powers on the instances, but the are allowed to log in, and do some minor tinkering like creating views and such. To keep track of users we implemented the LDAP-plugin with our Windows Domain controllers on the backend. At some point a few developers asked us to implement Single Sign On instead, sp the didn’t actively need to log in. Mulling over this for some time, we finally found a plugin that could suit our need for "dynamic" endpoints.

The issue at hand

To create a user token, you need a valid crumb. To get a crumb you need to send username and password to the Jenkins instance for verification of the user credentials. This works using a local user database or the LDAP plugin. It does not work with the Azure AD plugin. The reason being that the local instance does not validate the username/password supplied. It is immediately shipped of the the crendetials provider, which in this case is the Azure AD plugin. Since no validation occurs on the Jenkins instance, there is no crumb to be created. With no valid crumb, no user token can be delivered in the response.

The solution

Since we already utilize the power of groovy in a seed job to create different stuff on all instances, we expanded this seed job to include the script scan-organizations.sh to ensure the initial scan of the GitHub Organizations that were requested upon creation of the image.

Using the Jenkins plugins Configuration as Code, Groovy Postbuild Plugin, Job DSL Plugin, and Startup Trigger to build the necessary environment a Jenkins instance to make it work.

The components

seed job

The seed-job.yaml is a part of our Configuration as Code setup, and contains the seed-job itself. By having the Groovy Postbuild Plugin installed, we can utilize the power of "Jenkins Groovy" and by that we have access to basically everything the Jenkins instance has to offer, including access to the local bits of the domain users. Thus giving us a possibility to create a token without having to do a http-request which requires a valid crumb.

scan-organizations.sh

The scan-organizations.sh script is simple in its make up. Obviously it needs a username [$USER], a token [$TOKEN], a target host [$VIRTUAL_HOST], and a list of organizations to scan [$GIT_SEED].

Inspiration for this gist comes from timja jenkinsci/azure-ad-plugin#241 (comment)

scan-organizations.sh

#!/bin/bash
USER=<user>@<domain>
TOKEN=$1
ORGANIZATIONS=$(echo "$GIT_SEED" | xargs | sed 's/,/ /g')

echo "ORGANIZATIONS: $ORGANIZATIONS"

for ORGANIZATION in $ORGANIZATIONS
do
  echo "Initiating scan of $ORGANIZATION"
  curl -X POST -u "$USER:$TOKEN" "https://$VIRTUAL_HOST/job/$ORGANIZATION/build" 
done

seed-job.yaml

---
jobs:
  - script: >
      freeStyleJob('seed') {
        label("built-in")
        triggers {
          hudsonStartupTrigger {
            quietPeriod("0")
            runOnChoice("ON_CONNECT")
            label("")
            nodeParameterName("")
          }
        }
        scm {
          ### Define stuff
        }
        steps {
          dsl {
            ### Create some jobs
          }
        }
        publishers {
          groovyPostBuild("""import hudson.model.*
        import jenkins.model.*
        import jenkins.security.*
        import jenkins.security.apitoken.*
    
        def userName = '<user>@<domain>'
        def tokenName = 'apiToken'
    
        def user = User.get(userName, false)
        def apiTokenProperty = user.getProperty(ApiTokenProperty.class)
        apiTokenProperty.revokeToken(tokenName)
        def result = apiTokenProperty.tokenStore.generateNewToken(tokenName)
        user.save()
    
        String apiToken = result.plainValue
    
        new ProcessBuilder( 'sh', '-c', 'curl -o /tmp/scan-organizations.sh --url https://<raw.github.domain>/<organization>/<repository>/<branch>/scan-organizations.sh && chmod +x /tmp/scan-organizations.sh && cd /tmp/ && ./scan-organizations.sh ' + apiToken).redirectErrorStream(true).start().text
    
        apiTokenProperty.revokeToken(tokenName)""", Behavior.MarkFailed)
        }
      }