Skip to content

Instantly share code, notes, and snippets.

@stuartstevenson
Created September 16, 2014 07:50
Show Gist options
  • Star 12 You must be signed in to star a gist
  • Fork 10 You must be signed in to fork a gist
  • Save stuartstevenson/6a73fb70639486c01341 to your computer and use it in GitHub Desktop.
Save stuartstevenson/6a73fb70639486c01341 to your computer and use it in GitHub Desktop.
Jenkins job dsl with distributed job config files. The seed.groovy file scans stash repo branches for jenkins-job-dsl.groovy files and runs them centrally.
Closure job = {
// Job Name
name "${stashProjectKey}-${projectName}-${branchSimpleName}"
// Where should jenkins run the job
label ('master')
// Where should Jenkins get the source code from
scm {
git {
remote {
url ("<STASH URL>/git/${stashProjectKey}/${projectName}.git")
branch (branchName)
credentials (jenkinsGitCredential)
}
}
}
// How often should the job run
triggers {
scm ('H/10 * * * *')
}
// Gradle build steps to execute
steps {
def gradleTask = '<GRADLE BUILD STEPS>'
gradle (gradleTask, null, true) {
def makeExecutable = it / 'makeExecutable'
makeExecutable.setValue (true)
}
}
// Additional Report Settings
publishers {
jacocoCodeCoverage {
minimumLineCoverage '60'
maximumLineCoverage '90'
execPattern '**/build/**/*.exec'
}
archiveJunit ("**/build/test-results/*.xml", true, true)
}
}
return job
import groovy.json.JsonSlurper
def jenkinsGitCredential = 'JENKINS GIT CREDNETIALS GO HERE'
def jenkinsJobDslFileName = 'jenkins-job-dsl.groovy'
def projectsApi = new URL("<STASH URL>/rest/api/1.0/projects")
def projects = new JsonSlurper().parse(projectsApi.newReader())
projects.get("values").each {
def stashProjectKey = it.get("key")
if (stashProjectKey) {
println "INFO - Searching for repos in project $stashProjectKey"
def reposApi = new URL("<STASH URL>/rest/api/1.0/projects/${stashProjectKey}/repos")
def repos = new JsonSlurper().parse(reposApi.newReader())
if (repos) {
def repoList = repos.get("values")
println "INFO - Found ${repoList.size} repos for project ${stashProjectKey}."
repoList.each { repo ->
processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName)
}
}
}
}
private void processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName) {
def repoSlug = repo.get("slug")
if (repoSlug) {
println "INFO - Searching for branches in repo $repoSlug"
def branchApi = new URL("<STASH URL>/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/branches")
def branches = new JsonSlurper().parse(branchApi.newReader())
branches.get("values").each { branch ->
processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName)
}
}
}
private void processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName) {
def branchName = branch.get("displayId")
def branchSimpleName = branchName.replace("/", "-")
def jobDslUrl = new URL("<STASH URL>/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/browse/${jenkinsJobDslFileName}?raw&at=${branchName}")
println "INFO - Found branch ${repoSlug} - ${branchName}"
println "INFO - Looking for job dsl ${jobDslUrl}"
def urlConnection = (HttpURLConnection) jobDslUrl.openConnection()
def status = urlConnection.getResponseCode()
def repoAndBranchName = "${repoSlug} - ${branchName}"
if (status == 200) {
println "INFO - Found job dsl for " + repoAndBranchName
def dslJson = new JsonSlurper().parse(jobDslUrl.newReader())
if (dslJson.get("lines") != null) {
try {
println "INFO - Executing job dsl for $repoAndBranchName"
Binding binding = getJobDslBinding(
stashProjectKey,
repoSlug,
branchSimpleName,
branchName,
jenkinsGitCredential)
executeJobDsl(binding, dslJson)
} catch (Exception e) {
println "ERROR - Couldn't run job dsl for ${repoAndBranchName}: ${e.getMessage()} \n ${e.printStackTrace()}"
}
}
} else {
println "WARN - No jenkins job dsl found for $repoAndBranchName"
}
}
private void executeJobDsl(Binding binding, dslJson) {
def jobDsl = [];
dslJson.get("lines").each {
jobDsl.add(it.text)
}
def jobDslString = jobDsl.join("\n")
GroovyShell shell = new GroovyShell(binding);
Object value = shell.parse(jobDslString);
job value.run()
}
private Binding getJobDslBinding(stashProjectKey, repoSlug, branchSimpleName, branchName, jenkinsGitCredential) {
Binding binding = new Binding();
binding.setVariable('stashProjectKey', stashProjectKey)
binding.setVariable('projectName', repoSlug)
binding.setVariable('branchSimpleName', branchSimpleName)
binding.setVariable('branchName', branchName)
binding.setVariable('jenkinsGitCredential', jenkinsGitCredential)
binding
}
@legege
Copy link

legege commented Aug 24, 2015

Thanks a lot for that! I made a version for GitLab, see the fork.

@franciscamunhoz
Copy link

Great work! I tried to use your script but I'm getting an authentication error, I'm new to Groovy and job DSL, so I do apologise for my basic questions and I do appreciate your help!
I have setup in credentials my user and password to access stash and also another credential with the private key, on the groovy script I'm passing my credential id (stash-user) for the stash user and password, however I'm getting an authentication error. Do I need to extend it and pass my credentials in any other way?

Error message:

Building in workspace /var/jenkins_home/workspace/seed
Processing DSL script seed.groovy
INFO - Searching for repos in project TEST
FATAL: Server returned HTTP response code: 401 for URL: https://stash.xxxxx.au/rest/api/1.0/projects/TEST/repos
java.io.IOException: Server returned HTTP response code: 401 for URL: https://stash.xxxxx.au/rest/api/1.0/projects/TEST/repos

Here are the groovy scripts:

seed.groovy:

import groovy.json.JsonSlurper

def jenkinsGitCredential = 'stash-user'
def jenkinsJobDslFileName = 'jenkins_job_dsl.groovy'
def stashProjectKey = 'TEST'

if (stashProjectKey) {
println "INFO - Searching for repos in project $stashProjectKey"
def reposApi = new URL("https://stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos")
def repos = new JsonSlurper().parse(reposApi.newReader())

if (repos) {
    def repoList = repos.get("values")

    println "INFO - Found ${repoList.size} repos for project ${stashProjectKey}."

    repoList.each { repo ->
        processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName)
    }
}

}


I have also tried passing the user and password on the api url but it did not work on the groovy script, but use curl from my jenkins server the url does works, I'm using a jenkins docker image.

def reposApi = new URL("https://francisca:password@stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos")

Error:

Processing DSL script seed.groovy
INFO - Searching for repos in project TEST
FATAL: Server returned HTTP response code: 401 for URL: https://francisca:password@stash.xxxxx.au/rest/api/1.0/projects/TEST/repos
java.io.IOException: Server returned HTTP response code: 401 for URL: https://francisca:password@stash.xxxxx.au/rest/api/1.0/projects/TEST/repos
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at org.codehaus.groovy.runtime.ResourceGroovyMethods.configuredInputStream(ResourceGroovyMethods.java:2021)
at org.codehaus.groovy.runtime.ResourceGroovyMethods.newReader(ResourceGroovyMethods.java:2070)
at org.codehaus.groovy.runtime.dgm$983.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at seed.run(seed.groovy:10)
at seed$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at seed$run.call(Unknown Source)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment