Skip to content

Instantly share code, notes, and snippets.

@mocon

mocon/jenkins_notes.md

Last active December 14, 2016 22:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mocon/9766548af17453b9dd86810127a1481d to your computer and use it in GitHub Desktop.
Save mocon/9766548af17453b9dd86810127a1481d to your computer and use it in GitHub Desktop.
Download ZIP
Notes from Jenkins conference on 12/14/16
Raw
jenkins_notes.md

Seven habits of highly effective Jenkins

1. Have a reproducable environment

  • Make Jenkins master stable and restorable
  • Use Jenkins 2 LTS (2.7.x)
  • Have a Jenkins testbed/staging environment
  • Set up example jobs, test your builds in staging prior to production
  • Conservatively upgrade plugins, they can break
  • Backup your configuration (thinBackup plugin within Jenkins)
  • Avoid the "Maven" job type (use freestyle job, select Maven as a build step)

2. Break up the bloat

  • Multiple masters (split across team, function access...)
  • More agility & control
  • Isolates impact of upgrades/plugin installation
  • More masters with fewer jobs tend to be more stable
  • Break up or transform your jobs, reuse code
  • Multi-job builds allow generic jobs across multiple projects

3. Use Pipelines

  • Jenkins Pipeline makes larger jobs more understandable and durable
  • Parameterized trigger + conditional build steps
  • Copy artifact
  • Promoted builds
  • Transform Jobs to Pipelines (define your jobs in code, check Jenkinsfile into repository)
  • Run steps on different nodes, checkout multiple repositories
  • One Pipeline can do far more than one job
  • Script Console and Scriptler (access entire system, use Groovy scripts)
  • Scriptler examples at https://github.com/jenkinsci/jenkins-scripts
  • System Groovy build steps (caution: this gives the build full access to Jenkins)
  • Generate jobs from code (REST API & CLI let you POST new jobs and changes to existing jobs)
  • Jenkins Job Builder translates YAML into jobs
  • Autodiscovery and creation of items in Jenkins with BitBucket Team/Project Folder plugins
  • Blue Ocean, user experience built on top of Pipelines (see what stage your build is in)

4. Tend your plugin garden

  • Do I need this plugin? They can affect load and run time for jobs
  • Periodically uninstall plugins you don't need/use
  • Under "Manage Jenkins" view, watch for notices about "old data" and remove as necessary
  • Essential plugins: Job Config History, Pipeline Stage View, Timestamper, Rebuild
  • Pipelines replaces many older plugins
  • Global configuration: Some plugins have global settings in "Configure Jenkins", remember to check this

5. Integrate with other tools and services

  • Browserstack, SauceLabs
  • Via plugins and REST API, trigger builds on pull requests, update JIRA etc... (find more on Jenkins wiki)
  • Source Control
  • Code Review, build every proposed change, enable automatic merging
  • Generate JIRA release notes on every build
  • Docker is smoothly integrated into Jenkins Pipeline (additional plugins for building Docker images, running builds in containers, tracing containers between jobs, integrating with container providers like Kubernetes)

6. Make system replaceable

  • Agents should be easily and readily replaced (if one fails or is busy, add more)
  • The easier it is to add agents, the easier it will be to replace them
  • Puppet, Chef, Docker
  • Try to make agents general purpose
  • Avoid customizing an agent for one job
  • Interchangeable agents allow for more efficient use of Jenkins resources
  • Use cloud resources (AWS)

7. Join the Jenkins community

Build a Jenkins Pipeline

  • Andy Pemberton
  • Install Jenkins Enterprise, create a Jenkins Pipeline, try the basics...
  • Install Jenkins Enterprise: https://cloudbees.com/get-started
  • What is Jenkins (an automation server, continuous deployment & integration)
  • Automate 1. build, 2. test, 3. deployment
  • java -jar jenkins.war
  • Extensible, exposes plugin API
  • Can connect additional agents to master, can run builds for iOS, Windows, anything that supports SSH
  • CloudBees Jenkins Enterprise (part of CloudBees Jenkins Platform)
  • Jenkins Pipeline is new job type (like freestyle job)
  • Very durable, will keep running even if Jenkins master is not (can restart master while Pipeline is running)
  • Pipeline Stage View provides status at a glance

Lab exercise

  • Store Jenkinsfile in Git, link it in Pipeline configuration
  • Keywords:
  • stage = group your steps into component parts (such as build, test, deploy)
  • node = schedule the steps to run by adding them to Jenkins build queue and creates a workspace specific to this pipeline
  • sh or bat = execute shell scripts, just like freestyle jobs
  • git = checkout from source control
  • stash = store some files for later use
  • unstash
  • writeFile
  • readFile
retry(5) {
    // some block
}

timeout(time: 30, unit: 'SECONDS') {
    // some block
}
  • input = get user input (such as "Should we continue deploy?")
stage('deploy')
    input message: 'Do you want to deploy?'
    node {
        echo 'Deployed'
    }
  • parallel = run tasks in parallel
  • checkpoint = allow failed Pipeline to restart from specified checkpoint
  • Any Jenkinsfile should run anywhere

Sample Pipeline code:

stage 'build'
node {
    // git '... .git'
    // sh 'pwd'
    echo 'Build something.'   
}

stage 'test'
node {
    echo 'Test something.'   
}

stage 'deploy'
node {
    echo 'Deploy something.'   
}

Tool step

  • Binds a tool installation to a variable
  • The tool home directory is returned
  • Only tools already configured are available
  • This may be less useful on AWS/Docker, since this caches resources
def mvnHome = tool 'M3'
sh "${mvnHome}/bin/mvn -B verify"

Use Docker containers

  • CloudBees Docker Pipeline plugin allows running inside a container
  • You can even build the container as part of the same Pipeline
docker.image('maven:3.3.3-jdk-8').inside() {
    sh 'mvn -B verify'
}

Send mail

mail body: 'Uh oh.', subject: 'Build failed!', to: 'myles@gumgum.com'

Deploy to Amazon Elastic Beanstalk

wrap([$class: 'AmazonAwsCliBuildWrapper', credentialsId: 'aws-beanstalk-credentials', defaultRegion: 'us-east-1']) {
    sh 'aws elasticbeanstalk create-application-version'
}

Integrate with JIRA

jiraComment(issueKey: "EX-111", body: "Job '${env.JOB_NAME}' (${env.BUILD_NUMBER}) built. Please go to ${env.BUILD_URL}.")

Pipeline multibranch

  • Use pull requests & specific branches

Docker

  • Viktor Farcic
  • sudo docker-compose -f docker-compose-test.yml up -d staging-dep
  • terraform destroy to destroy AWS instance (using Terraform)
  • Swarm cluster
  • docker network
  • docker service
  • docker service scale go-demo=5
  • docker service update
  • docker service create --name jenkins
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment