Skip to content

Instantly share code, notes, and snippets.

@voidfiles
Created September 23, 2016 21:47
Show Gist options
  • Save voidfiles/a24fea5fd56a8f81c361c84bd29e8411 to your computer and use it in GitHub Desktop.
Save voidfiles/a24fea5fd56a8f81c361c84bd29e8411 to your computer and use it in GitHub Desktop.
Multiservice Jenkinsfile Reuse

Let me know if this sounds, right.

I work at a company that has N services. Each service has it's own deployable artifact. Each artifact creation process is unique, but the pipeline, and deployment processes are similar. As we grow we are looking to create more services. I would like to manage this complexity in a couple of ways.

  1. We use Jenkins
  2. I'd like to use Jenkinsfile's to manage the jobs
  3. The artifact process should stay unique, some shared code loaded via fileLoader.fromGit
  4. I plan on using a shared deployment job configured via params
  5. I plan on using a shared pipeline job configured via params

My main reasoning for re-using deploy, and pipeline code is it makes it easier to rollout new changes to all pipelines, because they are shared.

Internally we currently manage our deployment pipeline via Slack, and I'd like to continue doing that. I'd also like to be able to use slack to do things like roll environments back to specific versions. So, for instance I need to be able to ask questions like what was the last version of application x that was rolled to production. So, I can roll back to that.

Am I in the right vicinity?

I have included example jenkins files

Here is a generic Jenkinsfile I have a for building an artifact.

node {
  stage('Checkout') {
    checkout scm
    BUILD_TAG = getBuildTag();
  }

  // Mostly static analysis tools
  stage('Preflight') {
    withEnv(["WORKSPACE=${WORKSPACE}", "BUILD_TAG=${BUILD_TAG}", "TARBALLS=1"]) {
      sh "./scripts/preflight.sh"
    }
  }

  stage('Unit Test') {
    withEnv(["WORKSPACE=${WORKSPACE}", "BUILD_TAG=${BUILD_TAG}", "TARBALLS=1"]) {
      sh "./scripts/unit_test.sh"
    }
  }

  stage('Build') {
    withEnv(["WORKSPACE=${WORKSPACE}", "BUILD_TAG=${BUILD_TAG}", "TARBALLS=1"]) {
      sh "./scripts/build.sh"
    }


    step([
      $class: 'S3BucketPublisher',
      profileName: 'Artifact',
      entries: [
        [sourceFile: 'dist/*', bucket: 'bepress-build-artifacts', selectedRegion: 'us-west-1', managedArtifacts: true],
      ]
    ])

  }

  stage('Start Pipeline') {
    build([
      job: 'org/pipeline/master',
      wait: false,
      parameters: [
        [$class: 'StringParameterValue', name: 'BUILD_TAG', value: BUILD_TAG],
        [$class: 'StringParameterValue', name: 'PLAYBOOK', value:'playbook.yml'],
        [$class: 'StringParameterValue', name: 'APPLICATION', value:'Readable Application name'],
      ]
    ])
  }
}

Then I have a deployment build like this

properties([
    [$class: "BuildDiscarderProperty", strategy: [$class: "LogRotator", numToKeepStr: "1"]
  ], [
    $class: 'ParametersDefinitionProperty',
    parameterDefinitions:[[
      $class: 'StringParameterDefinition',
      description: 'Artifact Build Tag',
      name: 'BUILD_TAG'
    ], [
      $class: 'StringParameterDefinition',
      description: 'Playbook to use to rollout artifact',
      name: 'PLAYBOOK'
    ], [
      $class: 'StringParameterDefinition',
      description: 'What environment to roll to',
      name: 'ENVIRONMENT'
    ], [
      $class: 'StringParameterDefinition',
      description: 'What application are we rolling for',
      name: 'APPLICATION'
    ]
  ]
]])

node {
  def build_tag_parts = BUILD_TAG.tokenize('-')
  def build_num = build_tag_parts[-2];
  def project_name_parts = build_tag_parts[0..-3]
  def project_name = project_name_parts.join('/')

  sh "rm -fR dist"
  sh "mkdir -p dist"
  step([
    $class: 'S3CopyArtifact',
    buildSelector: [
      $class: 'SpecificBuildSelector',
      buildNumber: build_num
    ],
    excludeFilter: '',
    filter: '*',
    flatten: false,
    optional: false,
    projectName: project_name,
    target: './dist/'
  ])

  // Perform deploy with ansible playbook
}

Finally I have a pipeline script that manages an artifacts delivery to production, through multiple manual gates.

#!groovy

properties([
    [$class: "BuildDiscarderProperty", strategy: [$class: "LogRotator", numToKeepStr: "10"]
  ], [
    $class: 'ParametersDefinitionProperty',
    parameterDefinitions:[[
      $class: 'StringParameterDefinition',
      description: 'Artifact Build Tag',
      name: 'BUILD_TAG'
    ], [
      $class: 'StringParameterDefinition',
      description: 'Playbook to use to rollout artifact',
      name: 'PLAYBOOK'
    ]
  ]
]])

// This step is automatic no approval required
stage('Integration Deployment') {

  build([
    job: 'org/deploy/master',
    parameters: [
      [$class: 'StringParameterValue', name: 'BUILD_TAG', value: BUILD_TAG],
      [$class: 'StringParameterValue', name: 'PLAYBOOK', value: PLAYBOOK],
      [$class: 'StringParameterValue', name: 'ENVIRONMENT', value: 'integration'],
      [$class: 'StringParameterValue', name: 'APPLICATION', value: APPLICATION],
      [$class: 'BooleanParameterValue', name: 'DOCS', value: DOCS.toBoolean()]
    ]
  ])
}

stage('Staging Deployment') {
  // Send a notification to team slack asking for somone to approve rolling to staging

  timeout(time: 5, unit: 'DAYS') {
      input(message:"Deploy ${BUILD_TAG} to staging", id:"staging")
  }

  build([
    job: 'org/deploy/master',
    parameters: [
      [$class: 'StringParameterValue', name: 'BUILD_TAG', value: BUILD_TAG],
      [$class: 'StringParameterValue', name: 'PLAYBOOK', value: PLAYBOOK],
      [$class: 'StringParameterValue', name: 'ENVIRONMENT', value: 'staging'],
      [$class: 'StringParameterValue', name: 'APPLICATION', value: APPLICATION],
      [$class: 'BooleanParameterValue', name: 'DOCS', value: DOCS.toBoolean()]
    ]
  ])
}

stage('Production Deployment') {
  // Send a notification to team slack asking for somone to approve rolling to staging

  timeout(time: 5, unit: 'DAYS') {
      input(message:"Deploy ${BUILD_TAG} to staging", id:"staging")
  }

  build([
    job: 'org/deploy/master',
    parameters: [
      [$class: 'StringParameterValue', name: 'BUILD_TAG', value: BUILD_TAG],
      [$class: 'StringParameterValue', name: 'PLAYBOOK', value: PLAYBOOK],
      [$class: 'StringParameterValue', name: 'ENVIRONMENT', value: 'staging'],
      [$class: 'StringParameterValue', name: 'APPLICATION', value: APPLICATION],
      [$class: 'BooleanParameterValue', name: 'DOCS', value: DOCS.toBoolean()]
    ]
  ])
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment