It took me few days to come up with a fully functional pipeline which can build a Spring Boot app and deploy it over SSH.
Especially when there are tutorials for scripted and declarative pipelines which can be pretty confusing for beginners. Here is a simple Jenkins declarative pipeline, set up as a shared library, which deploys Spring Boot application to a target node over SSH.
Feel free to modify it to suit your needs.
Star the gist if I saved you from few head injuries :o)
If you are new to Jenkins you will find few handy examples how to:
- define and work with variables and environment variables
- work with associative arrays
- do loops
- deploy stuff over ssh
- handle results
By shared pipeline I mean that it is set up as a library in Jenkins and can be reused across multiple jobs. In our setup we had multiple Spring Boot apps running on the same machine on different ports. In order to have streamlined deployment process across all services we used only one shared pipeline and as a argument we passed in only the name of service we wanted to deploy.
We had a separate Jenkins job for each service (and for each environment). Each job used the shared pipeline and just told the pipeline which service to deploy.
Entire pipeline is defiend inside a file called deliveryPipeline.groovy. You place this file in a separate repo and typically inside /vars folder. Here sit your library functions. Each file defines a function and the name of the file defines the name of the function. In this case you call it like deliveyPipeline('service name to deploy') from your Jenkins file once you setup this pipe as a shared library.
In repo we had also Jenkins file which tells Jenkins which library to use and what to do. The file looks like this:
@Library('library_name@master') _
deliveryPipeline('service to deploy')
This two lines of code tells Jenkins to use the latest version of the shared library and call function deliveryPipeline from that library. And that was entire configuration for a Jenkins job for each service!
There is a bunch of tutorials how to do it. Take a look here.
In our setup we had in repo /resources folder with /resources/dev, /resources/test, /resources/prod foldes and in each we had application.properties configured for each environment. One important thing to mention here is that we used this pipeline to make a build only from dev branch. When the build travelled to higher environments we deployed only resources for those environments along with already built application.jar.
At the beginning there is a bunch of associative arrays to define configuration for each service like repo, port, files to deploy. Then it defines environment variables, stages and finally handles the result.
If you need to deploy just one service make sure all the arrays have at least one entry.
Deploys Spring Boot app over SSH to a single node. Backs up current files restarts the app using UNIX service and does a health check if app is up and running.
This pipeline deploys artifacts only to one node. If you need to deploy to multiple nodes you need to add one more array and iterate over it.
It also does not handle situation when service does not start within defined time interval and does not make automatic rollback.
/*
*
* deliveryPipeline.groovy
*
* Jenkins shared library to deploy Spring Boot application over SSH.
*
* @param service - name of service to be deployed to target server
*/
def call(String service) {
/*
* Configuration of services
*/
// set your Git repo urls
def repositories = [
// keep it without https://
service1: "github.com/repo1/service1.git",
service2: "github.com/repo1/service2.git",
service3: "github.com/repo1/service3.git",
service4: "github.com/repo1/service4.git",
service5: "github.com/repo1/service5.git"
]
// set your port numbers
def ports = [
service1: 7100,
service2: 7200,
service3: 7300,
service4: 7400,
service5: 7500
]
// built artifacts which will be deployed after maven build from target folder, see Deploy stage
def artifacts = [
// this is an array, you can provide multiple files if needed
service1: ['application.jar'],
service2: ['application.jar'],
service3: ['application.jar'],
service4: ['application.jar'],
service5: ['application.jar']
]
// resources which will be deployed based on environment from resources/${env} folder, see Deploy stage
def resources = [
service1: ['application.properties', 'log4j2.xml'],
service2: ['application.properties'],
service3: ['application.properties'],
service4: ['application.properties'],
service5: ['application.properties']
]
// if invalid service name is passed then abort the job
if (!repositories.containsKey(service)) {
error "Invalid service ${service}!"
}
def serviceRepository = repositories.get(service)
def servicePort = ports.get(service)
def serviceArtifacts = artifacts.get(service)
def serviceResources = resources.get(service)
pipeline {
agent any
tools {
maven 'Maven 3.3.9'
jdk 'JDK8'
}
environment {
/*
* Environment. Based on the value resources from repository are deployed from resources/dev /resources/test /resources/prod folder.
* When set yo 'dev' you must have /resources/dev folder in your repo.
*/
ENVIRONMENT = 'dev'
/*
* Service name being deployed.
*/
SERVICE_NAME = "${service}"
/*
* Port where service is listening.
*/
PORT = "${servicePort}"
/*
* Node where artifacts will be deployed.
*/
NODE = 'change to your hostname'
/*
* Name of certificate to open SSH session which has been setup in Jenkins credentials.
*/
CERTIFICATE = 'change to id of your certificate'
/*
* User for deploying stuff over SSH.
*/
SSH_USER = 'change to ssh user'
/*
* How many times check if service is up and running after deployment.
*/
RETRY_COUNT = 20
/*
* How long wait between retries in seconds.
*/
RETRY_TIMEOUT = 5;
/*
* Repository.
*/
REPOSITORY = "${serviceRepository}"
/*
* Credentials defined in Jenkins to access the Git repository.
*/
REPOSITORY_CREDENTIALS = 'change to your username'
/*
* HTTP Basic Auth credentials setup in Jenkins to access service endpoints. See Health Check stage.
*/
ENDPOINT_CREDENTIALS = 'change it';
/*
* Service directory where artifacts will be deployed.
*/
SERVICE_DIRECTORY = "/appl/spring/${SERVICE_NAME}"
/*
* Directory where current service artifacts will be backed up.
*/
BACKUP_DIRECTORY = "${SERVICE_DIRECTORY}/backup"
}
stages {
// git checkout
stage('Checkout') {
steps {
sh "git config user.name ${REPOSITORY_CREDENTIALS}"
// change your email here
sh "git config user.email ${REPOSITORY_CREDENTIALS}@yourdoma.in"
// if you get repository not found error here make sure the user you are using has write access to the repository
// https://stackoverflow.com/questions/10116373/git-push-error-repository-not-found
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: "${REPOSITORY_CREDENTIALS}", usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
// here it clones dev branch
// change it or move it to variable
sh "git clone -b dev https://${USERNAME}:${PASSWORD}@${REPOSITORY}"
}
}
}
// print all variables
stage('Info') {
steps {
script {
sh 'printenv'
echo "Build Number: ${currentBuild.number}"
def pom = readMavenPom file: 'pom.xml'
echo "Service ${SERVICE_NAME} ${pom.version} is being deployed..."
}
}
}
// build project using maven and Artifactory
stage('Build') {
steps {
script {
def server = Artifactory.server('Your Artifactory Jenkins configuration name')
def rtMaven = Artifactory.newMavenBuild()
rtMaven.resolver server: server, releaseRepo: 'maven-release', snapshotRepo: 'maven-snapshot'
rtMaven.deployer server: server, releaseRepo: 'maven-release-local', snapshotRepo: 'maven-snapshot-local'
rtMaven.deployer.deployArtifacts = false
rtMaven.run pom: 'pom.xml', goals: 'clean package'
}
}
}
// back up currently deployed stuff
stage('Backup') {
steps {
echo "Backing up current artifacts. Moving existing artifacts from ${SERVICE_DIRECTORY} to ${BACKUP_DIRECTORY}..."
sshagent(credentials: ["${CERTIFICATE}"]) {
script {
// http://docs.groovy-lang.org/next/html/documentation/working-with-collections.html#_iterating_on_a_list
serviceArtifacts.each {
try {
sh "ssh ${SSH_USER}@${NODE} mv ${SERVICE_DIRECTORY}/${it} ${BACKUP_DIRECTORY}/${it}.${currentBuild.number}"
} catch (Exception e) {
echo "File ${it} does not exist."
}
}
serviceResources.each {
try {
sh "ssh ${SSH_USER}@${NODE} mv ${SERVICE_DIRECTORY}/${it} ${BACKUP_DIRECTORY}/${it}.${currentBuild.number}"
} catch (Exception e) {
echo "File ${it} does not exist."
}
}
}
}
}
}
// deploy built artifacts and resources for dev environment
stage('Deploy') {
steps {
echo "Deploying new artifacts to directory ${SERVICE_DIRECTORY}..."
sshagent(credentials: ["${CERTIFICATE}"]) {
script {
serviceArtifacts.each {
sh "scp ${WORKSPACE}/target/${it} ${SSH_USER}@${NODE}:${SERVICE_DIRECTORY}/${it}"
}
serviceResources.each {
// copy environment specific resources
sh "scp ${WORKSPACE}/resources/${ENVIRONMENT}/${it} ${SSH_USER}@${NODE}:${SERVICE_DIRECTORY}/${it}"
}
}
}
}
}
// restart deployed service running as UNIX service
stage('Restart') {
steps {
echo "Restarting service..."
sshagent(credentials: ["${CERTIFICATE}"]) {
// make sure you run Spring Boot as a UNIX service
sh "ssh ${SSH_USER}@${NODE} service ${SERVICE_NAME} restart"
}
}
}
// check if service is up and running
// after all retries stage will fail and next stages will be skipped and notification email will be sent
stage('Health Check') {
steps {
echo "Checking if service is up and running... Retrying ${RETRY_COUNT} times every ${RETRY_TIMEOUT} seconds."
retry("${RETRY_COUNT}") {
sleep "${RETRY_TIMEOUT}"
// ping monitoring endpoint (Spring Actuator) to check if service is up and running
httpRequest authentication: "${ENDPOINT_CREDENTIALS}", url: "https://${NODE}:${PORT}/${SERVICE_NAME}/monitoring", validResponseCodes: '200'
}
}
}
// do not litter
stage('Cleanup') {
steps {
echo "Cleaning up garbage..."
// in DEV delete everything from backup folder we don't need it
sshagent(credentials: ["${CERTIFICATE}"]) {
sh "ssh ${SSH_USER}@${NODE} rm ${BACKUP_DIRECTORY}/*"
}
}
}
}
post {
always {
echo "Build completed. currentBuild.result = ${currentBuild.result}"
}
changed {
echo 'Build result changed'
script {
if (currentBuild.result == 'SUCCESS') {
echo 'Build has changed to SUCCESS status'
}
}
}
failure {
script {
def pom = readMavenPom file: 'pom.xml'
emailext(
subject: "Failure during service deployment!!! ${SERVICE_NAME} ${pom.version} ${ENVIRONMENT}",
body: """Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'""",
recipientProviders: [[$class: 'DevelopersRecipientProvider']]
)
}
}
success {
script {
def pom = readMavenPom file: 'pom.xml'
emailext(
subject: "Service ${SERVICE_NAME} ${pom.version} has been deployed to ${ENVIRONMENT}",
body: """Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'""",
recipientProviders: [[$class: 'DevelopersRecipientProvider']]
)
}
}
unstable {
script {
def pom = readMavenPom file: 'pom.xml'
emailext(
subject: "Build became unstable! ${SERVICE_NAME} ${pom.version} ${ENVIRONMENT}",
body: """Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'""",
recipientProviders: [[$class: 'DevelopersRecipientProvider']]
)
}
}
// https://stackoverflow.com/questions/37468455/jenkins-pipeline-wipe-out-workspace
// this stage is executed always as the last one
cleanup {
deleteDir()
}
}
}
}