Skip to content

Instantly share code, notes, and snippets.

@cenkalti
Last active October 10, 2024 10:26
Show Gist options
  • Save cenkalti/5089392 to your computer and use it in GitHub Desktop.
Save cenkalti/5089392 to your computer and use it in GitHub Desktop.
Backup Jenkins home periodicallly with git.
#!/bin/bash
# Setup
#
# - Create a new Jenkins Job
# - Mark "None" for Source Control Management
# - Select the "Build Periodically" build trigger
# - configure to run as frequently as you like
# - Add a new "Execute Shell" build step
# - Paste the contents of this file as the command
# - Save
#
# NOTE: before this job will work, you'll need to manually navigate to the $JENKINS_HOME directory
# and do the initial set up of the git repository.
# Make sure the appropriate remote is added and the default remote/branch set up.
#
# Jenkins Configuraitons Directory
cd $JENKINS_HOME
# Add general configurations, job configurations, and user content
git add -- *.xml jobs/*/*.xml userContent/*
# only add user configurations if they exist
if [ -d users ]; then
user_configs=`ls users/*/config.xml`
if [ -n "$user_configs" ]; then
git add $user_configs
fi
fi
# mark as deleted anything that's been, well, deleted
to_remove=`git status | grep "deleted" | awk '{print $3}'`
if [ -n "$to_remove" ]; then
git rm --ignore-unmatch $to_remove
fi
git commit -m "Automated Jenkins commit"
git push -q -u origin master
@rodriguezsergio
Copy link

Saved me the time of writing my own backup script. Thanks 👍

@kuldazbraslav
Copy link

Thanks, it saved me a lot of time too.

@AlexanderDobrodey
Copy link

Thank you!

@squrppi
Copy link

squrppi commented Feb 22, 2017

I wonder if the line 34 awk should actually read '{print $2}' instead of '{print $3}'?
The output of "git status" is like

   deleted:    test.py

@Alokdavita
Copy link

Thanks a lot !!

@gardner
Copy link

gardner commented Aug 10, 2017

Thank you! I want to post a "You Da Real MVP" meme but I don't think that's allowed on GH.

@syquus
Copy link

syquus commented Sep 13, 2017

Just a note:
'{print $3}'
is returning empty string for me. Therefore there are elements not being git rm-ed....

I replaced it for
'{print $2}'
and works correctly.

@franc3000
Copy link

this is great, thank you!

what do you add from the $JENKINS_HOME directory when doing the initial set up of the git repository?

git add * ?

I added all the secret files, and now I'm getting this:

ubuntu@:~/jenkins$ git status
jenkins-data/secrets/filepath-filters.d/30-default.conf: Permission denied
jenkins-data/secrets/hudson.util.Secret: Permission denied
jenkins-data/secrets/initialAdminPassword: Permission denied
jenkins-data/secrets/jenkins.model.Jenkins.crumbSalt: Permission denied
jenkins-data/secrets/jenkins.security.ApiTokenProperty.seed: Permission denied
jenkins-data/secrets/master.key: Permission denied
jenkins-data/secrets/org.jenkinsci.main.modules.instance_identity.InstanceIdentity.KEY: Permission denied
jenkins-data/secrets/slave-to-master-security-kill-switch: Permission denied
jenkins-data/secrets/whitelisted-callables.d/default.conf: Permission denied
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

@maximba
Copy link

maximba commented Mar 9, 2018

Modified "to_remove" expression to be more robust:
to_remove=$(git status | grep "deleted" | cut -d ":" -f2 | sed -r 's/^[ \t]*//p')

@greilly
Copy link

greilly commented Mar 27, 2018

Adding user configurations gave me some errors, and can be simplified:

# only add user configurations if they exist
if [ -d users ]; then
    git add users/*/config.xml
fi

@msvticket
Copy link

A simpler way of doing deletion resilient to whitespace in file names:
git ls-files -dz | xargs -0r git rm

@transistorgit
Copy link

The initial script works fine for me, but leaves out some more complex (deeper nested) jobs.
So I added
shopt -s globstar
at the beginning and changed
git add -- .xml jobs//.xml userContent/
to
git add -- .xml jobs/**/config.xml userContent/

@ronensh-nice
Copy link

I used the following to cope with nested jobs folders
find -L jobs -type f -iname config.xml -follow -print0 -exec git add {} \;

@ilyaevseev
Copy link

This can be simplified:

if [ -d users ]; then
    user_configs=`ls users/*/config.xml`

    if [ -n "$user_configs" ]; then
        git add $user_configs
    fi
fi

to:

ls users/*/config.xml 2>/dev/null | xargs -r git add

@ilyaevseev
Copy link

And this can be simplified

to_remove=`git status | grep "deleted" | awk '{print $3}'`

if [ -n "$to_remove" ]; then
    git rm --ignore-unmatch $to_remove
fi

to:

git status | awk '/deleted/ { print $3 }' | xargs -r git rm --ignore-unmatch

@ilyaevseev
Copy link

ilyaevseev commented Mar 18, 2019

Final edition:

#!/bin/sh -e

cd "$JENKINS_HOME"

# Add general configurations, secrets, job configurations, nodes, user content, users and plugins info:
ls -1d *.xml secrets/ jobs/*/*.xml nodes/*/*.xml userContent/* users/*/config.xml \
    plugins/*/META-INF/MANIFEST.MF 2>/dev/null | xargs -r -d '\n' git add --

# Track deleted files:
LANG=C git status | awk '$1 == "deleted:" { print $2; }' | xargs -r git rm --ignore-unmatch

LANG=C git status | grep -q '^nothing to commit' || {
    git commit -m "Automated Jenkins commit at $(date '+%Y-%m-%d %H:%M')"
    git push -q -u origin master
}

@ilyaevseev
Copy link

@TheNotary
Copy link

TheNotary commented May 3, 2019

I'm noticing that when you restore jenkins, it's important to make sure the files are all owned by the jenkins user with the jenkins group or you might see the below error in the GUI:

jenkins.model.InvalidBuildsDir: ${ITEM_ROOTDIR}/builds does not exist and probably cannot be created
	at jenkins.model.Jenkins.checkRawBuildsDir(Jenkins.java:3126)
	at jenkins.model.Jenkins.loadConfig(Jenkins.java:3047)
Caused: java.io.IOException
	at jenkins.model.Jenkins.loadConfig(Jenkins.java:3050)
	at jenkins.model.Jenkins.access$1300(Jenkins.java:309)
	at jenkins.model.Jenkins$13.run(Jenkins.java:3145)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1096)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: org.jvnet.hudson.reactor.ReactorException
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:48)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1130)
	at jenkins.model.Jenkins.<init>(Jenkins.java:932)
	at hudson.model.Hudson.<init>(Hudson.java:85)
	at hudson.model.Hudson.<init>(Hudson.java:81)
	at hudson.WebAppMain$3.run(WebAppMain.java:233)
Caused: hudson.util.HudsonFailedToLoad
	at hudson.WebAppMain$3.run(WebAppMain.java:250)

Copy link

ghost commented Sep 26, 2020

transistorgit's comment was mangled by the Markdown renderer, but I think this was the intention:

The initial script works fine for me, but leaves out some more complex (deeper nested) jobs.
So I added
shopt -s globstar
at the beginning and changed
git add -- *.xml jobs/*/*.xml userContent/*
to
git add -- *.xml jobs/**/config.xml userContent/*

@mirkokg
Copy link

mirkokg commented Nov 4, 2020

This script has one issue though, folders and build history are not saved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment