How to keep Jenkins configurations in a git repository. Put .gitignore in Jenkins’ home directory (/var/lib/jenkins). Initialize a git repository and add a remote to push to. Create a new Jenkins job to run e.g. nightly with the content of backup.sh as shell execution build step.

.gitignore

# Non-config file types
*.log
*.log.*
*.tmp
*.old
*.bak
*.jar
*.jpi
*.jpi.pinned
*.json

# Security related
/.ssh
/secret.key*
/secrets
/credentials.xml
/identity.key
 
# Generated Jenkins state
/.owner
/queue.xml
/fingerprints/
/shelvedProjects/
/updates/
 
# Tools that Jenkins manages
/tools/
/cache/
 
# Extracted plugins
/plugins/*/
 
# Job state
builds/
workspace/
lastStable
lastSuccessful
nextBuildNumber
modules/

# Interactive shell stuff
.bash_history
/.vim*
.lesshst
.Xauthority

# Package cache
/.gradle/caches
/.ivy2/
/.m2/
/.node-gyp/
/.npm/
/.sbt/

# TFS plugin
/.microsoft

# FOP cache
/.fop

# Fontconfig cache
/.cache

backup.sh

# Move into the Jenkins master directory
cd $JENKINS_HOME

# Add all new, changed and deleted files
# Unwanted files must be listed in .gitignore
git add --all

# Commit, and ignore if the commit is empty
git commit -m "Jenkins config change" || true

# Push changes outside of the Jenkins directory
git push

@thegoldenmule "and whatever else" is the tricky part. Your approach requires knowing when Jenkins adds new config files or changes their locations. OP's approach will find anything new/unrecognized.