Instantly share code, notes, and snippets.

Embed
What would you like to do?
Backup shell script using rsync

About

This is simplified version of script I'm using for doing my backups. I have prepared it only recently because manual invocation of rsync was enough for me. But as the number of separately rsynced folders increased I was forced to introduce some automation and I had to solve some interesting problems along the way.

Persistent SSH connection

Normally a separate SSH connection is established for each rsync invocation. That may become irritating if, for some reason, establishing an SSH connection requires manual intervention (e. g. password, touch id...). A solution is to set up multiplexing, which is available in OpenSSH. Connection is then set up only once (see ControlMaster=yes) and rsync is then forced to use that connection (see --rsh argument). It's important to close the connection at the end of the script (see ssh -O exit...).

Local/remote backup

Sometimes (after huge changes in backuped data or on slow network) it may be better not to run backup over network, but instead connect the storage itself directly to backuped computer and do the backup over fast local interface (USB 3, SATA...). Rsync encodes storage type directly in source and target argument so it's not necessary to create separate scripts for those situations (see TARGET variable).

Conditionaly empty command arguments

Rsync is sensitive to zero length arguments ("") which most of the time triggers unintentional synchronization of current working directory. This means it's problematic to use conditionaly empty command line arguments (see variable DRY_RUN). Solution is to use eval shell builtin. This also allows to have multiple arguments stored in one variable (see variable OPTIONS).

#!/bin/bash
# prerequisites
# mkdir ~/.ssh/masters
if $(zenity --question --text="Dry run?") ; then
DRY_RUN="--dry-run"
else
DRY_RUN=""
fi
TARGET_SERVER=$(zenity --list --title "Backup storage selection" --radiolist --print-column=2 --column "" --column "Storage" TRUE "pi@server" FALSE "pi@otherserver" FALSE "USB")
if [ "$TARGET_SERVER" = "USB" ]; then
TARGET_REMOTE_SHELL=""
TARGET="/media/user/BACKUPS/"
LOCAL=true
else
TARGET_PATH_PREFIX=/mnt/BACKUPS/
SSH_MASTERS_PATH_TEMPLATE="$HOME/.ssh/masters/%L-%r@%h:%p"
TARGET_REMOTE_SHELL="'--rsh=ssh -o ControlPath=$SSH_MASTERS_PATH_TEMPLATE'"
TARGET="${TARGET_SERVER}:${TARGET_PATH_PREFIX}"
LOCAL=false
fi
OPTIONS="$DRY_RUN --archive --progress --delete $TARGET_REMOTE_SHELL"
if [ "$LOCAL" = false ] ; then
# https://unix.stackexchange.com/questions/50508/reusing-ssh-session-for-repeated-rsync-commands
echo "Creating persistent SSH connection..."
ssh -nNf -o ControlMaster=yes -o ControlPath="$SSH_MASTERS_PATH_TEMPLATE" "$TARGET_SERVER"
fi
echo "########## Synchronizing hobby projects ##########"
eval rsync "$OPTIONS" \
--exclude='/*/checkout/' \
--exclude='/*/workspace/' \
/home/user/hobby/ ${TARGET}hobby/
echo "########## Synchronizing work projects ##########"
eval rsync "$OPTIONS" \
--exclude='/*/checkout/' \
--exclude='/*/workspace/' \
--exclude='/*/*/webapps/' \
--exclude='/*/*/temp/' \
--exclude='/*/*/logs/' \
--exclude='/*/*/work/' \
--exclude="/home/.m2/" \
/home/user/work/ ${TARGET}work/
# ... additional synchronization folders
if [ "$LOCAL" = false ] ; then
echo "Terminating persistent SSH connection..."
ssh -O exit -o ControlPath="$SSH_MASTERS_PATH_TEMPLATE" "$TARGET_SERVER"
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment