Skip to content

Instantly share code, notes, and snippets.

@Ugrend
Last active April 10, 2017 00:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Ugrend/cd338b4bc7bb29e452d5417e38f98e04 to your computer and use it in GitHub Desktop.
Save Ugrend/cd338b4bc7bb29e452d5417e38f98e04 to your computer and use it in GitHub Desktop.
Simple rsync backup
#!/usr/bin/env bash
# This is a simple backup script
# I have lots of static non changing data, I wish to back it up accross multiple disks
# The idea is that in the event of total data loss, I can restore the data disk by disk
# If one of the disks dies or is faulty, I would only lose the data on that disk,
#I would still be able to recover other files from other disks
#
# Basically this script will rsync files from the specified path to the backup path until the disk is full
# Once the disk is full, rsync will error and send me a email stating the disk is full
# Once rsync is done (if it errored or not) the script will then read each file it copied and check if its on the backup disk
# If it finds the file it will create a .rsync-filter file in that files directory telling rsync to ignore this file in the future.
#
# THIS MAY FAIL, MAY MISS FILES/etc/etc
BACKUP_TO=/mnt/disk1/
ORIGIN=/storage/home/samba/
LOG=/var/log/backup.log
EMAIL_TO="EMAIL@EMAIL.COM"
DATE=$(date +"%Y%m%d_%H:%M:%S")
if [[ -a /root/backup_running ]];then
echo "backup still running, you may want to check that out" | mail -s "Backup still running ${DATE}" ${EMAIL_TO}
exit 1
fi
touch /root/backup_running
echo "Backup Started ${DATE}" | mail -s "BACKUP STARTED ${DATE}" ${EMAIL_TO}
rsync --exclude "${ORIGIN}torrents" --exclude "${ORIGIN}completed" -i -FF -a "${ORIGIN}" "${BACKUP_TO}" > /root/rsync-${DATE}.log 2> /root/rsync-error-${DATE}.log
if [[ -a /root/rsync-error-${DATE}.log ]];then
mail -s "RSYNC FAILURE ${DATE}" ${EMAIL_TO} < /root/rsync-error-${DATE}.log
fi
while read line; do
TYPE=$(echo ${line} | cut -f 1 -d " ")
if [ "${TYPE}" = '>f+++++++++' ]; then
FILE=$(echo "${line}" | cut -d " " -f 2-)
if [[ -a "${BACKUP_TO}${FILE}" ]]; then
if [[ -a "${ORIGIN}${FILE}" ]] ; then
originalHash=$(md5 "${ORIGIN}${FILE}" | cut -f 4 -d " ")
backupHash=$(md5 "${BACKUP_TO}${FILE}" | cut -f 4 -d " ")
if [ "${originalHash}" = "${backupHash}" ];then
dirName=$(dirname "${ORIGIN}${FILE}")
fileName=$(basename "${ORIGIN}${FILE}")
echo "- ${fileName}" | sed 's/\$/\\$/g' | sed 's/\[/\\[/g' | sed 's/\]/\\]/g' >> "${dirName}/.rsync-filter"
chmod 600 "${dirName}/.rsync-filter" # So normal users cannot view/see them
echo "${BACKUP_TO}${FILE} verified" >> ${LOG}
else
echo "${FILE} failed hash check" >> ${LOG}
fi
fi
else
echo "${FILE} not found in backup" >> ${LOG}
fi
fi
done < /root/rsync-${DATE}.log
rm /root/backup_running
echo "BACKUP FINISHED" | mail -s "BACKUP FINISHED $(date +'%Y%m%d_%H:%M:%S')" ${EMAIL_TO}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment