Skip to content

Instantly share code, notes, and snippets.

@jarvisms jarvisms/README.md
Created Apr 14, 2019

Embed
What would you like to do?
rsync incremental backup time machine to remote server of entire root tree

This acheives full file system backups to a destination. I this for backing up my entire Raspberry Pi SD cards (with a few minor modifications) daily via my crontab.

The rrsync script is required from https://www.samba.org/ftp/unpacked/rsync/support/rrsync

Sample line from authorized_key file must reside on the destination server where the backups will be. The path to backup it given here and all restrictions are in place so that this is the only thing the ssh user can do, and the only place they have access to. A public/private key combo without passphrase must have been created and shared out first etc. etc. https://www.guyrutenberg.com/2014/01/14/restricting-ssh-access-to-rsync/

The exclude-list can be added to to skip particular folders or files, although the rsync parameters will prevent it from crossing file systems.

The backup.sh file is where the deed is done and is inspired by https://blog.interlinked.org/tutorials/rsync_time_machine.html (link may now be broken) You specify the remote server and your exclude-list and a few other bits. You can look up what the various rsync parameters will do, but in essence everything will be mirrored or deleted but if existing file already reside in the LINKDIR, then these will be hard linked remote server side. Therefore only changed files need to be transfered. The backup will be timestamped and a sym-link will be made called "current" to the backup just made. The second rsync call moves the local log and new symlink to the remote server too. The first backup will copy everything, but subsequent ones should be faster and incremental. Since rrsync (restricted rsync) is used, the root of the destination folder is actually defined in the authorized_key file, and LINKDIR is relative to the destination and can't use . or .. relative folder names.

The resulting struction will be /path/to/backup/ containing sub-folders such as 2019-04-14T20-45-30 each incrementally hardlinked to the last but appearing to be complete in their own right. There will also be the "rsync.log" of the last backup, and "current" symlink to the last backup folder. Since all un-changing files are hardlinked, this is very space efficient. However it may not be obvious which these are and so all of the backups should be considered read-only.

Note that the script does not delete old backups and eventually you may just run out of space, particularly if the file systems change alot.

command="/path/to/rrsync /path/to/backup/",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa ....public_key....== root@backmeup
#!/bin/bash
### These assume public key without passphrase and rrsync is in use rooted to relevent remote backup destination
BACKUPROOT=root@host
LINKDIR=/current
EXCLUDELIST=/root/exclude-list
LOGFILE=/tmp/rsync.log
SYMLINK=/tmp/current
DATE=$(date "+%Y-%m-%dT%H-%M-%S")
rsync -viPhhhaHSx --no-R --delete-during --delete-excluded --stats --numeric-ids --link-dest="${LINKDIR}" --log-file="${LOGFILE}" --exclude-from="${EXCLUDELIST}" / "${BACKUPROOT}:/${DATE}"
ln -sfTv "${DATE}" "${SYMLINK}"
rsync -viPhhhaHSx --no-R --delete-during --stats --numeric-ids --remove-source-files "${LOGFILE}" "${SYMLINK}" "${BACKUPROOT}:/"
# System folders to exclude
# assuming -x option (does not cross file systems)
/tmp/*
/var/swap
/var/tmp/*
/media/*
/mnt/*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.