Periodic off-site replication with FreeBSD, duplicacy and B2
I've recently discovered duplicacy, and I've decided to use it with an B2 backend to keep a mirrored offsite replication of the main directories of my home NAS.
I use the
periodic system, which offers a nice abstraction over
crontab for periodic tasks, and duplicacy to perform the actual backup. To setup duplicacy, I've followed the guide. In simple words, for each directory that I want to replicate, I setup duplicacy in it, which in turns means that you'll have to have a
.duplicacy subdirectory in it.
First off, I've added the following to
/etc/defaults/periodic.conf. The prefix, 666, is on purpose.
# 666.backup monthly_backup_enabled="YES" # Replicate offsite
monthly job is invoked every month, and more specifically as defined in the
$ grep monthly /etc/crontab # Perform daily/weekly/monthly maintenance. 30 5 1 * * root periodic monthly
The above variable enables the
666.backup job, defined in
/etc/periodic/monthly/666.backup (make sure the executable bit is set).
#!/bin/sh # # If there is a global system configuration file, suck it in. # if [ -r /etc/defaults/periodic.conf ] then . /etc/defaults/periodic.conf source_periodic_confs fi case "$monthly_backup_enable" in [Yy][Ee][Ss]) echo "" echo "Backup:" periodic /etc/periodic/backup || rc=3;; *) rc=0;; esac exit $rc
So, to turn it on/off, just change the
monthly_backup_enabled variable accordingly.
backup periodic task triggers the execution of any executable script, in lexicographic order, found in the
/etc/periodic/backup directory. In there, I've defined two scripts:
/etc/periodic/backup/100.duplicacy- that is the actual backup
/etc/periodic/backup/101.duplicacy.prune- that prunes any old snapshot (see below)
#!/bin/sh - # # If there is a global system configuration file, suck it in. # if [ -r /etc/defaults/periodic.conf ] then . /etc/defaults/periodic.conf source_periodic_confs fi rc=0 . $backup_duplicacy_pre for target in $backup_duplicacy_targets do path=`echo $target | cut -d\@ -f 1` cd "$path" echo "[`date`] Backing up $path" $backup_duplicacy -log backup -stats sleep 1 done . $backup_duplicacy_post rc=$? exit $rc
The main variable is
$backup_duplicacy, which points to the duplicacy executable, which is defined in the
# 100.duplicacy # use <path>@<prune n:m> syntax - https://github.com/gilbertchen/duplicacy/blob/master/GUIDE.md#prune backup_duplicacy_targets="/etc@0:120 /data/home@0:31 /data/tm@0:31 /data/freezer@0:31 /data/dropbox@0:31" backup_duplicacy_pre="/root/bin/dup_pre.sh" # keep this safe! backup_duplicacy_post="/root/bin/dup_post.sh" backup_duplicacy="/usr/local/bin/duplicacy"
$backup_duplicacy_pre variable, which point to a script that define environment variables to hold the duplicacy secrets (that's why you wanna keep them safe). In my case:
#/root/bin/dup_pre.sh export DUPLICACY_PASSWORD="***" # duplicacy backup encryption password export DUPLICACY_B2_ID="***" # B2 client ID export DUPLICACY_B2_KEY="***" # B2 client secret
$backup_duplicacy_post script takes care of unsetting these variables after duplicacy is done.
#/root/bin/dup_pre.sh unset DUPLICACY_PASSWORD unset DUPLICACY_B2_ID unset DUPLICACY_B2_KEY
Backup target and retain policy definition
The backup targets (i.e., list of directories to be backed up) is defined in the
$backup_duplicacy_targets variable, which in my case is:
backup_duplicacy_targets="/etc@0:120 /data/home@0:31 /data/tm@0:31 /data/freezer@0:31 /data/dropbox@0:31"
So, I backup:
/etcand retain no snapshot older than 120 days
/data/homeand retain no snapshot older than 31 days
/data/tmand retain no snapshots older than 31 days
/data/freezerand retain no snapshots older than 31 days
/data/dropboxand retain no snapshots older than 31 days
Given that we run the job every month:
- I'll keep the last 4 snapshots of
- I'll keep the last 1-2 snapshots of
In other words, this is almost a plain mirror of the above directories, except for
/etc/, which is backed up with some versioning (up to 4 versions).
N:M numbers are as defined in the Prune section of the guide.
#!/bin/sh - # # If there is a global system configuration file, suck it in. # if [ -r /etc/defaults/periodic.conf ] then . /etc/defaults/periodic.conf source_periodic_confs fi rc=0 . $backup_duplicacy_pre for target in $backup_duplicacy_targets do path=`echo $target | cut -d\@ -f 1` keep=`echo $target | cut -d\@ -f 2` cd "$path" echo "[`date`] Pruning $path $keep" $backup_duplicacy -log prune $backup_duplicacy_prune_opts -keep $keep sleep 1 done . $backup_duplicacy_post rc=$? exit $rc
cron daemon checks
/etc/crontab, which triggers the
periodic monthly every month. This defines the
backup job, which comprises
101.duplicacy.prune tasks. Any output is automatically delivered via email (if set).