phretor/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Periodic off-site replication with FreeBSD, duplicacy and B2

I've recently discovered duplicacy, and I've decided to use it with an B2 backend to keep a mirrored offsite replication of the main directories of my home NAS.
I use the periodic system, which offers a nice abstraction over crontab for periodic tasks, and duplicacy to perform the actual backup. To setup duplicacy, I've followed the guide. In simple words, for each directory that I want to replicate, I setup duplicacy in it, which in turns means that you'll have to have a .duplicacy subdirectory in it.
FreeBSD's Periodic

First off, I've added the following to /etc/defaults/periodic.conf. The prefix, 666, is on purpose.
# 666.backup
monthly_backup_enabled="YES"                            # Replicate offsite

The monthly job is invoked every month, and more specifically as defined in the /etc/crontab:
$ grep monthly /etc/crontab
# Perform daily/weekly/monthly maintenance.
30      5       1       *       *       root    periodic monthly

The backup job

The above variable enables the 666.backup job, defined in /etc/periodic/monthly/666.backup (make sure the executable bit is set).
#!/bin/sh
#

# If there is a global system configuration file, suck it in.
#
if [ -r /etc/defaults/periodic.conf ]
then
    . /etc/defaults/periodic.conf
    source_periodic_confs
fi

case "$monthly_backup_enable" in
    [Yy][Ee][Ss])
        echo ""
        echo "Backup:"

        periodic /etc/periodic/backup || rc=3;;

    *)  rc=0;;
esac

exit $rc

So, to turn it on/off, just change the monthly_backup_enabled variable accordingly.
The backup periodic task triggers the execution of any executable script, in lexicographic order, found in the /etc/periodic/backup directory. In there, I've defined two scripts:

/etc/periodic/backup/100.duplicacy - that is the actual backup
/etc/periodic/backup/101.duplicacy.prune - that prunes any old snapshot (see below)

/etc/periodic/backup/100.duplicacy

#!/bin/sh -
#

# If there is a global system configuration file, suck it in.
#
if [ -r /etc/defaults/periodic.conf ]
then
    . /etc/defaults/periodic.conf
    source_periodic_confs
fi

rc=0

. $backup_duplicacy_pre

for target in $backup_duplicacy_targets
do
        path=`echo $target | cut -d\@ -f 1`
        cd "$path"
        echo "[`date`] Backing up $path"
        $backup_duplicacy -log backup -stats
        sleep 1
done

. $backup_duplicacy_post

rc=$?

exit $rc

The main variable is $backup_duplicacy, which points to the duplicacy executable, which is defined in the /etc/defaults/duplicacy.conf file:
# 100.duplicacy
# use <path>@<prune n:m> syntax - https://github.com/gilbertchen/duplicacy/blob/master/GUIDE.md#prune
backup_duplicacy_targets="/etc@0:120 /data/home@0:31 /data/tm@0:31 /data/freezer@0:31 /data/dropbox@0:31"
backup_duplicacy_pre="/root/bin/dup_pre.sh"             # keep this safe!
backup_duplicacy_post="/root/bin/dup_post.sh"
backup_duplicacy="/usr/local/bin/duplicacy"

Cleanup tasks

Notice the $backup_duplicacy_pre variable, which point to a script that define environment variables to hold the duplicacy secrets (that's why you wanna keep them safe). In my case:
#/root/bin/dup_pre.sh

export DUPLICACY_PASSWORD="***"     # duplicacy backup encryption password
export DUPLICACY_B2_ID="***"        # B2 client ID
export DUPLICACY_B2_KEY="***"       # B2 client secret

The $backup_duplicacy_post script takes care of unsetting these variables after duplicacy is done.
#/root/bin/dup_pre.sh

unset DUPLICACY_PASSWORD
unset DUPLICACY_B2_ID
unset DUPLICACY_B2_KEY

Backup target and retain policy definition

The backup targets (i.e., list of directories to be backed up) is defined in the $backup_duplicacy_targets variable, which in my case is:
backup_duplicacy_targets="/etc@0:120 /data/home@0:31 /data/tm@0:31 /data/freezer@0:31 /data/dropbox@0:31"

So, I backup:

/etc and retain no snapshot older than 120 days
/data/home and retain no snapshot older than 31 days
/data/tm and retain no snapshots older than 31 days
/data/freezer and retain no snapshots older than 31 days
/data/dropbox and retain no snapshots older than 31 days

Given that we run the job every month:

I'll keep the last 4 snapshots of /etc
I'll keep the last 1-2 snapshots of /data/home, /data/tm, /data/freezer, and /data/dropbox

In other words, this is almost a plain mirror of the above directories, except for /etc/, which is backed up with some versioning (up to 4 versions).
The N:M numbers are as defined in the Prune section of the guide.
/etc/periodic/backup/101.duplicacy.prune

#!/bin/sh -
#

# If there is a global system configuration file, suck it in.
#
if [ -r /etc/defaults/periodic.conf ]
then
    . /etc/defaults/periodic.conf
    source_periodic_confs
fi

rc=0

. $backup_duplicacy_pre

for target in $backup_duplicacy_targets
do
        path=`echo $target | cut -d\@ -f 1`
        keep=`echo $target | cut -d\@ -f 2`
        cd "$path"
        echo "[`date`] Pruning $path $keep"
        $backup_duplicacy -log prune $backup_duplicacy_prune_opts -keep $keep
        sleep 1
done

. $backup_duplicacy_post

rc=$?

exit $rc

Summary

The cron daemon checks /etc/crontab, which triggers the periodic monthly every month. This defines the backup job, which comprises 100.duplicacy and 101.duplicacy.prune tasks. Any output is automatically delivered via email (if set).