frankkusters/backup-synology-through-rsnapshot.md

## backup-synology-through-rsnapshot.md

      
    Raw
  

              backup-synology-through-rsnapshot.md
            
          
    Introduction

I need a backup solution for my Synology DS215j NAS. My requirements are:

a backup solution that pulls data from the Synology DS, so that it is not possible for malware on the DS to corrupt the backups.
a restore solution that is really easy to use for everyone else in my house.
highly frequent backups. With modern storage technologies, it's unnecessary to lose more than an hour's worth of work.

Also, I have a LaCie Network Space 2 lying around, which is perfectly up to this task. Let's install and configure rsnapshot*!
Note: proficiency with Linux is assumed!
* Although rsnapshot is easy to configure, it is slow (see here for more info - note that the NS2 also uses XFS). On my NS2, backing up 450GB (250,000 files) takes about 45 minutes, of which 25 minutes is spent on rm -rf hourly.24. Therefore I will look at transitioning to rdiff-backup in the near future.
Installing software on the Network Space 2

The NS2 does not have the ability to create backups from another NAS (for example through rsync) through its web interface. Therefore, I need to install a backup utility through SSH.


Update the firmware. The latest release (as of February 10th, 2015) fixes the Shellshock vulnerability. It would be a shame to have our little backup server hacked.


Disable software updates. This is so that our modifications will be left alone. Be sure to update manually once in a while.


Enable SSH (the procedure for firmware 2.2.8 should be fine).


As suggested in the manual for step 3, set up login with SSH keys. This makes it easier to administrate the NS2, as the password for root is disabled on every boot.


Install ipkg. Some small details differ. See install-ipkg.sh below.


Install rsnapshot: ipkg install rsnapshot.


Setting up an rsync connection from the NS2 to the Synology DiskStation


Enable the User Home Service on the DS.


Enable the rsync daemon on the DS. Otherwise you'll get strange rsync errors on the NS2.


Create a file named /opt/etc/diskstation-rsync-password.txt with the password of the DS root account in it. Using the root account is required, because DSM does not allow reading of shares through rsync for other user accounts.


Remove world read permissions from the file - else rsync will complain: chmod o-r /opt/etc/diskstation-rsync-password.txt


Configuring rsnapshot

Edit /opt/etc/rsnapshot.conf.

snapshot_root: make sure backups are stored in /shares/MyShare, as /shares/OpenShare is readable by everyone on the network - we don't want that. We'll open up MyShare later.
interval: choose what suits you. Do read the rsnapshot documentation on what these values mean.
rsync_long_args: using SSH and checksumming slows transfer speed down to 3MB/sec. Assuming a gigabit network, the NS2 CPU is the bottleneck. Setting --whole-file and directly using the rsync daemon on the DS will achieve full transfer speeds.
sync_first: if a backup is interrupted, you have an incomplete backup, which will take part in the rotation. Using sync_first eliminates this problem. Do take note of the special way you need to call rsnapshot (see crontab below).
backup: you need to specify every share explicitly.

My rsnapshot.conf is shown below.
Running rsnapshot automatically


Set an editor to edit crontab: export EDITOR=/bin/vi


Edit crontab: crontab -e. See my crontab below.


For added security


Disable SSH logins on the NS2 (disable sshd).


## crontab
# m h  dom mon dow   command
# until things are a bit faster, make backups every 2 hours - and skip hourly backups at night
  0 0,6,8,10,12,14,16,18,20,22   *   *   *    /opt/bin/rsnapshot sync && /opt/bin/rsnapshot hourly
  0 4   *   *   *    /opt/bin/rsnapshot daily
  0 3   *   *   1    /opt/bin/rsnapshot weekly
  0 2   2   *   *    /opt/bin/rsnapshot monthly
  0 2   1  3,9  *    /opt/bin/rsnapshot halfyearly

## install-ipkg.sh
cd /opt
feed=http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/cross/stable
# This path no longer exists.
#feednative=http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/native/unstable
ipk_name=`wget -qO- $feed/Packages | awk '/^Filename: ipkg-opt/ {print $2}'`
wget $feed/$ipk_name
tar -xOvzf $ipk_name ./data.tar.gz | tar -C / -xzvf -
mkdir -p /opt/etc/ipkg
echo "src cross $feed" > /opt/etc/ipkg/feeds.conf
#echo "src native $feednative" >> /opt/etc/ipkg/feeds.conf
export PATH=/opt/bin:$PATH
ipkg update
# I don't want mldonkey.
#ipkg install mldonkey

## rsnapshot.conf
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULES:       #
#                                               #
# This file requires tabs between elements      #
#                                               #
# Directories require a trailing slash:         #
#   right: /home/                               #
#   wrong: /home                                #
#                                               #
#################################################

#######################
# CONFIG FILE VERSION #
#######################

config_version	1.2

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
snapshot_root	/shares/MyShare/

# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire drive.
#
#no_create_root	1

#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp		/opt/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
cmd_rm		/opt/bin/rm

# rsync must be enabled for anything to work.
cmd_rsync	/opt/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
cmd_ssh	/opt/bin/ssh

# Comment this out to disable syslog support.
#cmd_logger	/path/to/logger

# Uncomment this to specify a path to "du" for disk usage checks.
cmd_du		/opt/bin/du

#########################################
#           BACKUP INTERVALS            #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc.      #
#########################################

interval	hourly	10
interval	daily	7
interval	weekly	4
interval	monthly	12
interval	halfyearly	10

############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################

# If your version of rsync supports --link-dest, consider enable this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
link_dest	1

sync_first	1

# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      More than you care to know
#
verbose		2

# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
loglevel	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
logfile		/opt/var/log/rsnapshot

# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# seperate line. Please look up the --include and --exclude options in the
# rsync man page for more details.
#
#include	???
#include	???
#exclude	???
#exclude	???

# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
#exclude_file	/path/to/exclude/file

# Default rsync args. All rsync commands have at least these options set.
#
#rsync_short_args	-a
rsync_long_args	--delete --numeric-ids --relative --delete-excluded --whole-file --password-file=/opt/etc/diskstation-rsync-password.txt

# ssh has no args passed by default, but you can specify some here.
#
#ssh_args	-p 22

# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
#
#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs		0

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile	/opt/var/run/rsnapshot.pid

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
#backup	/home/			localhost/
#backup	/etc/			localhost/
#backup	/opt/etc/		localhost/
#backup	/usr/local/		localhost/
#backup	/etc/passwd		localhost/
#backup	/home/foo/My Documents/		localhost/
#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog
#backup_script		/usr/local/bin/backup_pgsql.sh	localhost/postgres/

# EXAMPLE.COM
#backup	root@example.com:/home/	example.com/
#backup	root@192.168.178.4:/	./

# CVS.SOURCEFORGE.NET
#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	cvs.sourceforge.net/

# RSYNC.SAMBA.ORG
#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/
backup	rsync://root@192.168.178.4/unsorted/	unsorted/
	# m h dom mon dow command
	# until things are a bit faster, make backups every 2 hours - and skip hourly backups at night
	0 0,6,8,10,12,14,16,18,20,22 * * * /opt/bin/rsnapshot sync && /opt/bin/rsnapshot hourly
	0 4 * * * /opt/bin/rsnapshot daily
	0 3 * * 1 /opt/bin/rsnapshot weekly
	0 2 2 * * /opt/bin/rsnapshot monthly
	0 2 1 3,9 * /opt/bin/rsnapshot halfyearly
	cd /opt
	feed=http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/cross/stable
	# This path no longer exists.
	#feednative=http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/native/unstable
	ipk_name=`wget -qO- $feed/Packages \| awk '/^Filename: ipkg-opt/ {print $2}'`
	wget $feed/$ipk_name
	tar -xOvzf $ipk_name ./data.tar.gz \| tar -C / -xzvf -
	mkdir -p /opt/etc/ipkg
	echo "src cross $feed" > /opt/etc/ipkg/feeds.conf
	#echo "src native $feednative" >> /opt/etc/ipkg/feeds.conf
	export PATH=/opt/bin:$PATH
	ipkg update
	# I don't want mldonkey.
	#ipkg install mldonkey
	#################################################
	# rsnapshot.conf - rsnapshot configuration file #
	#################################################
	# #
	# PLEASE BE AWARE OF THE FOLLOWING RULES: #
	# #
	# This file requires tabs between elements #
	# #
	# Directories require a trailing slash: #
	# right: /home/ #
	# wrong: /home #
	# #
	#################################################

	#######################
	# CONFIG FILE VERSION #
	#######################

	config_version 1.2

	###########################
	# SNAPSHOT ROOT DIRECTORY #
	###########################

	# All snapshots will be stored under this root directory.
	snapshot_root /shares/MyShare/

	# If no_create_root is enabled, rsnapshot will not automatically create the
	# snapshot_root directory. This is particularly useful if you are backing
	# up to removable media, such as a FireWire drive.
	#
	#no_create_root 1

	#################################
	# EXTERNAL PROGRAM DEPENDENCIES #
	#################################

	# LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features.
	# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
	#
	# See the README file or the man page for more details.
	#
	cmd_cp /opt/bin/cp

	# uncomment this to use the rm program instead of the built-in perl routine.
	cmd_rm /opt/bin/rm

	# rsync must be enabled for anything to work.
	cmd_rsync /opt/bin/rsync

	# Uncomment this to enable remote ssh backups over rsync.
	cmd_ssh /opt/bin/ssh

	# Comment this out to disable syslog support.
	#cmd_logger /path/to/logger

	# Uncomment this to specify a path to "du" for disk usage checks.
	cmd_du /opt/bin/du

	#########################################
	# BACKUP INTERVALS #
	# Must be unique and in ascending order #
	# i.e. hourly, daily, weekly, etc. #
	#########################################

	interval hourly 10
	interval daily 7
	interval weekly 4
	interval monthly 12
	interval halfyearly 10

	############################################
	# GLOBAL OPTIONS #
	# All are optional, with sensible defaults #
	############################################

	# If your version of rsync supports --link-dest, consider enable this.
	# This is the best way to support special files (FIFOs, etc) cross-platform.
	# The default is 0 (off).
	#
	link_dest 1

	sync_first 1

	# Verbose level, 1 through 5.
	# 1 Quiet Print fatal errors only
	# 2 Default Print errors and warnings only
	# 3 Verbose Show equivalent shell commands being executed
	# 4 Extra Verbose Show extra verbose information
	# 5 Debug mode More than you care to know
	#
	verbose 2

	# Same as "verbose" above, but controls the amount of data sent to the
	# logfile, if one is being used. The default is 3.
	loglevel 3

	# If you enable this, data will be written to the file you specify. The
	# amount of data written is controlled by the "loglevel" parameter.
	logfile /opt/var/log/rsnapshot

	# The include and exclude parameters, if enabled, simply get passed directly
	# to rsync. If you have multiple include/exclude patterns, put each one on a
	# seperate line. Please look up the --include and --exclude options in the
	# rsync man page for more details.
	#
	#include ???
	#include ???
	#exclude ???
	#exclude ???

	# The include_file and exclude_file parameters, if enabled, simply get
	# passed directly to rsync. Please look up the --include-from and
	# --exclude-from options in the rsync man page for more details.
	#
	#include_file /path/to/include/file
	#exclude_file /path/to/exclude/file

	# Default rsync args. All rsync commands have at least these options set.
	#
	#rsync_short_args -a
	rsync_long_args --delete --numeric-ids --relative --delete-excluded --whole-file --password-file=/opt/etc/diskstation-rsync-password.txt

	# ssh has no args passed by default, but you can specify some here.
	#
	#ssh_args -p 22

	# Default arguments for the "du" program (for disk space reporting).
	# The GNU version of "du" is preferred. See the man page for more details.
	#
	#du_args -csh

	# If this is enabled, rsync won't span filesystem partitions within a
	# backup point. This essentially passes the -x option to rsync.
	# The default is 0 (off).
	#
	#one_fs 0

	# If enabled, rsnapshot will write a lockfile to prevent two instances
	# from running simultaneously (and messing up the snapshot_root).
	# If you enable this, make sure the lockfile directory is not world
	# writable. Otherwise anyone can prevent the program from running.
	#
	lockfile /opt/var/run/rsnapshot.pid

	###############################
	### BACKUP POINTS / SCRIPTS ###
	###############################

	# LOCALHOST
	#backup /home/ localhost/
	#backup /etc/ localhost/
	#backup /opt/etc/ localhost/
	#backup /usr/local/ localhost/
	#backup /etc/passwd localhost/
	#backup /home/foo/My Documents/ localhost/
	#backup /foo/bar/ localhost/ one_fs=1, rsync_short_args=-urltvpog
	#backup_script /usr/local/bin/backup_pgsql.sh localhost/postgres/

	# EXAMPLE.COM
	#backup root@example.com:/home/ example.com/
	#backup root@192.168.178.4:/ ./

	# CVS.SOURCEFORGE.NET
	#backup_script /usr/local/bin/backup_rsnapshot_cvsroot.sh cvs.sourceforge.net/

	# RSYNC.SAMBA.ORG
	#backup rsync://rsync.samba.org/rsyncftp/ rsync.samba.org/rsyncftp/
	backup rsync://root@192.168.178.4/unsorted/ unsorted/