Skip to content

Instantly share code, notes, and snippets.

@Jip-Hop
Created February 28, 2021 17:01
Show Gist options
  • Save Jip-Hop/b9ddb2cc124302a5558659e1298c36ec to your computer and use it in GitHub Desktop.
Save Jip-Hop/b9ddb2cc124302a5558659e1298c36ec to your computer and use it in GitHub Desktop.
Autorun Synology Hyper Backup and Integrity Check with Email Notifications
#!/bin/sh
# This script is to be used in combination with Synology Autorun:
# - https://github.com/reidemei/synology-autorun
# - https://github.com/Jip-Hop/synology-autorun
#
# You need to change the task_id to match your Hyper Backup task.
# Get it with command: more /usr/syno/etc/synobackup.conf
#
# I like to keep "Beep at start and end" disabled in Autorun, because I don't
# want the NAS to beep after completing (could be in the middle of the night)
# But beep at start is a nice way to confirm the script has started,
# so that's why this script starts with a beep.
#
# After the backup completes, the integrity check will start.
# Unfortunately in DSM you can't choose to receive email notifications of the integrity check results.
# So there's a little workaround, at the end of this script, to send an (email) notification.
# The results of the integrity check are taken from the synobackup.log file.
#
# In DSM -> Control Panel -> Notification I enabled email notifications,
# I changed its Subject to %TITLE% and the content to:
# Dear user,
#
# Integrity check for %TASK_NAME% is done.
#
# %OUTPUT%
#
# This way I receive email notifications with the results of the Integrity Check.
#
# Credits:
# - https://github.com/Jip-Hop
# - https://bernd.distler.ws/archives/1835-Synology-automatische-Datensicherung-mit-DSM6.html
# - https://www.beatificabytes.be/send-custom-notifications-from-scripts-running-on-a-synology-new/
task_id=6 # Hyper Backup task id, get it with command: more /usr/syno/etc/synobackup.conf
task_name="USB3 3TB Seagate" # Only used for the notification
/bin/echo 2 > /dev/ttyS1 # Beep on start
startTime=$(date +"%Y/%m/%d %H:%M:%S") # Current date and time
device=$2 # e.g. sde1, passed to this script as second argument
# Backup
/usr/syno/bin/synobackup --backup $task_id --type image
while sleep 60 && /var/packages/HyperBackup/target/bin/dsmbackup --running-on-dev $device
do
:
done
# Check integrity
/var/packages/HyperBackup/target/bin/detect_monitor -k $task_id -t -f -g
# Wait a bit before detect_monitor is up and running
sleep 60
# Wait until check is finished, poll every 60 seconds
/var/packages/HyperBackup/target/bin/detect_monitor -k $task_id -p 60
# Send results of integrity check via email (from last lines of log file)
IFS=''
output=""
title=
NL=$'\n'
while read line
do
# Compute the seconds since epoch for the start date and time
t1=$(date --date="$startTime" +%s)
# Date and time in log line (second column)
dt2=$(echo "$line" | cut -d$'\t' -f2)
# Compute the seconds since epoch for log line date and time
t2=$(date --date="$dt2" +%s)
# Compute the difference in dates in seconds
let "tDiff=$t2-$t1"
# echo "Approx diff b/w $startTime & $dt2 = $tDiff"
# Stop reading log lines from before the startTime
if [[ "$tDiff" -lt 0 ]]; then
break
fi
text=`echo "$line" | cut -d$'\t' -f4`
# Get rid of [Local] prefix
text=`echo "$text" | sed 's/\[Local\]//'`
if [ -z ${title} ]; then
title=$text
fi
output="$output${NL}$text"
done <<<$(tac /var/log/synolog/synobackup.log)
# Hijack the ShareSyncError event to send custom message.
# This event is free to reuse because I don't use the Shared Folder Sync (rsync) feature.
# More info on sending custom (email) notifications: https://www.beatificabytes.be/send-custom-notifications-from-scripts-running-on-a-synology-new/
/usr/syno/bin/synonotify "ShareSyncError" "{\"%OUTPUT%\": \"${output}\", \"%TITLE%\": \"${title}\", \"%TASK_NAME%\": \"${task_name}\"}"
# Sleep a bit more before unmounting the disk
sleep 60
# Unmount the disk
exit 100
@gersteba
Copy link

gersteba commented Jan 14, 2024

Hi!

Now I reworked the script trying to adapt your advice:

#!/bin/sh

# This script is to be called by a Scheduler task as root user,
# having 'Run command / User-defined script' filled in with your script's path.
# i. e. /bin/bash /volume1/Scripts/autobackup.sh
#
# You need to change the task_id to match your Hyper Backup task.
# Get it with command: more /var/packages/HyperBackup/etc/synobackup.conf
# You also need to change the location of USB device and name of the block device associated with 
# the filesystem partition on the USB disk. Find out with command 'df' having the USB device attached.
#
# I like to keep "Beep at start and end" disabled in Autorun, because I don't
# want the NAS to beep after completing (could be in the middle of the night)
# But beep at start is a nice way to confirm the script has started,
# so that's why this script starts with a beep.
#
# After the backup and the version rotation complete, the integrity check will start. 
# If you like to receive the log entries in an e-mail after this script finished,
# check 'Send run details by email' and fill in 'Email' in the Scheduler task settings.
#
# Tested with DSM 7.2-64570 Update 3 and Hyper Backup 4.1.0-3718.
#
# Credits:
# - https://gist.github.com/Jip-Hop/b9ddb2cc124302a5558659e1298c36ec
# - https://derwebprogrammierer.at/

function process_is_active() {
	local _process_name="$1"
	# Find one PID only (-s) across all shells (-x)
	pidof -s -x "/var/packages/HyperBackup/target/bin/${_process_name}" > /dev/null
	return $?
}

task_id=12 # Hyper Backup task id
task_name="[Omnia Auto-Backup]" # Only used for log entries

# Location of USB device and name of the block device associated with the filesystem partition on the USB disk. Find out with command 'df'.
USBDRV=/volumeUSB1/usbshare # See column 'Mounted on' in df result
device=sdq1 # See column 'Filesystem' in df result

#/bin/echo 2 > /dev/ttyS1 # Beep on start

startTime=$(date +"%Y/%m/%d %H:%M:%S")
echo -e "info\t${startTime}\tSYSTEM:\t${task_name} Started." >> /var/log/synolog/synobackup.log

# Backup - Begin
currTime=$(date +"%Y/%m/%d %H:%M:%S") # Current date and time
echo -e "info\t${currTime}\tSYSTEM:\t${task_name} Backup start ..." >> /var/log/synolog/synobackup.log
/usr/syno/bin/synobackup --backup $task_id --type image

sleep 60
while /var/packages/HyperBackup/target/bin/dsmbackup --running-on-dev $device; do
	sleep 60
done
# Backup - End

## Version rotation - Begin
sleep 60
currTime=$(date +"%Y/%m/%d %H:%M:%S")
echo -e "info\t${currTime}\tSYSTEM:\t${task_name} Rotation start ..." >> /var/log/synolog/synobackup.log
while process_is_active 'synoimgbkptool'; do
	sleep 60
done
## Version rotation - End

## Check integrity - Begin
sleep 60
currTime=$(date +"%Y/%m/%d %H:%M:%S")
echo -e "info\t${currTime}\tSYSTEM:\t${task_name} Integrity check start ..." >> /var/log/synolog/synobackup.log
/var/packages/HyperBackup/target/bin/detect_monitor --task-id $task_id --trigger --full --guard

sleep 60
while process_is_active 'detect_monitor'; do
	sleep 60
done
## Check integrity - End

# Sleep a bit more before unmounting the disk
sleep 60

## Unmount USB device - Begin
sync
sleep 10
umount $USBDRV
umountResult=$(/usr/syno/bin/synousbdisk -umount $device; >/tmp/usbtab)
currTime=$(date +"%Y/%m/%d %H:%M:%S")
echo -e "info\t${currTime}\tSYSTEM:\t${umountResult}" >> /var/log/synolog/synobackup.log
## Unmount USB device - End

currTime=$(date +"%Y/%m/%d %H:%M:%S")
echo -e "info\t${currTime}\tSYSTEM:\t${task_name} Finished." >> /var/log/synolog/synobackup.log

## Get results of auto backup (from last lines of log file) - Begin
IFS=''
output=()
NL=$'\n'

while read line
do
    
    # Compute the seconds since epoch for the start date and time
    t1=$(date --date="$startTime" +%s)
    
    # Date and time in log line (second column)
    dt2=$(echo "$line" | cut -d$'\t' -f2)
    # Compute the seconds since epoch for log line date and time
    t2=$(date --date="$dt2" +%s)
    
    # Compute the difference in dates in seconds
    let "tDiff=$t2-$t1"
    
    # Stop reading log lines from before the startTime
    if [[ "$tDiff" -lt 0 ]]; then
        break
    fi
    
    #text=`echo "$line" | cut -d$'\t' -f4`
    text=$(echo "$line" | cut -d$'\t' -f4)
    # Get rid of [Local] prefix
    text=$(echo "$text" | sed 's/\[Local\]//')
    # Add date and time
	text=$(echo "${dt2}  ${text}")
    
	output+=("$text")
    
done <<<$(tac /var/log/synolog/synobackup.log)

n=${#output[*]}
for (( i = n-1; i >= 0; i-- ))
do
    
	echo "${output[i]}"
	
done
## Get results ... - End

exit 0

Seems to work fine.
The only thing, I do not understand, yet:
Why does the rotation already start while dsmbackup is still running?
Compare script with following log entries:

2024/01/13 02:00:01  [Omnia Auto-Backup] Started.
2024/01/13 02:00:01  [Omnia Auto-Backup] Backup start ...
2024/01/13 02:00:15  [Omnia Backup] Backup task started.
2024/01/13 16:55:18  [Omnia Backup] Backup task finished successfully. [599281 files scanned] [71 new files] [9 files modified] [599201 files unchanged]
2024/01/13 16:55:19  [Omnia Backup] Trigger version rotation.
2024/01/13 16:55:57  [usbshare1] Version rotation started from ID [Omnia.hbk].
2024/01/13 16:56:57  [Omnia Auto-Backup] Rotation start ...
2024/01/13 18:16:29  [usbshare1] Rotate version [2023-12-09 02:00:39] from ID [Omnia.hbk].
2024/01/13 18:16:30  [usbshare1] Version rotation completed from ID [Omnia.hbk].
2024/01/13 18:16:59  [Omnia Auto-Backup] Integrity check start ...
2024/01/13 18:17:10  [Omnia Backup] Backup integrity check has started.
2024/01/14 06:48:30  [Omnia Backup] Data integrity check finished. 3574.4 GB data checked this time, accounting for 100.0% of the total data (3574.4 GB data in total, 100.0% checked already).
2024/01/14 07:34:47  [Omnia Backup] Backup integrity check is finished. No error was found.
2024/01/14 07:36:12  Unmount USB device sdq1 succeeded.
2024/01/14 07:36:12  [Omnia Auto-Backup] Finished.

When I tried to remove the
while process_is_active 'synoimgbkptool';
check, the integrity check failed, because the rotation was not finished, yet.
So this check is also necessary.

And:
How can I find out, whether the backup, the rotation and the integrity check worked fine or not, and create different texts in the last log entry accordingly?
i. e. 'Finished successfully.' or 'Failed during rotation.'

@Jip-Hop
Copy link
Author

Jip-Hop commented Jan 14, 2024

Hi @gersteba I suggest you fork this gist and continue the development (and the discussion) in there. Good luck with the script! 🙂

@gersteba
Copy link

I'd prefer to keep your script as it is - my adaption is targeted to my very special needs.
But maybe others can benefit from the additional input and thoughts, too.
Thank you again for your very valuable starting basis!

@Jip-Hop
Copy link
Author

Jip-Hop commented Jan 14, 2024

Sorry but that's exactly the reason I urge you to fork this gist (or create a new one) so you can work on it for your own very special needs. I have no interest in following the conversation any longer but I receive an email for each comment on this gist and I try to reduce the noise in my inbox. I'd prefer not to delete this gist so others may still find it in the future. So please take the development and discussion elsewhere. You can comment the new location here and ask the other participants to join at the new location.

@gersteba
Copy link

gersteba commented Jan 17, 2024

For further discussion on my adapted script please subscribe here:
https://gist.github.com/gersteba/6b07be49aa94c8df1bb88e7db615987d

@Jip-Hop: Sorry for any inconvenience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment