Disk backup rotation script

rotate-backups.sh

#!/bin/bash
# Rotates disk image backups inside working directory
# Maintains a rolling window of 4 hourly snapshots (uncompressed)
# Plus 7xdays and 6xmonths of older compressed archives

# config
HOURS=4
DAYS=7
MONTHS=6

# current timestamp (seconds)
NOW=$(date +%s);

# move uncompressed snapshots into daily rotation
for file in hourly-*.img; do
  if [ -f $file ]; then
    MOD=$( stat -c %Y $file );
    AGE=$((NOW - MOD));
    MAX=$((HOURS*3600));
    if test $AGE -gt $MAX; then
      echo "$file is over $HOURS hours old"
      DAY=$( date -d "@$MOD" +'%Y%m%d' );
      if [ -f "daily-$DAY.img.gz" ]; then
        echo "Removing hourly snapshot, already have daily ($DAY)"
        rm -v $file
      else
        echo "Compressing daily backup (this can take a long time)"
        mv -v $file "daily-$DAY.img";
        gzip -v "daily-$DAY.img"
      fi
    fi
  fi
done

# move daily snapshots over (DAYS) old into monthly rotation
for file in daily-*.img.gz; do
  if [ -f $file ]; then
    MOD=$( stat -c %Y $file );
    AGE=$((NOW - MOD));
    MAX=$((DAYS*86400));
    if test $AGE -gt $MAX; then
      echo "$file is over $DAYS days old"
      MTH=$( date -d "@$MOD" +'%Y%m' );
      if [ -f "monthly-$MTH.img.gz" ]; then
        echo "Removing daily snapshot, already have monthly ($MTH)"
        rm -v $file
      else
        echo "Preserving monthly backup"
        mv -v $file "monthly-$MTH.img.gz"
      fi
    fi
  fi
done

# remove monthly snapshots over (MONTHS) old
for file in monthly-*; do
  if [ -f $file ]; then
    MOD=$( stat -c %Y $file );
    AGE=$((NOW - MOD));
    MAX=$((MONTHS*2678400));
    if test $AGE -gt $MAX; then
      echo "$file is over $MONTHS months old, removing permanently"
      rm -v file
    fi
  fi
done

I use this to rotate disk images saved to attached storage on one of my Linode servers. (Long story for why).

The actual backup (snapshot) process is not in this script, but is based on this awesome method (I use for MongoDB). That process writes a new disk image every hour to a file named like "hourly-%Y%m%d%H%M%S.img". The rotate script runs beforehand in order to clear space, but could run any time I suppose.

It maintains a rolling window of 4 hourly backups. (This is just to save disk space, change it to 24 if you have room). These images are uncompressed, because compressing and uncompressing is SLOW. I don't want to wait 15 minutes if I have to restore in a hurry.

When an hourly backup becomes over 4 hours old, it is compressed to a daily archive unless one already exists. Then similarly into monthly rotation after a week. I keep six months, but probably only need one. Change that to whatever.

Notes:

• File names are not parsed for timestamps. Actual modification times are used.
• For a 5GB disk image (gzips to 1.5G), this full rotation fills a 40G disk
• Running hourly with HOURS=4: The daily archive compression would occur between 4-5am from the midnight snapshot
• Months are rounded to 31 days :)