Skip to content

Instantly share code, notes, and snippets.

@JensRantil
Created November 27, 2014 10:20
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save JensRantil/a8150e998250edfcd1a3 to your computer and use it in GitHub Desktop.
Save JensRantil/a8150e998250edfcd1a3 to your computer and use it in GitHub Desktop.
Basic Cassandra -> S3 backup script
#!/bin/bash
KEYSPACE=$1
DATE=`date +"%Y-%m-%d--%H-%M-%S"`
FOLDER_NAME=${DATE}-daily
S3_BUCKET=s3://YOUR-S3-BUCKET
S3_BUCKET_PATH=cassandra/full/`date +"%Y/%m/%d/%H/%M"`/`cat /etc/hostname`
SNAPSHOTID=`uuidgen --time`
PGP_KEY_RECIPIENT=YOUR-PGP-KEY-RECIPIENT
echo "------ NEW RUN ------"
echo "Taking daily db dump for $DATE with id=$SNAPSHOTID"
nodetool snapshot --tag $SNAPSHOTID $KEYSPACE
TABLES=`ls /var/lib/cassandra/data/$KEYSPACE`
for table in $TABLES
do
echo ""
echo "Encrypting and sending all $table files one by one..."
FILES=`find /var/lib/cassandra/data/$KEYSPACE/${table}/snapshots/$SNAPSHOTID -type f`
for filename in $FILES
do
# Need to deal with files one by one to not have the file usage explode
# because we are dealing with hardlinks.
gpg --encrypt --recipient "$PGP_KEY_RECIPIENT" --output $filename.gpg $filename
rm -v $filename
# s3cmd added support for writing files from stdin first somewhere in
# 1.5.0-alpha. Otherwise we would have been able to pipe `gpg` output
# directly into s3cmd. See https://github.com/s3tools/s3cmd/issues/270.
s3cmd put $filename.gpg $S3_BUCKET/$S3_BUCKET_PATH/$table/`basename $filename`.gpg
rm -v $filename.gpg
done
rmdir -v /var/lib/cassandra/data/$KEYSPACE/${table}/snapshots/$SNAPSHOTID
done
echo "DONE!"
echo ""
@dnileshp
Copy link

dnileshp commented Feb 1, 2017

What is PGP_KEY_RECIPIENT=YOUR-PGP-KEY-RECIPIENT in script..how can i get it???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment