Skip to content

Instantly share code, notes, and snippets.

@kwent
Last active November 27, 2019 05:38
Show Gist options
  • Save kwent/82f544dd0488619fd596 to your computer and use it in GitHub Desktop.
Save kwent/82f544dd0488619fd596 to your computer and use it in GitHub Desktop.
NEO4J Backup to AWS S3 Shell Script
#!/bin/sh -
#title :backup_neo4j_to_s3.sh
#description :This script is creating a NEO4J Backup through neo4j-backup tool,
# tar and compress the backup file via lzo algorithm compression, and upload it to AWS S3.
#author :Quentin Rousseau <contact@quent.in>
#date :2014-07-10
#version :1.0
#usage :sh backup_neo4j_to_s3.sh ip port destination | eg. sh backup_neo4j_to_s3.sh 127.0.0.1 6362 /mnt/datadisk/backup
#dependencies :apt-get update && apt-get install lzop && apt-get install awscli.
#==============================================================================
#Initialization
TODAY=$(date +%Y-%m-%d)
NEO4J_SERVER=$1
NEO4J_BACKUP_BINARY=/opt/neo4j-enterprise-1.9.8/bin/neo4j-backup
NEO4J_BACKUP_PORT=$2 # Default port 6362
NEO4j_BACKUP_DESTINATION_FOLDER=$3
BACKUP_FILE_NAME=neo4j-backup-$TODAY.tar.gz.lzo
AWS_BUCKET=mybucket.quent.in.s3-website-us-east-1.amazonaws.com
AWS_REGION=us-east-1
export TZ=America/Los_Angeles
export JAVA_HOME=/opt/jdk1.7.0_55
export PATH=$PATH:$JAVA_HOME/bin
export AWS_ACCESS_KEY_ID=SECRET_KEY
export AWS_SECRET_ACCESS_KEY=ACCESS_KEY
#Helping functions
benchmark_and_execute() {
echo "\n[START] - $1"
date1=$(date +"%s")
eval $2
date2=$(date +"%s")
echo "[END] - $1"
diff=$(($date2-$date1))
echo "[BENCHMARK] - $(($diff / 60)) minutes and $(($diff % 60)) seconds elapsed.\n"
}
echo "[START] - NEO4j BACKUP TO S3 - $TODAY"
echo "[CONF] - NEO4J SERVER: $NEO4J_SERVER | NEO4J BACKUP PORT: $NEO4J_BACKUP_PORT"
echo "[CONF] - NEO4J BACKUP DESTINATION FOLDER IS: $NEO4j_BACKUP_DESTINATION_FOLDER"
echo "[CONF] - AWS CONFIGURATION | AWS BUCKET: $AWS_BUCKET | AWS REGION: $AWS_REGION"
cmd="$NEO4J_BACKUP_BINARY -from single://$NEO4J_SERVER:$NEO4J_BACKUP_PORT -to $NEO4j_BACKUP_DESTINATION_FOLDER"
benchmark_and_execute "NEO4j BACKUP" "$cmd"
if [ ! -d "$NEO4j_BACKUP_DESTINATION_FOLDER" ]; then
echo "[ERROR] - $NEO4j_BACKUP_DESTINATION_FOLDER doesn't exist! Backup canceled."
return
fi
cmd="tar -cvf $NEO4j_BACKUP_DESTINATION_FOLDER.tar.gz $NEO4j_BACKUP_DESTINATION_FOLDER --remove-files"
benchmark_and_execute "TAR FOLDER" "$cmd"
cmd="lzop $NEO4j_BACKUP_DESTINATION_FOLDER.tar.gz -U" # -U unlink .tar.gz if success
benchmark_and_execute "COMPRESS TAR FILE WITH LZO ALGORITHM" "$cmd"
cmd="mv $NEO4j_BACKUP_DESTINATION_FOLDER.tar.gz.lzo $BACKUP_FILE_NAME"
benchmark_and_execute "RENAMING FILE" "$cmd"
cmd="aws s3 cp $BACKUP_FILE_NAME s3://$AWS_BUCKET --region $AWS_REGION"
benchmark_and_execute "UPLOAD COMPRESSED FILE TO AWS S3" "$cmd"
cmd="rm $BACKUP_FILE_NAME"
benchmark_and_execute "DELETE BACKUP FROM LOCAL" "$cmd"
echo "[END] - NEO4j BACKUP TO S3 - $TODAY"
Copy link

ghost commented May 31, 2016

cmd="rm $BACKUP_FILE_NAME && rm -Rf $NEO4j_BACKUP_DESTINATION_FOLDER"

This line looks pretty dangerous if someone passes / as an argument by mistake and are running it as root. An if check would be great to have here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment