Skip to content

Instantly share code, notes, and snippets.

@balajidl
Last active January 24, 2023 07:18
Show Gist options
  • Save balajidl/44c9389835547fa40e88e881ac1d40ee to your computer and use it in GitHub Desktop.
Save balajidl/44c9389835547fa40e88e881ac1d40ee to your computer and use it in GitHub Desktop.
Run gitlab backup script, upload repo and folders to s3, log and inform via sendgrid email
LOGFILE="/var/log/gitlab-backup.log"
echo "**************START*************************" >> $LOGFILE
#---------1. GITLAB git repos backup ---------
echo "Start the gitlab backup process" >> $LOGFILE
/opt/gitlab/bin/gitlab-rake gitlab:backup:create SKIP=uploads >> $LOGFILE
echo "git backup done" >> $LOGFILE
echo "Now, uploading gitlab repo to s3" >> $LOGFILE
DATE=`date +%Y_%m_%d`
FILENAME="*${DATE}*gitlab_backup*"
EE="$(find /mnt/data/gitlab/ -type f -name $FILENAME)"
# Todo check $EE is not empty
echo "Gitlab backup repo file to be uploaded is: " $EE >> $LOGFILE
/usr/local/bin/aws s3 cp ${EE} s3://meow/ >> $LOGFILE
#---------1. GITLAB uploads backup ---------
echo "Start - gitlab uploads folder to s3" >> $LOGFILE
BAK_DES=/mnt/data/gitlab/backups/
BAK_SOURCES=/mnt/data/gitlab/uploads/
BAK_DATE=`date +%F`
BAK_DATETIME=`date +%F-%H%M`
BAK_FOLDER=${BAK_DEST}/${BAK_DATE}
BAK_FILE=${BAK_DES}uploads-${BAK_DATETIME}.tar.gz
echo "creating tar.gz file at " ${BAK_FILE} >> $LOGFILE
tar czPf ${BAK_FILE} ${BAK_SOURCES}
echo "Now uploading gitlab uploads folder to s3"
/usr/local/bin/aws s3 cp ${BAK_FILE} s3://meow/ >> $LOGFILE
#---------3. Delete old files ---------
echo 'Deleting backup older than '${KEEP_DAYS}' days' >> $LOGFILE
find /mnt/data/gitlab/backups/ -type f -name '*.tar' -mtime +3 -exec rm {} \;
find /mnt/data/gitlab/backups/ -type f -name '*.tar.gz' -mtime +3 -exec rm {} \;
#---------4. Send email to me to make me smile ---------
SENDGRID_API_KEY="long-key-here"
BAK_DATETIME=`date +%F-%H:%M`
SUBJECT="Gitlab backup to succcessful: ${BAK_DATETIME}"
REQUEST_DATA='{"personalizations": [{
"to": [{ "email": "foo@foo.bar" }],
"subject": "'"$SUBJECT"'"
}],
"from": {
"email": "foo@foo.bar",
"name": "Code.spritle.com"
},
"content": [{
"type": "text/plain",
"value": "Keep smiling"
}]
}';
curl -X "POST" "https://api.sendgrid.com/v3/mail/send" \
-H "Authorization: Bearer $SENDGRID_API_KEY" \
-H "Content-Type: application/json" \
-d "$REQUEST_DATA"
echo "Sent email notification via sendgrid" >> $LOGFILE
echo "***************END***************************" >> $LOGFILE
@Hemant0601
Copy link

This script seems good. But its missing the custom configurations like where to make changes if I want to use my own s3 bucket with accesskey and secret

@ghstridr
Copy link

ghstridr commented Sep 1, 2022

The aws cli should automatically pick that up if it's configured correctly. You never place those or any other credentials in a executing script or environment variables where they might be exposed to others accessing the machine.

@Rohlik
Copy link

Rohlik commented Jan 23, 2023

Why you don't use native S3 integration in Gitlab for uploading backup into cloud? 🤔

@spritle
Copy link

spritle commented Jan 24, 2023

@Rohlik The very idea I created this custom script was my gitlab instance auto backup to s3 was not working and was failing for unknown reasons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment