This is a very basic method to do rotating snapshots for Elasticsearch. For this to work you will need to have jq installed (http://stedolan.github.io/jq/).
PLEASE, PLEASE, PLEASE don't put this blindly in your commandline and execute it :)
curl -s -S -XGET "localhost:9200/_snapshot/my_s3_repository/_all?pretty=true" | jq '.snapshots[] | .snapshot + " " + .end_time' | sed 's/^.\(.*\).$/\1/' | sort -k 2 -r | awk '{ if (NR > 1) { system("curl -XDELETE " "localhost:9200/_snapshot/my_s3_repository/"$1) } } END { system("curl -XPUT " "localhost:9200/_snapshot/my_s3_repository/`date +\%s`") }'
curl -s -S -XGET "localhost:9200/_snapshot/my_s3_repository/_all?pretty=true"
Fetch a list of all snapshots currently available in your repository.
jq '.snapshots[] | .snapshot + " " + .end_time'
Since the above command returns a JSON object, we fetch the relevant information with jq
, i.e. the name of the snapshots and the time when they were finished.
sed 's/^.\(.*\).$/\1/'
jq
returns a string with quotes at the beginning and the end, since they interfere with the next awk
command, we will get rid of them (there is probably an easier method for this).
sort -k 2 -r
Sort the output in a descending order based on the second column, i.e. date
awk '{ if (NR > 1) { system("curl -XDELETE " "localhost:9200/_snapshot/my_s3_repository/"$1) } } END { system("curl -XPUT " "localhost:9200/_snapshot/my_s3_repository/`date +\%s`") }'
Ok, this is the interesting part. awk
now loops through the backups and deletes every one after the first position (NR > 1
). If you want to keep more backups, you will need to change this number.
Once awk
is finished with that, it creates a new snapshot with a timestamp.
I'm running this script with @daily
in CRON.
Use the raw option for jq (-r) to avoid the extra sed command