Skip to content

Instantly share code, notes, and snippets.

@rduque1
Forked from nashjain/delete_all_aws_s3_object_versions.sh
Last active October 4, 2020 15:19
Show Gist options
  • Save rduque1/69781146bf22de257f66e1b1d0d23eb5 to your computer and use it in GitHub Desktop.
Save rduque1/69781146bf22de257f66e1b1d0d23eb5 to your computer and use it in GitHub Desktop.
Let's assume you backup a file every hour on AWS S3 and you want to clean up versions that are older than a week. However for the older versions, you want to leave one version per day just in case if you need them. Following script does that for you.
for id in `(cat toDel.txt)`; do echo "aws --profile wasabi s3 --endpoint-url=<endpoint> rm --recursive s3://<bucket>/id_$id/"; done
#!/bin/bash
deleteBefore=`date --date="1 week ago" +%F`
bucket=$1
fileToDelete=$2
fileName='aws_delete.json'
rm $fileName
echo "Removing all versions of $fileToDelete from $bucket"
versionsToDelete=`aws s3api list-object-versions --bucket "$bucket" --prefix "$fileToDelete" --query "Versions[?(LastModified<'$deleteBefore' && (contains(LastModified, 'T0') || contains(LastModified, 'T1') || contains(LastModified, 'T20') || contains(LastModified, 'T21') || contains(LastModified, 'T22')))].{Key: Key, VersionId: VersionId}"`
echo "{\"Objects\":$versionsToDelete, \"Quiet\":true}" > aws_delete.json
aws s3api delete-objects --bucket "$bucket" --delete file://$fileName
# s3api delete-objects can handle upto 1000 records
echo "Delete successsful"
# We leave the aws_delete.json file, in case you want to later see what happened.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment