Skip to content

Instantly share code, notes, and snippets.

@andy-b-84
Last active May 23, 2023 01:20
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save andy-b-84/9b9df3dc9ca8f7d50cd910b23cea5e0e to your computer and use it in GitHub Desktop.
Save andy-b-84/9b9df3dc9ca8f7d50cd910b23cea5e0e to your computer and use it in GitHub Desktop.
[AWS][S3] permanently delete every objects from a versioned bucket, typically before destroying the bucket, launching headless commands (make that CPU burn, Nero)
#!/bin/bash
bucket=$1
profile=$2
region=$3
set -e
echo "Removing all versions from $bucket"
versions=`aws --region $region --profile $profile s3api list-object-versions --bucket $bucket |jq '.Versions'`
markers=`aws --region $region --profile $profile s3api list-object-versions --bucket $bucket |jq '.DeleteMarkers'`
let count=`echo $versions |jq 'length'`-1
if [ $count -gt -1 ]; then
echo "removing files"
for i in $(seq 0 $count); do
key=`echo $versions | jq .[$i].Key |sed -e 's/\"//g'`
versionId=`echo $versions | jq .[$i].VersionId |sed -e 's/\"//g'`
echo "aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &"
aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &
done
fi
let count=`echo $markers |jq 'length'`-1
if [ $count -gt -1 ]; then
echo "removing delete markers"
for i in $(seq 0 $count); do
key=`echo $markers | jq .[$i].Key |sed -e 's/\"//g'`
versionId=`echo $markers | jq .[$i].VersionId |sed -e 's/\"//g'`
echo "aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &"
aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &
done
fi
@andy-b-84
Copy link
Author

@thelpbogeyman
Copy link

Nice. Thank you.
One change I made - the command execution.

From:

echo "aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &"
aws --region $region --profile $profile s3api delete-object --bucket $bucket --key $key --version-id $versionId &

To:

cmd="aws --region $region --profile $profile s3api delete-object --bucket $bucket --key \"$key\" --version-id $versionId  --no-cli-pager &"
echo $cmd
eval $cmd

This will better deal with embedded spaces in files/keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment