Skip to content

Instantly share code, notes, and snippets.

@DangerBlack
Last active July 4, 2023 08:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save DangerBlack/7d7e7baee05fd1bca01c62718be1047f to your computer and use it in GitHub Desktop.
Save DangerBlack/7d7e7baee05fd1bca01c62718be1047f to your computer and use it in GitHub Desktop.
How to delete the objects in a versioned bucket
#!/usr/bin/bash
# To configure aws cli on Cubbit you can visit
# https://docs.cubbit.io/integrations/aws-cli#installation
#MIT License
#Copyright (c) [2023] [DangerBlack]
#Permission is hereby granted, free of charge, to any person obtaining a copy
#of this software and associated documentation files (the "Software"), to deal
#in the Software without restriction, including without limitation the rights
#to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#copies of the Software, and to permit persons to whom the Software is
#furnished to do so, subject to the following conditions:
#The above copyright notice and this permission notice shall be included in all
#copies or substantial portions of the Software.
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
#SOFTWARE.
aws --endpoint https://s3.cubbit.eu s3api list-object-versions --bucket "bucket-name" --output=json --query='{Objects: [Versions,DeleteMarkers][].{Key:Key,VersionId:VersionId}}' --profile cubbit > keys_dump.json
file='keys_dump.json'
cat "$file" | jq -r '.Objects | keys[]' |
while IFS= read -r value; do
key=$(cat "$file" | jq ".Objects[$value] | .Key")
versionID=$(cat "$file" | jq ".Objects[$value] | .VersionId")
echo "[$value] storing $key $versionID"
echo "aws --endpoint https://s3.cubbit.eu s3api delete-object --bucket \"bucket-name\" --key \"$key\" --version-id \"$versionID\" --profile cubbit" >> remove_keys.sh
done
chmod +x remove_keys.sh
echo "Start actively deleting keys"
./remove_keys.sh
echo "done"
rm remove_keys.sh
rm keys_dump.json
echo "Now bucket is empty"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment