Skip to content

Instantly share code, notes, and snippets.

@verespej
Last active June 22, 2020 13:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save verespej/61cc4bf0d6147a469e6f33b02f2846c7 to your computer and use it in GitHub Desktop.
Save verespej/61cc4bf0d6147a469e6f33b02f2846c7 to your computer and use it in GitHub Desktop.
Delete an S3 versioned bucket (including all versioned objects within it)
#!/usr/bin/env python
# S3 versioned buckets with many objects can be difficult to delete due to
# (1) S3 using delete markers instead of deleting objects and
# (2) web console and cli operations time out when the bucket has many versioned objects.
# This script delete all versioned objects from a bucket, then deletes the bucket.
import boto3
BATCH_SIZE = 1000
BUCKET_NAME = ''
s3 = boto3.resource('s3')
bucket = s3.Bucket(BUCKET_NAME)
object_versions = bucket.object_versions.page_size(count=BATCH_SIZE)
delete_batch = []
batch_num = 1
for ov in object_versions:
delete_batch.append(ov)
if len(delete_batch) >= BATCH_SIZE:
print(f'Deleting batch {batch_num} of {BATCH_SIZE} items..')
bucket.delete_objects(
Delete={
'Objects': [
{
'Key': ver_to_delete.object_key,
'VersionId': ver_to_delete.id
} for ver_to_delete in delete_batch
]
# 'Quiet': True|False
}
)
delete_batch = []
batch_num += 1
bucket.delete()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment