Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
A script to delete all objects, versions and delete markers from an s3 bucket.
#!/usr/bin/env python
import boto3
client = boto3.client('s3')
Bucket = 'a-bucket'
Prefix = 'a-prefix' # leave blank to delete the entire contents
IsTruncated = True
MaxKeys = 1000
KeyMarker = None
while IsTruncated == True:
if not KeyMarker:
version_list = client.list_object_versions(
Bucket=Bucket,
MaxKeys=MaxKeys,
Prefix=Prefix)
else:
version_list = client.list_object_versions(
Bucket=Bucket,
MaxKeys=MaxKeys,
Prefix=Prefix,
KeyMarker=KeyMarker)
try:
objects = []
versions = version_list['Versions']
for v in versions:
objects.append({'VersionId':v['VersionId'],'Key': v['Key']})
response = client.delete_objects(Bucket=Bucket,Delete={'Objects':objects})
print response
except:
pass
try:
objects = []
delete_markers = version_list['DeleteMarkers']
for d in delete_markers:
objects.append({'VersionId':d['VersionId'],'Key': d['Key']})
response = client.delete_objects(Bucket=Bucket,Delete={'Objects':objects})
print response
except:
pass
IsTruncated = version_list['IsTruncated']
KeyMarker = version_list['NextKeyMarker']
@abrkn
Copy link

abrkn commented Apr 4, 2018

ubuntu@ip-172-31-1-250:~$ python3 delete_all_objects.py
  File "delete_all_objects.py", line 24
    try:
       ^
TabError: inconsistent use of tabs and spaces in indentation

Loading

@clouduser27
Copy link

clouduser27 commented Oct 16, 2018

ubuntu@ip-172-31-1-250:~$ python3 delete_all_objects.py
  File "delete_all_objects.py", line 24
    try:
       ^
TabError: inconsistent use of tabs and spaces in indentation

Try this:

#!/usr/bin/env python
import boto3

client = boto3.client('s3')
Bucket = 'bucketname' #Enter your bucket name
Prefix = 'prefix' # leave blank to delete the entire contents
IsTruncated = True
MaxKeys = 1000
KeyMarker = None

while IsTruncated == True:
    if not KeyMarker:
        version_list = client.list_object_versions(
            Bucket=Bucket,
            MaxKeys=MaxKeys,
            Prefix=Prefix
        )
    else:
        version_list = client.list_object_versions(
            Bucket=Bucket,
            MaxKeys=MaxKeys,
            KeyMarker=KeyMarker,
            Prefix=Prefix
        )
try:
    objects = []
    versions = version_list ['Versions']
    for v in versions:
        objects.append({'VersionId':v['VersionId'],'Key':v['Key']})
        response = client.delete_objects(Bucket=Bucket,Delete={'Objects':objects})
        print (response)
except:
    pass
    try:
        objects = []
        delete_markers = version_list['DeleteMarkers']
        for d in delete_markers:
            objects.append({'VersionId':d['VersionId'],'Key': d['Key']})
            response = client.delete_objects(Bucket=Bucket,Delete={'Objects':objects})
            print (response)
    except:
        pass
        IsTruncated = version_list['IsTruncated']            
        KeyMarker = version_list['NextKeyMarker']

Loading

@ashok-workouts
Copy link

ashok-workouts commented Oct 30, 2020

How to add marker to specific object in s3?

Loading

@dimitrigraf
Copy link

dimitrigraf commented Oct 4, 2021

Hi there!

Thanks a lot for this gist! I reused your code for a script that completely deletes the bucket's contents. I reworked and extended it a bit with an argument parser to be able to pass bucket name and profile. The user can also specify if he wants to delete the bucket after cleaning it.
It's work in progress but I'm happy to share it here, if anyone is interested: https://github.com/dimitrigraf/python-scripts/blob/main/empty-s3-bucket

I do have a question regarding the very last line of the script, where the KeyMarker variable is set. When I ran the script a few times on a test bucket with 1010 objects, the script fails in the very last loop because NextKeyMarker cannot be found/looked up anymore. I guess that makes sense. But since I'm not very experienced with Python, I was wondering how to properly catch such exceptions.

Any ideas?

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment