Delete everything in an S3 bucket (in a hurry)

delete-s3.sh

# I needed to delete ~300k objects from an S3 bucket in a hurry. The better and cheaper solution is to use a lifecycle rule, but those can take a day or two to take effect.

bucket="i-want-to-lose-all-my-data"

mkdir -p deletes
# 1. List all the objects in the bucket, transform them 1000 at a time into request objects for DeleteObjects, write each to a separate file
aws s3api list-objects-v2 --bucket "${bucket}" \
  | jq -c '.Contents | _nwise(1000) | map({ Key: .Key }) | { Objects: ., Quiet: true }' \
  | awk '{ f = "deletes/" NR ".json"; print $0 > f; close(f) }'

# 2. delete everything
exit 1 # remove this line if you really mean it
ls -1 deletes/*.json \
  | sort -n \
  | gxargs -n 1 -P 10 -I % aws s3api delete-objects --bucket "${bucket}" --delete file://%

On macOS, the built-in xargs does not have -P. Install gxargs with brew install findutils. On Linux, use the regular xargs.