Skip to content

Instantly share code, notes, and snippets.

@rocket-ron
Created August 2, 2015 00:51
Show Gist options
  • Save rocket-ron/952cf4980e3acae4dbf9 to your computer and use it in GitHub Desktop.
Save rocket-ron/952cf4980e3acae4dbf9 to your computer and use it in GitHub Desktop.
Extracting total size of AWS S3 bucket space
# Use the AWS CLI S3 tools to calculate the amount of space used in a bucket from a listing of the bucket contents
# The first awk extracts the 3rd column, which is the size of the key contents in KB
# The second awk sums the values and outputs the total multiplied by 1024 for total number of bytes
aws s3 ls s3://w205-project-twitter-streams/isis | awk '{ print $3 }' | awk '{s+=$1} END {print s*1024}'
# This command uses the AWS CLI S3 ls command to count the number of keys in a bucket
aws s3 ls s3://w205-project-twitter-streams/isis | tee >(wc -l)
# or this
aws s3 ls s3://w205-project-twitter-streams/isis | awk '{ print } END { print NR }'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment