Skip to content

Instantly share code, notes, and snippets.

@skout23
Created February 1, 2019 19:27
Show Gist options
  • Save skout23/005b44a8c5a51bce2bd598b263cbfe12 to your computer and use it in GitHub Desktop.
Save skout23/005b44a8c5a51bce2bd598b263cbfe12 to your computer and use it in GitHub Desktop.
Get the latest size in bytes of all s3 buckets given a list of profiles
#!/bin/bash
aws_profile=("default" "otherprofile");
region="us-east-1"
# setting the expected date() format BSD style (macos)
start_time="$(date -v-2d '+%Y-%m-%d')"
end_time="$(date '+%Y-%m-%d')"
#loop AWS profiles array incase we provide more than 1 profile
for profile in "${aws_profile[@]}"; do
# pull array of s3 buckets
buckets=($(aws --profile "${profile}" --region $region s3 ls s3:// --recursive | awk '{print $3}'))
#loop S3 buckets
for bucket in "${buckets[@]}"; do
# pull bucket size in bytes, extract last .Average value with jq, convert to human readable with numfmt
bucket_size=$(aws --profile "${profile}" --region $region cloudwatch get-metric-statistics \
--start-time "$start_time" --end-time "$end_time" --period 86400 --namespace AWS/S3 \
--statistics Average --metric-name BucketSizeBytes \
--dimensions Name=BucketName,Value=$bucket Name=StorageType,Value=StandardStorage | \
jq -r '.Datapoints[-1] | [.Average] | @csv' | \
numfmt --to=iec)
echo "$bucket $bucket_size"
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment