Skip to content

Instantly share code, notes, and snippets.

@shapeshed
Last active September 12, 2022 09:10
  • Star 8 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save shapeshed/e25bdf3b1116899fa8c47c16db9aa8e0 to your computer and use it in GitHub Desktop.
Fetch AWS Cloudfront Logs, decompress, combine into a single file and remove comments
#!/usr/bin/env bash
BUCKET=$1
CWD=$(pwd)
if [[ -n $1 ]]; then
aws s3 sync s3://$BUCKET/cf-logs .
cat *.gz > combined.log.gz
find $CWD ! -name 'combined.log.gz' -name '*.gz' -type f -exec rm -f {} +
gzip -d combined.log.gz
sed -i '/^#/ d' combined.log
exit 0
else
echo "Error: no bucket name provided"
exit 1
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment