Skip to content

Instantly share code, notes, and snippets.

@aldegoeij

aldegoeij/gist:4e67b672af4fc687ca69ddeba1ec38c2

Last active September 6, 2021 09:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aldegoeij/4e67b672af4fc687ca69ddeba1ec38c2 to your computer and use it in GitHub Desktop.
Save aldegoeij/4e67b672af4fc687ca69ddeba1ec38c2 to your computer and use it in GitHub Desktop.
Download ZIP
How to download a bunch of log files from AWS S3 if they are not using prefixes properly
Raw
gistfile1.sh
# brew install parallel first, forget about xargs and friends
aws s3 ls s3://___your_bucket_name___ --output=table | awk '{print $4}' | grep ___the_date_prefix___ | parallel -a - -exec aws s3 cp s3://___your_bucket_name___/{} .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment