Rachitlohani/grep through s3 (gz files)

Created April 30, 2014 17:48

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/Rachitlohani/c78a33c2cca76548865b.js"></script>
Save Rachitlohani/c78a33c2cca76548865b to your computer and use it in GitHub Desktop.

Download ZIP

s3 commands

Raw

grep through s3 (gz files)

s3cmd ls --recursive s3://mys3.com/bucket1/ | awk '{print $4}' | grep '.gz' | xargs -I@ s3cmd get @ - | zgrep 'request' | wc -l

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment