Skip to content

Instantly share code, notes, and snippets.

@raine
Created March 22, 2016 07:37
Show Gist options
  • Save raine/3954bc0172b57a429ca0 to your computer and use it in GitHub Desktop.
Save raine/3954bc0172b57a429ca0 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash
aws s3api list-objects --bucket stuff --prefix photo/ --delimiter '/' |\
ramda '.CommonPrefixes' 'pluck \Prefix' 'map split "/"' 'map nth 1' -R |\
xargs -P4 -I % aws s3api list-objects --bucket stuff --prefix photo/% \
--query '{%: [sum(Contents[].Size), length(Contents[])]}' |\
ramda --slurp -R \
merge-all to-pairs \
'map -> zip-with ((k,v) -> "#k.#{it.0}:#v|g"), <[ s3_total_size s3_count ]>, it.1' \
flatten \
"map concat '$statsd__prefix.'" |\
xargs -t -n1 ./statsd-client
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment