gistfile1.txt

a bunch of files named loglines_... contain pixel requests, these correspond to roughly logparse output or instance id logs in s3

joseph 11:46:33 @ hephaestus ~/loglines> /bin/grep -H -F "e=24&" loglines_* > error_pixels


To take a 60 second sample from all live nginx (requires moreutils, as well as probably discover in your path):
#!/bin/bash
parallel -j 150 -i bash -c "source ~/.bashrc; ssh {} \"timeout 60 tail -f /mnt/nginx_access.log\" > loglines_{}" -- $( discover.py nginx | cut -f 3 | sort )

Get a sorted list of top 10 i keys which threw errors:
 /bin/grep -o -E "&i=[^&]+&" error_pixels  | sort | uniq -c | sort -nr | head -n 10
   1957 &i=WEATHERV3&
   1579 &i=AOL2&
   1319 &i=ADVANCEDIGITALV3&
   1143 &i=BROLLNETWORK1&
    950 &i=CENTRO1&
    893 &i=GUMGUM1&
    769 &i=VIACOM1C&
    581 &i=BLEACHERREPORT2&
    487 &i=FOX3&
    303 &i=TURNER1&

On a stats-cache in /mnt/tmp there is an ensure log for stats-cache.py

<stats-cache> vicky ubuntu@54.227.146.39 /mnt/tmp$ cat thepatterns 
KeyError
Exception
ValueError

/bin/grep --color=always -f thepatterns /mnt/tmp/ENSURE-TEMPORARY-stats-cache.py-*