Skip to content

Instantly share code, notes, and snippets.

@ottomata
Created January 30, 2014 15:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ottomata/8711809 to your computer and use it in GitHub Desktop.
Save ottomata/8711809 to your computer and use it in GitHub Desktop.
### udp2log packet loss monitoring
pipe 10 /usr/bin/packet-loss 10 '\t' >> /var/log/udp2log/packet-loss.log
### Capture all logs with 'zero=' set. The X-Analytics header is set with this
# by mobile varnish frontends upon getting a Wikipedia Zero request.
pipe 1 /bin/grep -P 'zero=\d{3}-\d{2}' >> /a/log/webrequest/zero.tsv.log
### All edits
pipe 1 /usr/bin/udp-filter -F '\t' -p action=submit,action=edit >> /a/log/webrequest/edits.tsv.log
### Mobile traffic filter
# All mobile traffic goes through CC-cp1044 varnish hosts.
pipe 100 /bin/grep -P '(cp1046.eqiad.wmnet|cp1047.eqiad.wmnet|cp1059.eqiad.wmnet|cp1060.eqiad.wmnet|cp3011.esams.wikimedia.org|cp3012.esams.wikimedia.org|cp4011.ulsfo.wmnet|cp4012.ulsfo.wmnet|cp4019.ulsfo.wmnet|cp4020.ulsfo.wmnet)' >> /a/log/webrequest/mobile-sampled-100.tsv.log
### Kafka producer???
pipe 1 kafka-console-producer.sh --broker-list blablabla ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment