Skip to content

Instantly share code, notes, and snippets.

View joshdevins's full-sized avatar

Josh Devins joshdevins

View GitHub Profile
@joshdevins
joshdevins / tcpdump.bash
Last active November 12, 2019 12:11
Capture Elasticsearch queries
# everything
sudo tcpdump -i lo0 -A -n -s 0 'tcp dst port 9200 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
@joshdevins
joshdevins / extract.bash
Last active October 16, 2019 14:22
Extract structured query source from a Elasticsearch slow query log for a given an index pattern
jq -r .message | sed -n '/\[\(\INDEX_PATTERN\)\]\[.*\] .*/p' | sed -n 's/.* source\[\(.*\)\]\, id.*/\1/p' | jq -C .
@joshdevins
joshdevins / sparkTuning.scala
Created March 27, 2017 08:40
Spark 1.4.1 shell script to tune memory settings
/**
* Determines the compute and memory layouts given a Spark 1.4.x configuration.
*
* This should be run by applying all of the configuration except for the
* executor settings (as this would waste resources). E.g.:
*
spark-shell \
--master "local" \
--deploy-mode "client" \
--driver-memory 1G \