Skip to content

Instantly share code, notes, and snippets.

View randomstatistic's full-sized avatar

Jeff Wartes randomstatistic

View GitHub Profile
@randomstatistic
randomstatistic / solr_garbage_analysis.txt
Last active March 22, 2016 00:00
Solr Garbage Analysis
This was against a pretty specific index, using a very specialized query corpus, with lots of caveats. Be careful about comparisions.
Index stats: 85M docs/shard, 3 shards, 1 node.
Query stats: 142k queries. 78k use a simple facet.query, 27k do geospatial radius, 115 use CollapseQParser
CollapsingQParserPlugin:
22% of garbage by size
These were huge, perhaps a half-dozen allocations
Lines 510,512 CollapsingQParserPlugin
SolrIndexSearcher: (getDocListAndSetNC)
56% of garbage by size
@randomstatistic
randomstatistic / FutureBuffer
Created July 17, 2015 18:03
Block simultaneous future creation beyond a threshold
import java.util.concurrent.{TimeUnit, LinkedBlockingQueue}
import scala.annotation.tailrec
import scala.collection.JavaConverters._
import scala.concurrent.Future
/**
* Thread-safe lock on Future generation. The put() method accepts futures without blocking so long as there are less than
* $size futures that have been added via put() that are still alive. If more than $size futures are still running,
* calling put() *blocks* the calling thread until some of the current futures finish.
@randomstatistic
randomstatistic / cached_by_extension.sh
Last active January 23, 2018 17:13
Investigate linux filesystem cache usage
#!/bin/bash
dir=$1
if [ "$1" == "" ]; then
echo "Must provide a directory as an argument"
exit 1
fi
ftypes=$(find $1 -type f -size +10c | grep -E ".*\.[a-zA-Z0-9]*$" | sed -e 's/.*\(\.[a-zA-Z0-9]*\)$/\1/' | sort | uniq)
@randomstatistic
randomstatistic / gist:a7a026798880e1003777
Created May 7, 2014 18:38
Microsecond timing of DNS resolution
for x in `seq 1 20`; do strace -f -tt -o /tmp/st dig
www.google.com > /dev/null && grep -P '(send|recv)msg\(20'
/tmp/st | grep -v EAGAIN | awk '/sendmsg/ {f= substr($2,7)} /recvmsg/
{e= substr($2,7)} END {print e-f}'; done | sort -n