Skip to content

Instantly share code, notes, and snippets.

View philz's full-sized avatar

Philip Zeyliger philz

35 followers · 3 following
View GitHub Profile
@philz
philz / Visualizing Puppet Circular Dependencies
Created March 5, 2009 21:37
# Make sense of puppet circular dependency error messages.
# Tags: graphviz, puppet
# See output at http://i41.tinypic.com/b6yeqv.jpg
$ cat circular.pp
node default {
exec { "/a": require => Exec["/b"] }
exec { "/b": require => Exec["/c"] }
exec { "/c": require => Exec["/a"] }
}
@philz
philz / gist:131367
Created June 17, 2009 17:20
#!/usr/bin/env python
#
# Python code for http://www.cloudera.com/blog/2009/06/17/analyzing-apache-logs-with-piganalyzing-apache-logs-with-pig/
import sys
import math
def rescale(values, low=0, high=4095):
"""Linearly rescales values to be strictly between low and high."""
maxval = max(values)
@philz
philz / unixtime.sh
Created November 30, 2012 17:59
seconds since epoch to readable
# $unixtime 1354232717
# local Thu 29 Nov 2012 03:45:17 PM PST utc Thu 29 Nov 2012 11:45:17 PM GMT
unixtime ()
{
gawk "BEGIN { print \"local\", strftime("'"'"%c"'"'", $1), \"utc\", strftime("'"'"%c"'"'", $1, 1) ; }"
}
# Alternately,
# $date -d @1354298146
# Fri Nov 30 09:55:46 PST 2012
@philz
philz / gist:4440055
Created January 3, 2013 01:30

Header 1

Header 2

Header 3 ### (Hashes on right are optional)

Header 4

Header 5

Markdown plus h2 with a custom ID ## {#id-goes-here}

Link back to H2

This is a paragraph, which is text surrounded by whitespace. Paragraphs can be on one

@philz
philz / gist:4773497
Created February 12, 2013 21:20
Draw depending graph of jstack/stacks debug servlet output
dot -Tpng -o out.png <(cat stacks.txt | awk '/^Thread/ { cur = $2 } /Blocked by/ { print "t"cur, "->", "t"$3 } BEGIN { print "digraph G {"} END { print "}" }')
@philz
philz / gist:4992741
Created February 20, 2013 03:54
Changing the core file size at runtime
echo -n "Max core file size=unlimited:unlimited" > /proc/$(pgrep -f some_name)/limits
@philz
philz / gist:5305400
Created April 3, 2013 21:17
Executing a shell script via MR.
hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.3.0-SNAPSHOT.jar -input /user/philip/single_line -output $(mktemp) -mapper 'touch /tmp/philip_was_here' -jobconf mapred.reduce.tasks=0 -jobconf mapred.map.tasks=0
@philz
philz / gist:5355729
Created April 10, 2013 15:39
Query impala directly from Python. Note that doesn't do things like query cancellation that impala-shell actually does.
#!/usr/bin/python
# Apache License
#
# $python impala.py 'select "hello", "there"'
# ['hello\tthere']
import sys
import os
@philz
philz / gist:5605978
Created May 18, 2013 22:33
Processes with the largest ratio of virtual memory size to resident size.
ps hax -o rss,vsz,command | awk '{ if ($1 > 0) { print $2/$1, $3 } }' | sort -n | tail
@philz
philz / gist:5631456
Last active December 17, 2015 15:28
Using sqlite3 to analyze two jmap heap histograms. Using SQL is a powerful way to query and compare two data sets, like the output of jmap.
jmap -histo:live $pid > old.jmap
sleep $((60*60))
jmap -histo:live $pid > new.jmap
cat old.jmap | awk 'BEGIN { OFS="|" } /:/ { print $2, $3, $4 }' > old.txt
cat new.jmap | awk 'BEGIN { OFS="|" } /:/ { print $2, $3, $4 }' > new.txt
sqlite3
sqlite> create table new(count, bytes, class);
sqlite> create table old(count, bytes, class);
sqlite> .import new.txt "new";
sqlite> .import old.txt "old";