Skip to content

Instantly share code, notes, and snippets.

View tinkerware's full-sized avatar

Cagatay Kavukcuoglu tinkerware

24 followers · 12 following
View GitHub Profile
@esammer
esammer / dist.xml
Last active August 29, 2015 14:00
How to set up a maven project for sane docbook generation.
<!--
~ Copyright (c) 2014 Scaling Data. All rights reserved. This gist is licensed under the Apache Software License v2.0.
-->
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>dist</id>
@lkuper
lkuper / peter-alvaro-halpern-pwl-notes.md
Last active August 29, 2015 14:05
Notes from Peter Alvaro's talk at PWL, August 21, 2014

"Consensus is impossible" is a gloss for "there's always an execution that doesn't terminate".

Once you've internalized that you can't distinguish slow nodes from dead nodes, how do you deal with that?

"When you're engaged in a battle of wits with a Sicilian and death is on the line, you certainly need to think about epistemic logic."

A correct distributed program achieves (nontrivial) distributed property X. So we need to ask:

  1. is X even attainable?
  2. what's the cheapest* protocol that gets me X? (* according to some cost metric)
@jdmaturen
jdmaturen / example_output
Last active August 29, 2015 14:16
From an individual box's viewpoint probabilistically limit the global concurrency of requests of a given actor based on observed local concurrency, the cluster size, and a set threshold.
burp-2:foo jd$ python probabilistic_strategy.py 1000 16 64
request_count: 1000 cluster_size: 16 threshold: 64
probability of being quotad at a given local concurrency:
1 0.015070684079
2 0.07629533815
3 0.202571187171
4 0.378831226431
mean observed global concurrency limit: 64.033
@aphyr
aphyr / mostly-small-nonempty-subset.clj
Created May 2, 2015 00:14
(defn mostly-small-nonempty-subset
"Returns a subset of the given collection, with a logarithmically decreasing
probability of selecting more elements. Always selects at least one element.
(->> #(mostly-small-nonempty-subset [1 2 3 4 5])
repeatedly
(map count)
(take 10000)
frequencies
sort)
anonymous
anonymous / file1.ab
Created May 7, 2011 19:44
cagatayk@Odyssey ~
$ for dep in $(brew deps imagemagick); do brew remove --force $dep; done
Uninstalling jasper...
Uninstalling jpeg...
Uninstalling libtiff...
Uninstalling little-cms...
cagatayk@Odyssey ~
$ brew install imagemagick
Also installing dependencies: jpeg, libtiff, little-cms, jasper
@Kalyse
Kalyse / arrowgasm.less
Created May 9, 2012 13:55
LessCSS Mixing for Providing Arrows
.arrowgasm(@position: top, @size : "4px", @background-color : #88b7d5, @border-width: "2px", @border-color : #c2e1f5, @arrowClass : "arrow_box"){
(~".@{arrowClass}") {
position: relative;
background: @background-color;
border: @size solid @border-color;
}
(~".@{arrowClass}:after"), (~".@{arrowClass}:before") {
bottom: 100%;
border: solid transparent;
@h2oai
h2oai / latency.txt
Created June 5, 2012 01:02 — forked from jboner/latency.txt
Latency numbers every programmer should know
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD 150,000 ns 0.15 ms
Read 1 MB sequentially from memory 250,000 ns 0.25 ms
Round trip within same datacenter 500,000 ns 0.5 ms
@coltfred
coltfred / HFileInputFormat.scala
Created August 14, 2012 19:05 — forked from leifwickland/HFileInputFormat.scala
Allows an HFile to be used as the input to MapReduce.
import org.apache.hadoop.fs.Path
import org.apache.hadoop.hbase.io.hfile.{ HFile, HFileScanner }
import org.apache.hadoop.hbase.io.hfile.HFile.Reader
import org.apache.hadoop.hbase.io.ImmutableBytesWritable
import org.apache.hadoop.hbase.KeyValue
import org.apache.hadoop.mapreduce.{ JobContext, InputSplit, TaskAttemptContext, RecordReader }
import org.apache.hadoop.mapreduce.lib.input.{ FileInputFormat, FileSplit }
/**
* A MapReduce InputFormat for HBase's HFile.
@aphyr
aphyr / gist:3746800
Created September 18, 2012 23:53
// lol also not a class
public class Debug {
public static void pr(Object o) {
// lol if you think java has a type system
if (o == null) {
pr("nil");
} else if (o instanceof String) {
System.out.print('"' + (String) o + '"');
@headius
headius / 1.txt
Created September 20, 2012 05:28
system ~/projects/jruby $ jruby -J-Xmx1024m -J-Xmx1024m -Isrc gc_stress.rb
ID Type Timestamp(sec) Before(kB) After(kB) Delta(kB) Heap(kB) GC Time(ms)
2 PS Scavenge 1.5624 48541 31837 16703 108288 7.1670000000
3 PS Scavenge 1.7100 66597 32293 34304 108672 6.8490000000
4 PS Scavenge 1.7796 66797 32429 34368 143104 6.0990000000
5 PS Scavenge 1.8868 101293 32445 68847 143104 6.2590000000
6 PS Scavenge 1.9809 101309 32509 68800 215936 5.9520000000
7 PS Scavenge 2.2277 170258 32634 137623 216256 6.6460000000
8 PS Scavenge 2.4213 170363