Skip to content

Instantly share code, notes, and snippets.

View stephenh's full-sized avatar

Stephen Haberman stephenh

267 followers · 37 following
View GitHub Profile
@stephenh
stephenh / gist:a93a715d0d15eaced9eb2cb910daa51d
Created November 3, 2018 01:12
keybase.md
### Keybase proof
I hereby claim:
* I am stephenh on github.
* I am stephenhaberman (https://keybase.io/stephenhaberman) on keybase.
* I have a public key ASBgKUp5p77PdVZdtSABrgJit4xQ9iahkzipyPmQwPAPuQo
To claim this, I am signing this object:
@stephenh
stephenh / gist:87f940a1f1356f5ab9f4a8579c567443
Created November 2, 2018 14:29
role1 = { name, perms: [p1, p2, p3] }
role2 = { name, perms: [p3, p4] }
roles = [role1, role2, ...]
# put lists in sorted sort
# O(N_roles * N_permissions log N_permissions)
roles.forEach { sortInPlace(_.perms) }
# compare each role to the other, N^2 = 16m comparison
@stephenh
stephenh / boltsort.hs
Created May 26, 2018 06:05
newtype Nut = Nut Int deriving Show
newtype Bolt = Bolt Int deriving Show
compareNutBolt :: Nut -> Bolt -> Int
compareNutBolt (Nut i) (Bolt j) = i - j
-- given a compare function, breaks [a] into smaller/equal/bigger
partition :: (a -> Int) -> [a] -> ([a], [a], [a])
partition f [] = ([], [], [])
@stephenh
stephenh / gist:cf928c4adb371077421a
Created March 18, 2016 17:50
grpc output
[shaberma@shaberma-ld1 ~ master]$ java -Xmx1G -cp mirror-all.jar mirror.MirrorServer ~/linkedin 10000
2016-03-18 09:57:42 INFO mirror.MirrorServer initialSync Starting new session
2016-03-18 09:57:48 INFO mirror.MirrorServer initialSync Server has 49517 paths
2016-03-18 09:57:48 INFO mirror.MirrorServer initialSync Client has 7050 paths
2016-03-18 09:57:48 INFO mirror.MirrorSession seedQueueForInitialSync Queueing 43891 paths to send to the remote host
...
2016-03-18 10:00:11 SEVERE mirror.MirrorServer$1 onError Error from incoming client stream
io.grpc.StatusException: CANCELLED [43/17808]
at io.grpc.Status.asException(Status.java:509)
at io.grpc.stub.ServerCalls$2$1.onCancel(ServerCalls.java:227)
@stephenh
stephenh / Test.java
Created November 3, 2013 23:53
package temp;
import java.math.BigDecimal;
import java.math.BigInteger;
public class Test {
public static void main(String[] args) {
System.out.println(toSeventeenDigits(1000));
System.out.println(toSeventeenDigits(100));
System.out.println(toSeventeenDigits(10));
@stephenh
stephenh / gist:6342916
Created August 26, 2013 15:45
13/08/24 14:38:52 INFO cluster.TaskSetManager: Loss was due to java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66)
at java.lang.Runtime.addShutdownHook(Runtime.java:211)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1441)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:256)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:77)
at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:51)
at spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:79)
at spark.rdd.HadoopRDD.compute(HadoopRDD.scala:70)
@stephenh
stephenh / hadoop1.0.3.java
Created April 18, 2013 18:31
retries
public synchronized int read() throws IOException {
int result = in.read();
if (result != -1) {
pos++;
}
return result;
}
@stephenh
stephenh / stderr
Created March 26, 2013 21:20
registrar not found
This file has been truncated, but you can view the full file.
SLF4J: This version of SLF4J requires log4j version 1.2.12 or later. See also http://www.slf4j.org/codes.html#log4j_version
13/03/26 20:19:54 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/03/26 20:19:54 INFO actor.ActorSystemImpl: RemoteServerStarted@akka://spark@10.224.6.227:52400
13/03/26 20:19:54 INFO spark.SparkEnv: Registering BlockManagerMaster
13/03/26 20:19:54 INFO storage.MemoryStore: MemoryStore started with capacity 323.9 MB.
13/03/26 20:19:54 INFO storage.DiskStore: Created local directory at /mnt/spark/local/spark-local-20130326201954-696e
13/03/26 20:19:55 INFO network.ConnectionManager: Bound socket to port 41101 with id = ConnectionManagerId(ip-10-224-6-227.ec2.internal,41101)
13/03/26 20:19:55 INFO storage.BlockManagerMaster: Trying to register BlockManager
13/03/26 20:19:55 INFO storage.BlockManagerMaster: Registered BlockManager
13/03/26 20:19:55 INFO server.Server: jetty-7.5.3.v20111011
@stephenh
stephenh / gist:5156672
Created March 13, 2013 21:50
object foo {
var comparisons = 0 //> comparisons : Int = 0
class Foo {
override def equals(other: Any) = { comparisons += 1; super.equals(other) }
}
class Bar extends Foo
class Baz extends Foo
val s1: Set[Foo] = Seq.fill(10)(new Bar).toSet //> s1 : Set[foo.Foo] = Set(foo$$anonfun$main$1$Bar$1@1529d183, foo$$anonfun$ma
//| in$1$Bar$1@7d1f0c98, foo$$anonfun$main$1$Bar$1@1740f55, foo$$anonfun$main$1$
//| Bar$1@c8a3c71, foo$$anonfun$main$1$Bar$1@6a9a9631, foo$$anonfun$main$1$Bar$1
@stephenh
stephenh / gist:5108259
Created March 7, 2013 14:04
kryo snapshot stack trace
[info] BlockManagerSuite:
[info] - master + 2 managers interaction *** FAILED ***
[info] java.lang.IllegalArgumentException: Unable to create serializer "com.esotericsoftware.kryo.serializers.FieldSerializer" for class: scala.Tuple2$mcII$sp
[info] at com.esotericsoftware.kryo.Kryo.newSerializer(Kryo.java:347)
[info] at com.esotericsoftware.kryo.Kryo.newDefaultSerializer(Kryo.java:326)
[info] at com.esotericsoftware.kryo.Kryo.getDefaultSerializer(Kryo.java:319)
[info] at de.javakaffee.kryoserializers.KryoReflectionFactorySupport.getDefaultSerializer(KryoReflectionFactorySupport.java:45)
[info] at com.esotericsoftware.kryo.Kryo.register(Kryo.java:363)
[info] at spark.KryoSerializer$$anonfun$createKryo$1.apply(KryoSerializer.scala:140)
[info] at spark.KryoSerializer$$anonfun$createKryo$1.apply(KryoSerializer.scala:139)