Skip to content

Instantly share code, notes, and snippets.

Avatar

Sadayuki Furuhashi frsyuki

View GitHub Profile
View DeserBenchmark.java
import java.util.Random;
import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
View msgpack-v07-unpacker-design-doc.java
public interface XyzInput {
public Buffer getNextBuffer() throws IOException;
}
// non-reference counter
public abstract class Buffer {
private Unsafe unsafe;
private long pointer;
private Object base;
View DeserBenchmark.java
import java.util.Random;
import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
View Makefile
all: run
MSGPACK_06_JAR = msgpack-java-0.6.11.jar
$(MSGPACK_06_JAR):
curl "http://search.maven.org/remotecontent?filepath=org/msgpack/msgpack/0.6.11/msgpack-0.6.11.jar" -o $@
JAVASSIST_JAR = javassist-3.18.2-GA.jar
$(JAVASSIST_JAR):
curl "http://search.maven.org/remotecontent?filepath=org/javassist/javassist/3.18.2-GA/javassist-3.18.2-GA.jar" -o $@
View gist:e73c89db36c2069a7589
Add workaround for a deadlock issue of Class.getAnnotation() (JDK-7122142)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
index a80feb9..c92b4f6 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
@@ -1613,16 +1613,20 @@ public static boolean isDeterministic(GenericUDF genericUDF) {
// the deterministic annotation declares
return false;
}
View gist:12271f0fe2b02535202d
"main" prio=10 tid=0x00007f0fd800e800 nid=0x934a runnable [0x00007f0fdec8f000]
java.lang.Thread.State: RUNNABLE
at java.lang.Class.initAnnotationsIfNecessary(Class.java:3067)
- locked <0x000000009e5c7f70> (a java.lang.Class for org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge)
at java.lang.Class.getAnnotation(Class.java:3029)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1186)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isDeterministic(FunctionRegistry.java:1161)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:143)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.evaluate(ExprNodeGenericFuncEvaluator.java:153)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:64)
View java-socketread0-hang-bug.txt
Reading data from a socket hangs for at least 1 hour even if I set Socket.setSoTimeout to non-0 value. It happens very rarely (once a month under 100 or more connections in parallel with continuously 10MB/s or more data transfer).
Although it rarely happens, it leads significant impact to applications if it happens.
When it hanged, thread dump includes following stack trace:
----
"pool-3-thread-6" prio=10 tid=0x00007fd43006e800 nid=0xdf23 runnable [0x00007fd424e56000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
View ruby-struct-init.rb
require 'benchmark'
LOOP = 500000
NKEYS = 20
vals = (0..NKEYS).map {|i| "val#{i}" }
keys = (0..NKEYS).map {|i| "col#{i}" }
S = Struct.new(*keys.map {|k| k.to_sym })