Skip to content

Instantly share code, notes, and snippets.

@tsuna
Created May 30, 2013 21:31
Show Gist options
  • Save tsuna/5681407 to your computer and use it in GitHub Desktop.
Save tsuna/5681407 to your computer and use it in GitHub Desktop.
Benchmark to measure the performance improvement of lock-less UID assignment in OpenTSDB
import org.slf4j.Logger;
import net.opentsdb.uid.UniqueId;
import org.hbase.async.HBaseClient;
final class uidbench {
private static void log(final String msg) {
System.out.println("[" + Thread.currentThread().getName() + "] " + msg);
}
public static void main(final String[] args) throws Exception {
log("Starting.");
final HBaseClient client = new HBaseClient("localhost");
final int numthreads = Integer.parseInt(args[0]);
final String basename = args[1];
final int iterations = 10000;
final class Test extends Thread {
final int id;
Test(final int id) {
super("Test" + id);
this.id = id;
}
public void run() {
log("Test thread starting...");
final UniqueId uid = new UniqueId(client, "tsdb-uid".getBytes(),
"test", 3);
final long start = System.nanoTime();
for (int i = 0; i < iterations; i++) {
uid.getOrCreateId(basename + i + "-" + id);
}
final long finish = System.nanoTime();
final long duration = (finish - start) / 1000000;
log("Created " + iterations + " UIDs in " + duration + "ms => "
+ (iterations * 1000 / duration) + " UIDs allocated per sec");
}
}
try {
final Test[] tests = new Test[numthreads];
for (int i = 0; i < numthreads; i++) {
tests[i] = new Test(i);
}
log("Kicking off test threads");
for (int i = 0; i < numthreads; i++) {
tests[i].start();
}
for (int i = 0; i < numthreads; i++) {
tests[i].join();
}
} finally {
log("Shutting down.");
client.shutdown().join();
}
log("Success!");
}
}
@tsuna
Copy link
Author

tsuna commented May 30, 2013

Results, on a MacBook Pro with 8 hardware threads, a recent build of the HBase 0.94 branch. The baseline is OpenTSDB v1.1.0, and the lock-less implementation is based on an approach I suggested on the mailing list. This table shows the sum of the number of UID allocated per second for all threads.

# ThreadsBaselineLock-less
17601170
28902534
39363402
49443912
89205552

This shows that the new lock-less code is almost linearly scalable. There is still obviously some level of contention within HBase, to perform atomic increments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment