Skip to content

Instantly share code, notes, and snippets.

@tsuna
Created February 29, 2012 07:45
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save tsuna/1938931 to your computer and use it in GitHub Desktop.
Save tsuna/1938931 to your computer and use it in GitHub Desktop.
Get average size of a KeyValue from HBase using asynchbase in Scala with the Finagle compatibility layer
import java.util.ArrayList
import scala.collection.JavaConversions._
import com.stumbleupon.async.{Callback, Deferred}
import org.hbase.async.{HBaseClient, KeyValue, Scanner}
import com.twitter.util.{Future, Promise, Return, Throw}
/** Converts a Deferred into a Future. */
implicit def futureFromDeferred[A](d: Deferred[A]): Future[A] = {
val promise = new Promise[A]
d.addBoth(new Callback[Unit, A] {
def call(arg: A) = promise() = arg match {
case e: Throwable => Throw(e)
case _ => Return(arg)
}
})
promise
}
val client = new HBaseClient("localhost")
val scanner = client.newScanner("tsdb")
var numkvs = 0
var totalsize = 0
def cb(rows: ArrayList[ArrayList[KeyValue]]): Future[(Int, Int)] = {
if (rows != null) {
numkvs += rows.map(_.size).sum
totalsize += rows.flatten.map(_.value.length).sum
scanner.nextRows andThen cb
} else Future((numkvs, totalsize))
}
println("I am starting in thread " + Thread.currentThread.getName)
scanner.nextRows andThen cb map { result =>
println("I have results in thread " + Thread.currentThread.getName)
println("Got " + result._1 + " KVs totaling " + result._2 + " bytes")
client.shutdown.get() // Don't forget this or your JVM won't exit because of unterminated threads
}
println("I am done in thread " + Thread.currentThread.getName)
@tsuna
Copy link
Author

tsuna commented Feb 29, 2012

Expected output:

I am starting in thread run-main
[some asynchbase logging]
I am done in thread run-main
[some more asynchbase logging]
I have results in thread New I/O client worker #1-1
Got 39054 KVs totaling 1899340 bytes

This is normal because the I have results message is coming later asynchronously, in a different thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment