Using Twitter's ZK client

twitter-zk.md

Using Twitter's ZK client

Introduction

I'm a Scala noob.

I am deploying some Spark jobs which need to talk to various services in our production environment. These services are managed by a ZooKeeper installation that registers their host and ports for service discovery. Hence, our Scala code needs to interact with ZooKeeper to discover the host and ports to use when making HTTPS calls to these internal services.

There is some ScalaDoc for util-zk that describes the classes, objects and function signatures but those don't help me very much since I'm a newb and none of it really makes a ton of sense right now.

So I am writing this document with my discoveries about how the util-zk library behaves empirically.

If you feel like explaining what I got wrong and so forth, please leave a comment. I'd appreciate a better understanding of how it all works since our interaction with ZooKeeper on the Erlang side of the house is reasonably straight forward.

Connections

I made a connection like this

import com.twitter.util.JavaTimer
import com.twitter.conversion.time._
import com.twitter.zk._

implicit val timer = new JavaTimer(true)
val zk = ZkClient("localhost:2181", Some(5.seconds), 30.seconds)

Specifying a Znode path

ZooKeeper manages data in a tree format where Znodes are addressed using a path that resembles a filesystem path: /foo/bar/baz

val zNode = zk("/foo/bar/baz")

Operations on a Znode

Once you have a Znode object, you can execute various operations on it. Some of them are:

• exists - does this node exist?
• getData - get the data contents of this node
• getChildren - get the (possibly empty) list of children of this node

Actually getting results

There are three functions for each ZOp:

• apply - returns a Future satisfied when the operation is performed
• watch - creates a ZooKeeper watch on the specified node. Future contains the Watch
• monitor - repeatedly watch this node and return watch events as an "Offer" stream.

Dealing with Futures

There's some reasonable docs about Futures here, so it looks like the way to go might be something like

zkClient("/a/path").getData onSuccess{ res: => println("value: " + res.bytes) }

Await

If you can't be arsed to figure out all of the Futures fancy pants, try using Await.result. If you use Await, your fancy asynchronous request will suddenly become a blocking, synchronous request. (And that's just fine with me, tbh.) It looks something like this:

import com.twitter.util.Await

// initialized zkClient somewhere
val f = zkClient("/a/path").getData()
val res = Await.result(f)
val resStr = new String(res.bytes)