Hey all,
In preparation for Clojure/West, I'm formally releasing the latest Aleph and the libraries that surround it. Aleph 0.4.0 has been running in production at Factual for half a year now, and across a variety of services is handling at peak 600k HTTP requests/sec (spread across 15-20 machines).
Since the landscape of Clojure HTTP servers is pretty crowded these days, it's worth taking some time to explain how Aleph differs. To be clear, most Clojure deployments likely use Jetty, and should continue to do so. However, Aleph has some unique properties:
- It uses the Netty library, which is a high-performance and very battle-tested network layer for the JVM
- It is the only HTTP server that has ubiquitous asynchronous streams wherever data can be received or sent (all other libraries can only represent streaming requests using InputStreams, or like http-kit don't support streaming HTTP requests at all)
- It is the only server that has a WebSocket implementation with any support for per-connection backpressure. I won't make this post even longer by going into why this is important, but this will be a central theme of my talk at Clojure/West next week if you're interested in hearing more.
- It uses consistent abstractions to represent network connections over a variety of protocols, which makes it straightforward to use the same application logic for all of them.
Again, none of these points mean you should immediately drop whatever you're using and move over to Aleph instead. However, I do feel it represents the only (current) good option for using core.async or a similar stream abstraction to represent network data, which is an idea a number of people seem to be playing with lately. Some examples of this can be found at http://ideolalia.com/aleph/literate.html.
A full list of the libraries:
aleph - https://github.com/ztellman/aleph - uses the excellent Netty library to expose HTTP, TCP, and UDP using a consistent asynchronous stream representation.
manifold - https://github.com/ztellman/manifold - an unopinionated stream representation designed to cleanly interoperate with other stream representations (Clojure's seqs, core.async channels, Java's BlockingQueues, and others). This is the base stream representation for all network sources and sinks in Aleph.
dirigiste - https://github.com/ztellman/dirigiste - a pure-Java library that provides instrumented, dynamically sized thread and object pools. This is used for thread pools in Aleph's HTTP server, and for connection pools in Aleph's HTTP client.
byte-streams - https://github.com/ztellman/byte-streams - a means of translating any byte representation into another. Want to turn a core.async channel that emits byte-arrays into an InputStream, or maybe the other way around? Look no further. The library's conversion mechanism is extensible, which is used in Aleph to make Netty's custom byte representations interoperable with more familiar representations.
byte-transforms - https://github.com/ztellman/byte-transforms - a curated collection of byte compression, hashing, and encoding mechanisms, which can work on anything byte-streams can convert.
While all these libraries are used in concert to create Aleph, I've been very careful to make sure any of them can be used by themselves. If anyone has questions about them, the best place to get my attention is the Aleph mailing list: https://groups.google.com/forum/#!forum/aleph-lib.
I will be mentioning some of these libraries at my upcoming Clojure/West talk (http://clojurewest.org/speakers#ztellman), but I've also set aside an Unsession for specifically discussing these libraries: https://github.com/clojurewest/clojurewest2015/wiki/Unsessions. If you're interested, please add your name to the list.
Zach