I've been hacking away recently at a JVM framework for doing asynchronous, non-blocking applications using a variation of the venerable Reactor pattern. The core of the framework is currently in Java. I started with Scala then went with Java and am now considering Scala again for the core. What can I say: I'm a grass-is-greener waffler! :) But it understands how to invoke Groovy Closures, Scala anonymous functions, and Clojure functions, so you can use the framework directly without needing wrappers.
I've been continually micro-benchmarking this framework because I feel that the JVM is a better foundation on which to build highly-concurrent, highly-scalable, C100K applications than V8 or Ruby. The problem has been, so far, no good tools exist for JVM developers to leverage the excellent performance and manageability of the JVM. This yet-to-be-publicly-released framework is an effort to give Java, Groovy, Scala, [X JVM language] developers access to an easy-to-use programming model that removes the necessity to use synchronization and worry about concurrency issues, while making it easy to respond to events in multiple threads if it's more efficient for your application to do so (unlike the strict single-threadedness of Node.js, this framework gives you a choice of single-threaded efficiency or multi-threaded parallelism).
The benchmark below is of this reactor-based framework that uses the old-school Java NIO FileChannel.transferTo (sendfile) method to stream data from the filesystem to the client. The Node.js application uses streaming to pipe a file directly to the client.
The Groovy code looks like this:
def server = new HttpServer(3000)
.on("/lib/{resource}**", {HttpMessage request ->
def file = request.pathParam("resource")
def path = Paths.get(resources, file)
if (Files.exists(path)) {
request.respond(StandardHttpResponses.ok(contentType, path))
} else {
request.respond(StandardHttpResponses.notFound(request.uri().path))
}
})
.start()
A similar application could be built with pure Java using annotations. The POJO delegate would look something like:
@On("/lib/{resource}**") @Get
public void static(HttpMessage request) {
String file = request.pathParam("resource");
Path path = Paths.get(resources, file);
if (Files.exists(path)) {
request.respond(StandardHttpResponses.ok(contentType, path));
} else {
request.respond(StandardHttpResponses.notFound(request.uri().path));
}
}
The Node.js code looks like this:
http.createServer(
function (req, res) {
var pth = path.join("static", req.url)
var rs = fs.createReadStream(pth);
rs.on("error", function() {
res.writeHead(404);
res.end();
});
rs.once("fd", function() {
res.writeHead(200, {'Content-Type': 'application/octet-stream'});
});
rs.pipe(res);
}
).listen(3001, "127.0.0.1");
When I ran Apache Bench against these servers (100 concurrent users downloading a 1MB file), the JVM framework handily out-performed the Node.js version. By handily, I mean it was more than twice as fast and supported twice the throughput:
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 3 1.0 3 4
Processing: 246 252 2.3 252 254
Waiting: 21 32 6.3 34 41
Total: 248 254 2.4 255 257
Percentage of the requests served within a certain time (ms)
50% 255
66% 256
75% 256
80% 256
90% 257
95% 257
98% 257
99% 257
100% 257 (longest request)
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 3 0.9 3 4
Processing: 64 118 17.1 123 135
Waiting: 4 16 9.2 13 43
Total: 67 121 17.2 126 138
Percentage of the requests served within a certain time (ms)
50% 126
66% 132
75% 135
80% 135
90% 136
95% 138
98% 138
99% 138
100% 138 (longest request)
From the gallery: "No fair! You're using multiple threads!"
Since setting up 4 Node.js processes and configuring the load balancing was more than I wanted to take on just for a simple microbenchmark, I dropped the concurrent users to 1 and re-ran the tests:
Requests per second: 270.12 [#/sec] (mean)
Time per request: 3.702 [ms] (mean)
Time per request: 3.702 [ms] (mean, across all concurrent requests)
Transfer rate: 276612.41 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 4
Processing: 3 4 2.3 3 32
Waiting: 0 1 1.0 0 25
Total: 3 4 2.3 3 32
WARNING: The median and mean for the waiting time are not within a normal deviation
These results are probably not that reliable.
Percentage of the requests served within a certain time (ms)
50% 3
66% 3
75% 4
80% 4
90% 4
95% 4
98% 6
99% 15
100% 32 (longest request)
Requests per second: 228.89 [#/sec] (mean)
Time per request: 4.369 [ms] (mean)
Time per request: 4.369 [ms] (mean, across all concurrent requests)
Transfer rate: 234428.13 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 2 4 0.5 4 8
Waiting: 0 0 0.1 0 3
Total: 2 4 0.5 4 8
Percentage of the requests served within a certain time (ms)
50% 4
66% 4
75% 5
80% 5
90% 5
95% 5
98% 5
99% 6
100% 8 (longest request)
With only a single thread at a time, the JVM competed slightly better in response time (drastically smaller standard deviation with identical mean times) while Node.js supported about 6% greater bandwidth.
The point of this is not to bash Node.js. It's a great platform for some applications that can benefit from the things it does well. It's also fine for dogmatic JVM-haters to dismiss any of these tests as flawed or irrelevant. They'll never be open-minded enough to take an honest look at what the JVM can do for this new class of applications that need C100K capabilities but would benefit from the decades of engineering that's gone into the JVM and the plethora of management tools and operational experience with the platform.
Combining the JVM with a better framework for writing non-blocking, evented applications is the goal. These tests just confirm for me that that goal is achievable and that there is benefit to be had from such a framework. It also tells me that the weaknesses of the JVM for C100K applications have more to do with the programming model than they do with the JVM itself. If I were to tune the JVM running these tests, rather than use the default settings, I could likely get even better numbers than these. The JVM has a boatload of knobs to turn that I simply didn't take the time to tweak.