var http = require('http') var fs = require('fs') http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}) res.end("hello world") }).listen(3000);
This is just run with node server.js
ab -n 100000 -c100 http://localhost:3000/
Requests per second: 1339.35 [#/sec] (mean)
Requests per second: 1019.17 [#/sec] (mean)
Requests per second: 2899.45 [#/sec] (mean)
Requests per second: 4085.26 [#/sec] (mean)
for x in xrange(100000000): x + 1
time python test.py
real 0m13.531s user 0m13.487s sys 0m0.060s
real 0m10.337s user 0m10.302s sys 0m0.054s
real 0m9.177s user 0m9.120s sys 0m0.040s
real 0m8.909s user 0m8.897s sys 0m0.012s
The first iterations of this had node clustered with 2 forks, incrementing a remote redis counter, and ab being run from a remote machine. While the numbers were higher ( about 50% ) they were proportionally the same. This version just removes all variables.
I understand ab isn't exact but it's margin of error is tight enough to make it suitable for a ballpark. I've worked with Joyent and Amazon in the past. Infact I pulled 100 machines off AWS and put them on Joyent precisely because my own benchmarks - much more thorough than this - showed Joyent being 5 - 10 times faster.
When I ran the above I expected Joyent to completely demolish AWS.
I'm happy to be told I've done something terribly wrong. In fact I'm hoping for it.
@mhart. Nice catch. Copy Paste is exactly what I did.
wrk looks interesting. Though it looks like it runs under the same paradigm as ab and zeusbench. One process on one machine opening n connections.
What we found was that the issue wasn't the benchmarking tool. ab and zb are booth good tools for what they do. The issue is in the the SmartOS TCP stack, it doesn't like getting a ton of requests from a single ip address. In order to get a reliable benchmark we had to reduce the concurrency and distribute the benchmark clients across multiple machines. 3 machines running at a concurrency of 33. jmeter, funkload, and tsung should all be able to perform this reasonably well. One could in theory fire ab, zb, wrk on muliple machines simultaneously and aggregate the results but the other tools I list already have that built in.