-
-
Save nrk/911358 to your computer and use it in GitHub Desktop.
$ ab -n 1000 -c 50 http://10.1.1.82:80/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: lighttpd/1.4.28 | |
Server Hostname: 10.1.1.82 | |
Server Port: 80 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.488 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 418418 bytes | |
HTML transferred: 257257 bytes | |
Requests per second: 2048.74 [#/sec] (mean) | |
Time per request: 24.405 [ms] (mean) | |
Time per request: 0.488 [ms] (mean, across all concurrent requests) | |
Transfer rate: 837.14 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 4 | |
Processing: 2 23 2.7 23 27 | |
Waiting: 2 23 2.7 23 27 | |
Total: 7 24 2.2 24 30 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 24 | |
66% 24 | |
75% 25 | |
80% 25 | |
90% 25 | |
95% 26 | |
98% 26 | |
99% 26 | |
100% 30 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:80/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: lighttpd/1.4.28 | |
Server Hostname: 10.1.1.82 | |
Server Port: 80 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.371 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 418418 bytes | |
HTML transferred: 257257 bytes | |
Requests per second: 2691.84 [#/sec] (mean) | |
Time per request: 18.575 [ms] (mean) | |
Time per request: 0.371 [ms] (mean, across all concurrent requests) | |
Transfer rate: 1099.92 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.9 1 5 | |
Processing: 4 17 1.9 18 20 | |
Waiting: 4 17 1.9 18 20 | |
Total: 10 18 1.3 18 23 | |
Percentage of the requests served within a certain time (ms) | |
50% 18 | |
66% 18 | |
75% 19 | |
80% 19 | |
90% 19 | |
95% 20 | |
98% 20 | |
99% 22 | |
100% 23 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:6789/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: Xavante | |
Server Hostname: 10.1.1.82 | |
Server Port: 6789 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.699 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 397000 bytes | |
HTML transferred: 257000 bytes | |
Requests per second: 1431.30 [#/sec] (mean) | |
Time per request: 34.933 [ms] (mean) | |
Time per request: 0.699 [ms] (mean, across all concurrent requests) | |
Transfer rate: 554.91 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 5 | |
Processing: 3 21 17.2 18 224 | |
Waiting: 3 20 17.3 18 224 | |
Total: 7 22 17.2 19 225 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 19 | |
66% 20 | |
75% 20 | |
80% 21 | |
90% 22 | |
95% 29 | |
98% 63 | |
99% 63 | |
100% 225 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:6789/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: Xavante | |
Server Hostname: 10.1.1.82 | |
Server Port: 6789 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.383 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 397000 bytes | |
HTML transferred: 257000 bytes | |
Requests per second: 2613.76 [#/sec] (mean) | |
Time per request: 19.130 [ms] (mean) | |
Time per request: 0.383 [ms] (mean, across all concurrent requests) | |
Transfer rate: 1013.34 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 4 | |
Processing: 2 13 11.3 12 216 | |
Waiting: 2 12 11.3 12 216 | |
Total: 6 14 11.5 13 219 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 13 | |
66% 13 | |
75% 14 | |
80% 14 | |
90% 14 | |
95% 17 | |
98% 23 | |
99% 24 | |
100% 219 (longest request) |
# This is the most basic lighttpd.conf (derived from the default one shipped with | |
# the Debian package of Lighttpd) that can be used to test Hops with FastCGI. | |
server.modules = ( | |
"mod_access", | |
"mod_fastcgi", | |
"mod_rewrite", | |
) | |
# Replace with the full path to the 'example' directory of Hops. | |
server.document-root = "/home/adaniele/development/hops/example" | |
server.errorlog = "/var/log/lighttpd/error.log" | |
server.pid-file = "/var/run/lighttpd.pid" | |
server.username = "www-data" | |
server.groupname = "www-data" | |
url.access-deny = ( "~", ".inc" ) | |
static-file.exclude-extensions = ( ".fcgi" ) | |
fastcgi.server = ( | |
".lua" => ( | |
"localhost" => ( | |
"socket" => "/tmp/lua-fastcgi_hops.socket", | |
# Replace with the actual path on your system where the | |
# wsapi.fcgi file (provided by wsapi-fcgi) is located. | |
# NOTE: to switch between Lua 5.1 and Luajit2 you can edit | |
# wsapi.fcgi so that it points to a different executable | |
# used by the 'exec' command. | |
"bin-path" => "/home/adaniele/.luarocks/bin/wsapi.fcgi", | |
# Specify how many Lua processes are used by FastCGI | |
# to elaborate requests. Setting max-procs to 1 means | |
# using just one Lua process. | |
"min-procs" => 1, | |
"max-procs" => 1, | |
) | |
) | |
) | |
url.rewrite-once = ( | |
"^/([^.]+)?$" => "/app.lua/$1", | |
) |
So, is it Mongrel2 so much faster than lighty? It maybe worth for me to switch my engine from nginx/spawn-fcgi to it...
@nrk: can you point me somewhere where it is written how to setup mongrel with wsapi?
@agladysh I reconfigured my virtual machine to use 4 CPUs and set max-procs = 3 in the lighttpd.conf: with this configuration I can get up to 6052.38 fetches/sec using Luajit2, which is a nice improvement but it's still far from the results that I can get with 2 processes of Luajit2 and Mongrel2. By the way, Mongrel2 with 3 Luajit2 processes easily tops 8100 req/sec, but I guess it's reaching its limit for that configuration.
So, unless I got something horribly wrong, I'd say that Mongrel2 is indeed damn fast. I still haven't tried with nginx, but it's up next.
As for how to get it up and running, I didn't follow any instruction but it's actually quite easy (at least without proper process management), you just need mongrel2-lua and mongrel2_wsapi and a few lines of Lua for additional configuration. I will post a step-by-step by tomorrow, just in case.
@nrk: nginx should be about the same speed as lighty, but it worth benchmarking.
step-by-step guide would be most useful (maybe you can post it somewhere on the web as a standalone document, not as comment?) :-)
@agladysh yup, will use a separate gist for now so that users can also commit their corrections :-)
Thanks for the benchmarks and info - great to see you guys playing around with my code. :) BTW I'm pretty sure Orbit, Mercury and other web frameworks would have similar performance though too, there's nothing too special about Hops in terms of what it does to be fast. Really the credit goes entirely to Lua/LuaJIT and WSAPI.
Well yeah, it actually turned into a quick and nice demonstration about how the Mongrel2+Luajit2 pair can be really fast allowing for an high number of RPS with a very low memory footprint and CPU usage. Sticking with Hops as a reference microframework for these tests is now a matter of consistency, maybe I'll do the same with other frameworks such as Orbit just out of curiosity :-) Anyway, nice work on Hops!
Hi Danielle,
this all kicks ass ... where is it heading to, are you gonna make something other people can easily re-use .. I ask because I am becoming more and more of a Lua-junky.
Some questions:
1.) why not run this on bare metal?
2.) use 4 cores, or 6 cores - get some BIG benchmark numbers (like 30-50K)
3.) please please plug this all into a non-blocking redis backend framework ... please please :)
I hope all of this stuff results in some sort of real framework, something I can just be lazy and use, and all this work, you should write it up in a blog, cause Lua needs the press, its the absolute shit and people just dont know about it.
nice work ... keep going
- jak
Thanks! Glad you're liking Hops. I'm currently working on better template handling (caching, responding with different templates based on content type headers, the request and extension like Rails), but other than that the focus is more on ease of use than high performance. A certain good level of performance is guaranteed just by using Lua and WSAPI.
If you're looking for a non-blocking web development API for Lua then you might want to check out LuaNode.
@JakSprats: hey Jak it's been a while, and it's good to know that you are enjoying Lua more and more. In reply to your questions:
- the reason is simple: we, at work, rely heavily on our in-house virtualization infrastructure. We still have a few bare-metal installations, but that hardware is relatively old.
- I can still configure my virtual machine to use 8 cores :-) That ESX host is brand new and still unused aside from some test stuff, so there's actually nothing running while I do the tests.
- I don't know how and if a WSAPI-based application can behave with non-blocking stuff, but aside from that I'd love to make redis-lua completely async with the next major version. As @norman pointed out you should definitely take a look at LuaNode.
@norman: Jak and performances go hand in hand ;-)
@nrk hey Danielle, 1 & 2 were just me wanting to see higher numbers, I can do the math in my head already, and this platform would smoke on a 6 core @3.2Ghz :)
I asked Ignacio of LuaNode about 3 months ago about redis integration and he said: no time right now.
So the non-blocking Lua-webserver w/ nonblocking redis backend remains to-be-done (redis2-nginx-module has done it, but I remember you wanted it done differently than they did it).
Anyways, I am polluting this gist, I have an idea about serving Lua real quick that I will offline email you about.
- Jak
@nrk Any news about that Mongrel configs? :-)
@JakSprats: hehe, too bad I didn't have the time to test thoroughly that beast when it still was without the hypervisor installed (it's a dual Xeon X5660, 2 x 6 cores @ 2.8 GHz). PS: just got your mail, will reply tomorrow.
@agladysh: as soon as I can get that VM up and running again (I don't have access to our VPN right now) and review my hackish installation, I've been busy today but I guess I can get it done over the weekend.
@nrk: OK, thanks, no hurry here :-)
Hey @norman I updated my comment about the performance measurements using mongrel2 since I was inadvertently running my tests with only 1 backend process... so yes, 4501 req/sec with only one luajit2 process. With 2 luajit2 backend processes, the benchmark reaches 7689.87 req/sec. Awesome stuff.