$ ab -n 1000 -c 50 http://10.1.1.82:80/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: lighttpd/1.4.28 | |
Server Hostname: 10.1.1.82 | |
Server Port: 80 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.488 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 418418 bytes | |
HTML transferred: 257257 bytes | |
Requests per second: 2048.74 [#/sec] (mean) | |
Time per request: 24.405 [ms] (mean) | |
Time per request: 0.488 [ms] (mean, across all concurrent requests) | |
Transfer rate: 837.14 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 4 | |
Processing: 2 23 2.7 23 27 | |
Waiting: 2 23 2.7 23 27 | |
Total: 7 24 2.2 24 30 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 24 | |
66% 24 | |
75% 25 | |
80% 25 | |
90% 25 | |
95% 26 | |
98% 26 | |
99% 26 | |
100% 30 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:80/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: lighttpd/1.4.28 | |
Server Hostname: 10.1.1.82 | |
Server Port: 80 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.371 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 418418 bytes | |
HTML transferred: 257257 bytes | |
Requests per second: 2691.84 [#/sec] (mean) | |
Time per request: 18.575 [ms] (mean) | |
Time per request: 0.371 [ms] (mean, across all concurrent requests) | |
Transfer rate: 1099.92 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.9 1 5 | |
Processing: 4 17 1.9 18 20 | |
Waiting: 4 17 1.9 18 20 | |
Total: 10 18 1.3 18 23 | |
Percentage of the requests served within a certain time (ms) | |
50% 18 | |
66% 18 | |
75% 19 | |
80% 19 | |
90% 19 | |
95% 20 | |
98% 20 | |
99% 22 | |
100% 23 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:6789/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: Xavante | |
Server Hostname: 10.1.1.82 | |
Server Port: 6789 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.699 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 397000 bytes | |
HTML transferred: 257000 bytes | |
Requests per second: 1431.30 [#/sec] (mean) | |
Time per request: 34.933 [ms] (mean) | |
Time per request: 0.699 [ms] (mean, across all concurrent requests) | |
Transfer rate: 554.91 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 5 | |
Processing: 3 21 17.2 18 224 | |
Waiting: 3 20 17.3 18 224 | |
Total: 7 22 17.2 19 225 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 19 | |
66% 20 | |
75% 20 | |
80% 21 | |
90% 22 | |
95% 29 | |
98% 63 | |
99% 63 | |
100% 225 (longest request) |
$ ab -n 1000 -c 50 http://10.1.1.82:6789/hello/world | |
This is ApacheBench, Version 2.3 <$Revision: 655654 $> | |
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | |
Licensed to The Apache Software Foundation, http://www.apache.org/ | |
Benchmarking 10.1.1.82 (be patient) | |
Completed 100 requests | |
Completed 200 requests | |
Completed 300 requests | |
Completed 400 requests | |
Completed 500 requests | |
Completed 600 requests | |
Completed 700 requests | |
Completed 800 requests | |
Completed 900 requests | |
Completed 1000 requests | |
Finished 1000 requests | |
Server Software: Xavante | |
Server Hostname: 10.1.1.82 | |
Server Port: 6789 | |
Document Path: /hello/world | |
Document Length: 257 bytes | |
Concurrency Level: 50 | |
Time taken for tests: 0.383 seconds | |
Complete requests: 1000 | |
Failed requests: 0 | |
Write errors: 0 | |
Total transferred: 397000 bytes | |
HTML transferred: 257000 bytes | |
Requests per second: 2613.76 [#/sec] (mean) | |
Time per request: 19.130 [ms] (mean) | |
Time per request: 0.383 [ms] (mean, across all concurrent requests) | |
Transfer rate: 1013.34 [Kbytes/sec] received | |
Connection Times (ms) | |
min mean[+/-sd] median max | |
Connect: 0 1 0.8 0 4 | |
Processing: 2 13 11.3 12 216 | |
Waiting: 2 12 11.3 12 216 | |
Total: 6 14 11.5 13 219 | |
WARNING: The median and mean for the initial connection time are not within a normal deviation | |
These results are probably not that reliable. | |
Percentage of the requests served within a certain time (ms) | |
50% 13 | |
66% 13 | |
75% 14 | |
80% 14 | |
90% 14 | |
95% 17 | |
98% 23 | |
99% 24 | |
100% 219 (longest request) |
# This is the most basic lighttpd.conf (derived from the default one shipped with | |
# the Debian package of Lighttpd) that can be used to test Hops with FastCGI. | |
server.modules = ( | |
"mod_access", | |
"mod_fastcgi", | |
"mod_rewrite", | |
) | |
# Replace with the full path to the 'example' directory of Hops. | |
server.document-root = "/home/adaniele/development/hops/example" | |
server.errorlog = "/var/log/lighttpd/error.log" | |
server.pid-file = "/var/run/lighttpd.pid" | |
server.username = "www-data" | |
server.groupname = "www-data" | |
url.access-deny = ( "~", ".inc" ) | |
static-file.exclude-extensions = ( ".fcgi" ) | |
fastcgi.server = ( | |
".lua" => ( | |
"localhost" => ( | |
"socket" => "/tmp/lua-fastcgi_hops.socket", | |
# Replace with the actual path on your system where the | |
# wsapi.fcgi file (provided by wsapi-fcgi) is located. | |
# NOTE: to switch between Lua 5.1 and Luajit2 you can edit | |
# wsapi.fcgi so that it points to a different executable | |
# used by the 'exec' command. | |
"bin-path" => "/home/adaniele/.luarocks/bin/wsapi.fcgi", | |
# Specify how many Lua processes are used by FastCGI | |
# to elaborate requests. Setting max-procs to 1 means | |
# using just one Lua process. | |
"min-procs" => 1, | |
"max-procs" => 1, | |
) | |
) | |
) | |
url.rewrite-once = ( | |
"^/([^.]+)?$" => "/app.lua/$1", | |
) |
This comment has been minimized.
This comment has been minimized.
This is awesome! Any chance you could attach your lighttpd config for anyone else that might want to try? |
This comment has been minimized.
This comment has been minimized.
Sure, I reduced the whole lighttpd.conf so that only the basic stuff needed to get Hops up and running is there, but the relevant bits can be easily extracted to have it configured on an existing lighttpd install using virtual hosts. |
This comment has been minimized.
This comment has been minimized.
~2K RPS per thread is to be expected for wsapi+luajit2 on noop handlers. |
This comment has been minimized.
This comment has been minimized.
Good stuff. At least it looks like Hops isn't doing anything insanely stupid to ruin performance. :) |
This comment has been minimized.
This comment has been minimized.
A few more tests, this time using Mongrel2 (via mongrel2_wsapi) and producing numbers that should be compared with Lighttpd configured with The difference between the standard Lua interpreter and Luajit2 is much more enjoyable here, going from great to awesome numbers. Using Lua 5.1 - http://10.1.1.82:8080/hello/world
Using luajit2 - http://10.1.1.82:8080/hello/world
|
This comment has been minimized.
This comment has been minimized.
Good stuff! |
This comment has been minimized.
This comment has been minimized.
Hey @norman I updated my comment about the performance measurements using mongrel2 since I was inadvertently running my tests with only 1 backend process... so yes, 4501 req/sec with only one luajit2 process. With 2 luajit2 backend processes, the benchmark reaches 7689.87 req/sec. Awesome stuff. |
This comment has been minimized.
This comment has been minimized.
So, is it Mongrel2 so much faster than lighty? It maybe worth for me to switch my engine from nginx/spawn-fcgi to it... |
This comment has been minimized.
This comment has been minimized.
@nrk: can you point me somewhere where it is written how to setup mongrel with wsapi? |
This comment has been minimized.
This comment has been minimized.
@agladysh I reconfigured my virtual machine to use 4 CPUs and set max-procs = 3 in the lighttpd.conf: with this configuration I can get up to 6052.38 fetches/sec using Luajit2, which is a nice improvement but it's still far from the results that I can get with 2 processes of Luajit2 and Mongrel2. By the way, Mongrel2 with 3 Luajit2 processes easily tops 8100 req/sec, but I guess it's reaching its limit for that configuration. So, unless I got something horribly wrong, I'd say that Mongrel2 is indeed damn fast. I still haven't tried with nginx, but it's up next. As for how to get it up and running, I didn't follow any instruction but it's actually quite easy (at least without proper process management), you just need mongrel2-lua and mongrel2_wsapi and a few lines of Lua for additional configuration. I will post a step-by-step by tomorrow, just in case. |
This comment has been minimized.
This comment has been minimized.
@nrk: nginx should be about the same speed as lighty, but it worth benchmarking. step-by-step guide would be most useful (maybe you can post it somewhere on the web as a standalone document, not as comment?) :-) |
This comment has been minimized.
This comment has been minimized.
@agladysh yup, will use a separate gist for now so that users can also commit their corrections :-) |
This comment has been minimized.
This comment has been minimized.
Thanks for the benchmarks and info - great to see you guys playing around with my code. :) BTW I'm pretty sure Orbit, Mercury and other web frameworks would have similar performance though too, there's nothing too special about Hops in terms of what it does to be fast. Really the credit goes entirely to Lua/LuaJIT and WSAPI. |
This comment has been minimized.
This comment has been minimized.
Well yeah, it actually turned into a quick and nice demonstration about how the Mongrel2+Luajit2 pair can be really fast allowing for an high number of RPS with a very low memory footprint and CPU usage. Sticking with Hops as a reference microframework for these tests is now a matter of consistency, maybe I'll do the same with other frameworks such as Orbit just out of curiosity :-) Anyway, nice work on Hops! |
This comment has been minimized.
This comment has been minimized.
Hi Danielle, Some questions: I hope all of this stuff results in some sort of real framework, something I can just be lazy and use, and all this work, you should write it up in a blog, cause Lua needs the press, its the absolute shit and people just dont know about it. nice work ... keep going
|
This comment has been minimized.
This comment has been minimized.
Thanks! Glad you're liking Hops. I'm currently working on better template handling (caching, responding with different templates based on content type headers, the request and extension like Rails), but other than that the focus is more on ease of use than high performance. A certain good level of performance is guaranteed just by using Lua and WSAPI. If you're looking for a non-blocking web development API for Lua then you might want to check out LuaNode. |
This comment has been minimized.
This comment has been minimized.
@JakSprats: hey Jak it's been a while, and it's good to know that you are enjoying Lua more and more. In reply to your questions:
@norman: Jak and performances go hand in hand ;-) |
This comment has been minimized.
This comment has been minimized.
@nrk hey Danielle, 1 & 2 were just me wanting to see higher numbers, I can do the math in my head already, and this platform would smoke on a 6 core @3.2Ghz :)
|
This comment has been minimized.
This comment has been minimized.
@nrk Any news about that Mongrel configs? :-) |
This comment has been minimized.
This comment has been minimized.
@JakSprats: hehe, too bad I didn't have the time to test thoroughly that beast when it still was without the hypervisor installed (it's a dual Xeon X5660, 2 x 6 cores @ 2.8 GHz). PS: just got your mail, will reply tomorrow. @agladysh: as soon as I can get that VM up and running again (I don't have access to our VPN right now) and review my hackish installation, I've been busy today but I guess I can get it done over the weekend. |
This comment has been minimized.
This comment has been minimized.
@nrk: OK, thanks, no hurry here :-) |
This comment has been minimized.
Some benchmarks for Hops using various configurations. Hops is executed on a virtualized (VMware ESX) Debian Squeeze with only two CPUs assigned and ab is launched on a completely different host machine connected with a gigabit link (same network). A couple of additional notes:
Xavante: the reload parameter is set to false to prevent the app from reloading on each request.
Lighttpd: the fastcgi.server backend for Lua is configured with the max-procs directive set to 1 (only one Lua process is used) so that we can obtain results that can be compared directly with the benchmarks that use Xavante. Just for reference, Lighttpd with a plain HTML file can reach an average of 8026.71 req/sec.
As one would expect, Lighttpd can yield much better results just by setting max-procs to 2:
Lua 5.1: 3590.84 #/sec
Luajit2: 3490.41 #/sec
Strangely enough Luajit2 isn't faster than Lua 5.1 here. I guess it's just because the VM is configured with only 2 CPUs, even if the actual load of the server does not even reach 80% during these tests.