Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Hello world of Sinatra-like libraries on Elixir, Ruby and Node.js

Lies, Damned Lies and Benchmarks

This is just an exercise to measure the performance between Sinatra-like libraries in:

The basic example uses only a single route at /hello_world and return Hello world. These benchmarks are extremely simple and should not be used as a decision factor for any of the tools presented (unless you are building hello world apps ;). I am just using them as performance guideline when working on Dynamo.

What happens if we add more routes?

This benchmark also measures what happens if you add 10, 100 and 1000 /hello_world/#{i} routes and try to match the last /hello_world/1000 route.

What happens if I am serving a file?

We also check the behavior of the app when it has to serve a file with 100kb of size.

What happens if I implement a Fibonacci server?

Just trolling bro.

Results

Results were obtained by running the examples below in a Macbook Pro 2011 2.3GHz dual-core Intel Core i5. No customization was done to the underlying VM/Engine/Webserver besides passing the "production" flag to all libraries, i.e. I am measuring the out-of-the-box experience. This means Elixir is running on both cores, while the other solutions aren't. This is one of the several benefits of using the Erlang VM.

req/s average for 8000 requests, concurrency 40 (ab -n 8000 -c 40) (more is better):

   Routes     | Dynamo/Elixir | SinatraS/Ruby 1.9.2 | Express/node.js 0.4.7
      1       |    9135.83    |      3833.54        |      4730.74
   10 + 1     |    9105.35    |      3561.69        |      4635.8
  100 + 1     |    9103.91    |      2472.51        |      4171.94
 1000 + 1     |    9043.72    |       598.47        |      1904.71
 100kb file   |    6087.99    |      2088.66        |      1734.17

Notes

Thanks to the Erlang VM, Dynamo performs very well in the basic hello world case. Indeed, Dynamo is in very early stage and the throughput will reduce by a couple dozens req/s as we add more basic features. Regardless, very good results.

It is also important to notice that increasing the number of routes increases the CPU usage which will block on single process, non threaded solutions like Node.js, affecting throughput. Also the performance of the algorithm used in the routes matching matters in the results above.

In the future, I plan to draw graphs on how those examples behave when increasing the concurrency. In early experiments, both Sinatra and Express throughput starts to decrease early on when increasing the concurrency. I also want to run benchmarks against those servers deployed on Heroku, but not sure if i need their permission first or anything. :)

defmodule MyApp do
use Dynamo::Routes
get "/hello_world" do
"Hello World"
end
end
Dynamo::Cowboy.run MyApp
require 'sinatra/base'
require 'sinatra/synchrony'
class HelloWorld < Sinatra::Base
register Sinatra::Synchrony
get '/hello_world' do
'Hello World'
end
end
var app = require('express').createServer();
app.get('/hello_world', function(req, res){
res.send('hello world');
});
app.listen(3000);
@josevalim
Copy link
Author

josevalim commented Mar 8, 2012

@burnhype I don't have this information anymore. I hope to re-run those benches in the future.

@fredwu
Copy link

fredwu commented Jun 30, 2013

@josevalim Hey Jose, any chance of a new benchmark for the latest versions? :)

@abhijitiitr
Copy link

abhijitiitr commented May 4, 2014

@josevalim I think NodeJS would perform better than Elixir after clustering.

@wess
Copy link

wess commented May 23, 2014

@abhijitiitr - Defeats the purpose of out of box if he has to go add a bunch of extensions, no?

@abhijitiitr
Copy link

abhijitiitr commented May 31, 2014

@wess it does...but does it really matter since clustering is commonly used in all nodejs apps and has become part of the standard code in nodejs

@erickhsiao
Copy link

erickhsiao commented Jul 1, 2014

@abhijitiitr if 1 instance of elixir can beat 1 instance of node.js in any scenario, why would cluster node.js perform better than elixir?

@jocull
Copy link

jocull commented Jan 15, 2018

Um, I actually do have a Node+Express Fibonacci service... :trollface:
https://github.com/jocull/node-fibonacci-service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment