heronmedeiros/0_README.markdown

## 0_README.markdown

      
    Raw
  

              0_README.markdown
            
          
    Lies, Damned Lies and Benchmarks

This is just an exercise to measure the performance between Sinatra-like libraries in:


Elixir v0.3.0 (running on top of the Erlang VM R14B02) with Frankie;


Ruby 1.9.2 with Sinatra Synchrony + Thin;


node.js 0.4.7 with express


All the examples use only a single route at /hello_world and return Hello world. These benchmarks are extremely simple and should not be used as a decision factor for any of the tools presented (unless you are building hello world apps ;). I am just using them as performance guideline when working on Frankie.
What happens if we add more routes?

This benchmark also measures what happens if you add 10, 100 and 1000 /abc routes before the /hello_world route. It is expected that performance will be affected in all cases as all libraries try to match all /abc routes before finally matching /hello_world.
Results

Results were obtained by running the examples below in a Macbook Pro 2011 2.3GHz dual-core Intel Core i5. No customization was done to the underlying VM/Engine/Webserver besides passing the "production" flag to all libraries, i.e. I am measuring the out-of-the-box experience. This means Elixir is running on both cores, while the other solutions aren't. This is one of the several benefits of using the Erlang VM.
req/s average for 8000 requests, concurrency 10 (ab -n 8000 -c 10):
   Routes     | Frankie/Elixir | SinatraS/Ruby 1.9.2 | Express/node.js 0.4.7
      1       |     8589.62    |      3794.94        |      5251.52
   10 + 1     |     8183.44    |      3527.25        |      5015.64
  100 + 1     |     6378.33    |      2361.22        |      4638.69
  1000 + 1    |     1916.89    |      612.68         |      2028.34

req/s average for 8000 requests, concurrency 40 (ab -n 8000 -c 40):
   Routes     | Frankie/Elixir | SinatraS/Ruby 1.9.2 | Express/node.js 0.4.7
      1       |     9138.8     |      3833.54        |      4730.74
   10 + 1     |     8761.95    |      3561.69        |      4635.8
  100 + 1     |     6479.53    |      2472.51        |      4171.94
  1000 + 1    |     1906.63    |      598.47         |      1904.71

Conclusion

Thanks to the Erlang VM, Frankie performs very well in the basic hello world case. However, the performance is reduced greatly as we add more routes. node.js seems to be the one less affected with the addition of more routes.
The performance degradation seen in Frankie is explained by the fact Elixir is still a language in development and many parts of the language still haven't been optimized yet.
In any case, it is very exciting that tools like Frankie can already be built with Elixir on top of the Erlang VM and we could definitely go a long way if we continue working on it.
PS: If you are interested, here is Frankie's source code.

  
## 1_elixir_frankie.exs
module MyApp
  mixin Frankie::App

  get "/hello_world", def
    "Hello World"
  end
end

Frankie.run 'mochiweb, MyApp

## 2_ruby_1.9.2_sinatra_synchrony.rb
require 'sinatra/base'
require 'sinatra/synchrony'

class HelloWorld < Sinatra::Base
  register Sinatra::Synchrony

  get '/hello_world' do
    'Hello World'
  end
end

## 3_node_express.js
var app = require('express').createServer();

app.get('/hello_world', function(req, res){
  res.send('hello world');
});

app.listen(3000);
	module MyApp
	mixin Frankie::App

	get "/hello_world", def
	"Hello World"
	end
	end

	Frankie.run 'mochiweb, MyApp
	require 'sinatra/base'
	require 'sinatra/synchrony'

	class HelloWorld < Sinatra::Base
	register Sinatra::Synchrony

	get '/hello_world' do
	'Hello World'
	end
	end
	var app = require('express').createServer();

	app.get('/hello_world', function(req, res){
	res.send('hello world');
	});

	app.listen(3000);