kemal_vs_sinatra.md

Kemal vs Sinatra Benchmark

This is just for demonstration purpose. A simple app just returning "Hello World". First in Kemal (Crystal) and then in Sinatra(Ruby with thin).

Kemal (Crystal)

Kemal Version: 0.6.0 Crystal Version: 0.10.0

# app.cr
require "kemal"

get "/" do
  "Hello World!"
end

Sinatra (Ruby with Thin)

Sinatra Version: 1.4.6 Ruby Version: 2.2.2p95 Thin Version: 1.6.4

# app.rb
require "sinatra"

set :environment, :production
set :server, %w[thin]

get "/" do
  "Hello World!"
end

Running the benchmark

wrk is used to run this benchmark with 100 connections.

To run Kemal: crystal build --release src/app.cr && ./app To run Sinatra: ruby app.rb -p 3000

Finally to run benchmark:

wrk -c 100 -d 20 http://localhost:3000

So I did it in Flask on uwsgi and it's only slightly better than Sinatra:

Installing:

sudo pip install Flask uwsgi

Running:

uwsgi --wsgi-file app.py --http :3000 --callable app

Code:

# app.py
from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == "__main__":
    app.run()

Results on my Dell Vostro:

Running 20s test @ http://localhost:3000
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    30.29ms    3.94ms  56.38ms   77.82%
    Req/Sec     1.65k   179.57     1.98k    72.25%
  65776 requests in 20.01s, 5.71MB read
  Socket errors: connect 0, read 65774, write 0, timeout 0
Requests/sec:   3286.50
Transfer/sec:    292.06KB

I'll try the same with Rails API later.

Ruby on Rails API only mode with Passenger:

Creation:

rails new hello --api
rails generate controller site index

#app/controllers/site_controller.rb 
class SiteController < ApplicationController
  def index
    render text: 'Hello World!'
  end
end

#config/routes.rb 
Rails.application.routes.draw do
  root 'site#index'
end

Installing (Debian):

\curl -sSL https://get.rvm.io | bash
source ~/.rvm/scripts/rvm
rvm install ruby --latest
rvm default
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo apt-get install -y apt-transport-https ca-certificates
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger jessie main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update
sudo apt-get install -y passenger
sudo gem install rails --pre

Running:

bundle exec passenger start

Results:

Running 20s test @ http://localhost:3000
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    62.13ms    5.26ms 158.56ms   87.41%
    Req/Sec   807.09     63.45     0.93k    84.50%
  32151 requests in 20.02s, 12.05MB read
Requests/sec:   1606.26
Transfer/sec:    616.39KB

So, I've enabled threads in uwsgi and rerun the test:

Running:

uwsgi --wsgi-file app.py --http :3000 --callable app -T

Running 20s test @ http://localhost:3002
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    30.59ms    7.60ms 228.06ms   97.35%
    Req/Sec     1.65k   125.58     1.96k    70.50%
  65673 requests in 20.01s, 5.70MB read
  Socket errors: connect 0, read 65672, write 0, timeout 0
Requests/sec:   3281.32
Transfer/sec:    291.60KB

Well I run Sinatra with passenger and it's a lot better:

#app.rb
require 'sinatra/base'

class HelloApp < Sinatra::Base
  get '/' do
    'Hello World!'
  end
end

#config.ru
require File.absolute_path("app.rb")

run HelloApp

#Gemfile
source 'https://rubygems.org/'

gem 'sinatra'
gem 'passenger'

Running:

set -x RACK_ENV production
bundle exec passenger start

Results:

Running 20s test @ http://localhost:3002
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    20.52ms    8.85ms 210.95ms   93.66%
    Req/Sec     2.51k   274.79     3.05k    74.75%
  99720 requests in 20.01s, 33.28MB read
Requests/sec:   4982.54
Transfer/sec:      1.66MB

You should list the command line options on the website. Had trouble finding the production mode.

You could write that kemal needs ssl development headers to build (libssl-dev).
Made some other benchmarks, put them in a docker. Gonna test rust frameworks someday.
https://github.com/wieczorek1990/benchmarks

Added rust: nickel and iron.