concurrent requests in tc-analytics

notes.md

Webrick

Conventional wisdon in the ruby community is that webrick in unsuitable for production, partly because it's single threaded.

For example, the heroku docs say:

By default WEBrick is single threaded, single process

And this

Webrick is a single-thread, single-process web server

And this

by default WEBrick cannot handle more than one request at a time

In reality, Webrick can run multi-threaded, but rails < 5 added a mutex (via Rack::Lock) to serialise requests.

In Rails 5.0, Rack::Lock isn't used any more so requests to webrick can run in parallel.

To test this, I changed HealthController:

class HealthController < ApplicationController
  def show
    sleep(10)
    head :no_content
  end
end

... then started rails (./bin/rails s webrick -p 3005) and ran ab to confirm 10 parallel requests completed in ~10 seconds.

$ ab -n 10 -c 10 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WEBrick/1.3.1
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   10.649 seconds
Complete requests:      10
Failed requests:        0
...

I then wanted to test concurrent requests with a DB query, so I changed HealthController to

class HealthController < ApplicationController
  def show
    Filter.connection.select_all("select pg_sleep(10)")
    head :no_content
  end
end

The same ab test triggered 5 exceptions:

$ ab -n 10 -c 10 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WEBrick/1.3.1
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      10
Time taken for tests:   10.709 seconds
Complete requests:      10
Failed requests:        0
Non-2xx responses:      5

5 of the 10 requests resulted in an exception:

ActiveRecord::ConnectionTimeoutError - could not obtain a connection from the pool within 5.000 seconds (waited 5.002 seconds); all pooled connections were in use

Interestingly, it seems that webrick doesn't have a cap on it's thread count. The activerecrod connection pool is a limitation, but if we use ab to test a request that doesn't access the DB then webrick will successfully process 50 parallel requests:

$ ab -n 50 -c 50 -l http://127.0.0.1:3005/health

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WEBrick/1.3.1
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   10.293 seconds
Complete requests:      50
Failed requests:        0

The same test against an action that requires DB connections is catastrophic - only 10% of requests complete:

$ ab -n 50 -c 50 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:        WEBrick/1.3.1
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   11.036 seconds
Complete requests:      50
Failed requests:        0
Non-2xx responses:      45

Conclusion? On highly concurrent apps, dDon't run webrick in development on rails 5.0+

Puma

If I start the rails server with puma instead (./bin/rails s -p 3005) and test with a ruby sleep (no DB requests):

$ ab -n 10 -c 10 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      10
Time taken for tests:   20.554 seconds
Complete requests:      10
Failed requests:        0

No failed requests. Our puma config (config/puma.rb) establishes 6 requests, so 10 requests can complete in 20 seconds (6 requests in the first 10 seconds, then 4 requests in the next 10 seconds).

Here's the same test with a DB sleep:

$ ab -n 10 -c 10 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      10
Time taken for tests:   21.165 seconds
Complete requests:      10
Failed requests:        0
Non-2xx responses:      2

The rails default DB connection pool size is 5 - one less than the number of puma threads we start. That means one request at a time is likely to be unable to open a DB connection.

If I edit config/database.yml to increase the DB connection pool to 10 in development and re-run ab:

$ ab -n 10 -c 10 -l http://127.0.0.1:3005/health
Benchmarking 127.0.0.1 (be patient).....done

Server Software:
Server Hostname:        127.0.0.1
Server Port:            3005

Document Path:          /health
Document Length:        Variable

Concurrency Level:      10
Time taken for tests:   20.610 seconds
Complete requests:      10
Failed requests:        0

Every request completes.