Threads vs Async Ruby

async.rb

require "async"

CONCURRENCY = 1000
ITERATIONS = 100

def work
  Async do |task|
    CONCURRENCY.times do
      task.async do
        sleep 1
      end
    end
  end
end

def duration
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  work
  Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
end

def average
  ITERATIONS.times.sum {
    duration
  }.fdiv(ITERATIONS)
end

puts average # => 1.01772911996115

thread.rb

CONCURRENCY = 1000
ITERATIONS = 100

def work
  CONCURRENCY.times.map {
    Thread.new do
      sleep 1
    end
  }.each(&:join)
end

def duration
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  work
  Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
end

def average
  ITERATIONS.times.sum {
    duration
  }.fdiv(ITERATIONS)
end

puts average # => 1.045861059986055

Thank you for participating in the discussion.

If you double the number of threads doing the job the overhead roughly doubles... I think what we're seeing is thread contention at play.

This makes sense.

However, the overall workload is not 3x faster. It's 1.45 / 1.17 # => 1.24x faster.

Hm, it's 45ms vs 17ms, so I think the math here should be 1.045 / 1.017 # => 1.027x or 2.7% faster.

If our workload was an API request that takes 0.2 seconds then it would be 0.65 / 0.27 # => 2.4x faster for a net gain of 18ms for switching to fibers which seems significant.

Same here, I think the math should be 0.245 / 0.217 #=> 1.129x or 12.9% faster.

The average delay per request is 0.045 / 1000 # => 0.000045 for threads and 0.017 / 1000 #=> 0.000017 for fibers meaning that the delta would be (0.2 + 0.000045) / (0.2 + 0.000017) # => 1.0001x faster per request which seems far less significant.

I'm not sure if this is the right way to look at it. I can't prove your math wrong, but the way I'm looking at it is: for 1k threads, some requests that take 0.2s will return in exactly 0.2s (thread scheduler schedules them first, no overhead), while some will return in 0.245s (thread scheduler schedules them last, 45ms overhead). My conclusion is that the average request duration for threads is 0.2225s or about 11% overhead on average.

For fibers, the average request duration would be 0.2085s or 4.2% average overhead per request (with 1000 concurrent requests).

My take is that threads have 2-3x more overhead than fibers, but it's still just 11% (threads) vs 4% (fibers) per-request average overhead. Frankly, I expected this difference to be bigger.

I'm not sure if there's other prior art on how to think about/measure or compare overall expected impact to overall workload.

Noah Gibbs did an interesting threads vs fibers comparison:

https://engineering.appfolio.com/appfolio-engineering/2019/9/13/benchmarking-fibers-threads-and-processes

Unfortunately, links to his code examples on github return 404.

You should test real world case, like one event loop creating lots of fibers, rather than creating lots of event loops.

@bruno- thanks for catching my math mistakes 🙊 .

I'm not sure if this is the right way to look at it. I can't prove your math wrong,

In general, I'm nervous when there's a variable that I can arbitrarily change to get different (comparative) results. It makes me worried that I've gamed my own microbenchmark.

Not sure if you've seen but I've done a bunch of work in the space of trying to ensure a benchmark result are actually valid

• Take a look at comparative histograms and statistical significance checks https://github.com/zombocom/derailed_benchmarks#i-made-a-patch-to-to-rails-how-can-i-tell-if-it-made-my-rails-app-faster-and-test-for-statistical-significance
• Not specifically performance but about tuning a system that varies when number inputs are changed https://www.schneems.com/2020/07/08/a-fast-car-needs-good-brakes-how-we-added-client-rate-throttling-to-the-platform-api-gem/

My conclusion is that the average request duration for threads is 0.2225s or about 11% overhead on average.

That's the average across 200 runs. Each run does 1000 operations which is what I was dividing by (if that makes sense).

I've seen the noah article ages ago, but forgotten about it (thanks for the reminder). The code is here https://github.com/noahgibbs/fiber_basic_benchmarks/tree/master/benchmarks that file got renamed it looks like.