CallbackResumePattern-RubyFibers.md

The Callback-Resume pattern

The callback-resume pattern is used to leverage Ruby fibers for async-IO tasks. You can avoid callback nesting a la Node, and having to manually manage all the context and state from one callback to another.

You can get the key concepts about Ruby fibers from here:

1. [The best description of Ruby fiber] (lee.hambley.name/2012/06/19/the-best-description-of-a-ruby-fiber.html)
2. [Ruby Fiber documentation] (ruby-doc.org/core-2.2.0/Fiber.html)

Per chance, if you have used Boost Coroutines (C++), you know what Ruby Fibers are (and vice versa).

Premise

We want to write a service that uses async-IO but provides a synchronous interface.

Key concepts

1. The key underlying concept is that each Fiber has a context. A function runnning in the context of a fiber can call other functions and the entire stack shares the same context. Whenever any function in the call stack yields, the control goes back to the code that started / resumed the Fiber in which the function was running, not necessarily the immediate caller of the function.

2. Additional necessary concepts are that when we resume a Fiber that had yielded, the control inside the Fiber starts from the last call to Fiber.yield. This call to Fiber.yield returns the value(s) passed to the resume call that resumed the fiber this time.

3. A call to Fiber.yield inside a fiber can be passed arguments. When the resume call that resumed the fiber returns, it will return these arguments.

The pattern

Synchronous call:

handle = get_endpoint
data = get_data(handle)
handle.close
process_data(data)

Async call:

get_endpoint_async do |endpoint|
  get_data_async(endpoint) do |data|
    endpoint.close_async
  end

  process_data(data)
end

Callback-resume pattern:

f = Fiber.current
get_endpoint_async do |endpoint|
  f.resume(endpoint)
end

endpoint = Fiber.yield

get_data_async(endpoint) do |data|
  f.resume(data)
end

data = Fiber.yield

endpoint.close_async

process_data(data)

Additional references

1. [Untangling evented code] (https://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/)
2. [Pipelines using Ruby fibers] (https://pragdave.me/blog/2007/12/30/pipelines-using-fibers-in-ruby-19/)