Asynchronous programming in Tornado

README.md

Asynchronous programming with Tornado

Asynchronous programming can be tricky for beginners, therefore I think it's useful to iron some basic concepts to avoid common pitfalls.

For an explanation about generic asynchronous programming, I recommend you one of the many resources online.

I will focus on solely on asynchronous programming in Tornado. From Tornado's homepage:

FriendFeed's web server is a relatively simple, non-blocking web server written in Python. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the non-blocking web server and tools.

Tornado is an open source version of this web server and some of the tools we use most often at FriendFeed. The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll or kqueue, it can handle thousands of simultaneous standing connections, which means the framework is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)

The first step as a beginner is to figure out if you really need to go asynchronous. Asynchronous programming is more complicated that synchronous programming, because, as someone described, it does not fit human brain nicely.

You should use asynchronous programming when your application needs to monitor some resources and react to changes in their state. For example, a web server sitting idle until a request arrives through a socket is an ideal candidate. Or an application that has to execute tasks periodically or delay their execution after some time. The alternative is to use multiple threads (or processes) to control multiple tasks and this model becomes quickly complicated.

The second step is to figure out if you can go asynchronous. Unfortunately in Tornado, not all the tasks can be executed asynchronously. Tornado is single threaded (in its common usage, although in supports multiple threads in advanced configurations), therefore any "blocking" task will block the whole server. This means that a blocking task will not allow the framework to pick the next task waiting to be processed. The selection of tasks is done by the IOLoop, which, as everything else, runs in the only available thread.

For example, this is a wrong way of using IOLoop:

[gist id=3826189 file=blocking.py]

Note that blocking_call is called correctly, but, being blocking (time.sleep blocks!), it will prevent the execution of the following task (the second call to the same function). Only when the first call will end, the second will be called by IOLoop. Therefore, the output in console is sequential ("sleeping", "awake!", "sleeping", "awake!").

Compare the same "algorithm", but using an "asynchronous version" of time.sleep, i.e. add_timeout:

[gist id=3826189 file=async_sleep_1.py]

In this case, the first task will be called, it will print "sleeping" and then it will ask IOLoop to schedule the execution of the rest of the routine after 1 second. IOLoop, having the control again, will fire the second call the function, which will print "sleeping" again and return control to IOLoop. After 1 second IOLoop will carry on where he left with the first function and "awake" will be printed. Finally, the second "awake" will be printed, too. So, the sequence of prints will be: "sleeping", "sleeping", "awake!", "awake!". The two function calls have been executed concurrently (not in parallel, though!).

So, I hear you asking, "how do I create functions that can be executed asynchronously"?

In Tornado, every function that has a "callback" argument can be used with gen.engine.Task. Beware though: being able to use Task does not make the execution asynchronous! There is no magic going on: the function is simply scheduled to execution, executed and whatever is passed to callback will become the return value of Task. See below:

[gist id=3826189 file=async_generic.py]

Most beginners expect to be able to just say: Task(my_func) and automagically execute my_func asynchronously. This is not how Tornado works. This is how Go works!

And this is my last remark: In a function that is going to be used "asynchronously", only asynchronous libraries should be used. By this, I mean that blocking calls like time.sleep or urllib2.urlopen or db.query will need to be substituted by their equivalent asynchronous version. For example, IOLoop.add_timeout instead of time.sleep, AsyncHTTPClient.fetch instead of urllib2.urlopen etc. For DB queries, the situation is more complicated and specific asynchronous drivers to talk to the DB are needed. For example: Motor for MongoDB.

async_generic.py

import time
from tornado.ioloop import IOLoop
from tornado import gen


def my_function(callback):
    print 'do some work'
    # Note: this line will block!
    time.sleep(1)
    callback(123)


@gen.engine
def f():
    print 'start'
    # Call my_function and return here as soon as "callback" is called.
    # "result" is whatever argument was passed to "callback" in "my_function".
    result = yield gen.Task(my_function)
    print 'result is', result
    IOLoop.instance().stop()


if __name__ == "__main__":
    f()
    IOLoop.instance().start()

async_sleep_1.py

# Example of non-blocking sleep.
import time
from tornado.ioloop import IOLoop
from tornado import gen


@gen.engine
def f():
    print 'sleeping'
    yield gen.Task(IOLoop.instance().add_timeout, time.time() + 1)
    print 'awake!'


if __name__ == "__main__":
    # Note that now code is executed "concurrently"
    IOLoop.instance().add_callback(f)
    IOLoop.instance().add_callback(f)
    IOLoop.instance().start()

async_sleep_2.py

# Inspired by: http://emptysquare.net/blog/pausing-with-tornado/
# Test with:
# ab -c 10 -n 10 "http://localhost:8888/"

import time
import tornado.web
from tornado.ioloop import IOLoop
from tornado import gen


class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @gen.engine
    def get(self):
        self.write("Going to sleep...")
        # Note that nothing is written to the client until `finish` is called!
        yield gen.Task(IOLoop.instance().add_timeout, time.time() + 5)
        self.write("I'm awake!")
        self.finish()
        # or use `self.render` that calls `finish` itself
        # See http://www.tornadoweb.org/documentation/_modules/tornado/web.html#RequestHandler.render


application = tornado.web.Application([
    (r"/", MainHandler),
    ])


if __name__ == "__main__":
    application.listen(8888)
    IOLoop.instance().start()

blocking.py

# Example of misuse of callback. DON'T DO THIS!
import time
from tornado.ioloop import IOLoop


def blocking_func():
    print 'sleeping'
    time.sleep(1)
    print 'awake!'


if __name__ == "__main__":
    # Note that code is executed sequantially!
    IOLoop.instance().add_callback(blocking_func)
    IOLoop.instance().add_callback(blocking_func)
    IOLoop.instance().start()

periodic_1.py

import time
from tornado.ioloop import IOLoop, PeriodicCallback


def task():
    print time.time()


if __name__ == "__main__":
    PeriodicCallback(task, 1000).start()
    IOLoop.instance().start()

periodic_2.py

# Hand crafted periodic call
import time
from tornado.ioloop import IOLoop


def delayed_print(s):
    if len(s) == 0:
        # Stop loop when finished writing
        IOLoop.instance().stop()
    else:
        # Print first character
        print s[0]
        # Schedule next cycle in 1 second
        IOLoop.instance().add_timeout(
                time.time() + 1, lambda: delayed_print(s[1:]))


if __name__ == "__main__":
    # Start 2 processes: output will be interleaved
    delayed_print('hello, world!')
    delayed_print('ciao, mondo!')
    IOLoop.instance().start()

Nice, thanks.

Thanks

Thanks