Introduce to AsyncIO

asyncio intro.md

AsyncIO

Prerequisite

Overviews

PEP: 3165
python module name: asyncio, new in Python 3.4
reference implementation: tulip
backport: trollius ( python >= 2.6)

I/O Multiplexing

• select (windows, unix)
• poll & epoll (linix)
• kqueue (BSD)
• devpoll (Solaris)
• iocp (Windows) [w/o select, w/ asyncio]

Builtin Modules

select - low level sys call

import select, socket

rlist, wlist, xlist = select.select(rlist, wlist, xlist[, timeout])

poller = select.epoll()
sock = socket.socket(...)
_mapping = {sock.fileno(): sock}
poller.register(sock.fileno(), select.EPOLLIN | select.EPOLLOUT | select.EPOLLPRI)
for fd, event in poller.poll():
	fileobj = _mapping[fd]
	if event & select.EPOLLIN:
		...

selectors - high level API, unifies platforms

import selectors
import socket

sel = selectors.DefaultSelector()

def accept(sock, mask):
    conn, addr = sock.accept()  # Should be ready
    print('accepted', conn, 'from', addr)
    conn.setblocking(False)
    sel.register(conn, selectors.EVENT_READ, read)

def read(conn, mask):
    data = conn.recv(1000)  # Should be ready
    if data:
        print('echoing', repr(data), 'to', conn)
        conn.send(data)  # Hope it won't block
    else:
        print('closing', conn)
        sel.unregister(conn)
        conn.close()

sock = socket.socket()
sock.bind(('localhost', 1234))
sock.listen(100)
sock.setblocking(False)
sel.register(sock, selectors.EVENT_READ, accept)

while True:
    events = sel.select()
    for key, mask in events:
        callback = key.data
        callback(key.fileobj, mask)

• asyncore & asynchat (deprecated)
• asyncio - application level framework

Generator

• yield - PEP 255 -- Simple Generators

itertools.count

def count(firstval=0, step=1):
	x = firstval
	while 1:
	    yield x
	    x += step

• generator.send(None) - PEP 342 -- Coroutines via Enhanced Generators

import random
def countdown(x):
	step = 1
	while x > 0:
		x -= step
		step = yield x
		
down = countdown(10)
down.send(None)  # start generator, = next(down)
while True:
	try:
		down.send(random.randrange(1, 10))
	except StopIteration:
		break

• yield from generator - PEP 380 -- Syntax for Delegating to a Subgenerator

def g(x):
	for i in range(x):
		yield x

def f(x):
	for i in range(x):
		yield from g(i)

f(4)
# generator

list(f(4))
# [0,  0,1, 0,1,2, 0,1,2,3]

Old & Good Ways

• twisted/tornado - based on select/epoll/...
• gevent - based on greenlet & libev

Pitfall

• tightly coupled, not pluggable

AsyncIO comes to rescue