Slides from the talk given at FOSDEM 2013 Python Devroom about how async I/O and event loops work in Python.

1. How do event loops
work in Python?
Saúl Ibarra Corretgé
FOSDEM 2013

2. print(self)
• Hi there!
• @saghul
• I work on VoIP and Real Time Communications
• Python and C are my tools

3. try: sockets
from __future__ import print_function
import socket
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)
server.bind(('127.0.0.1', 1234))
server.listen(10)
print("Server listening on: {}".format(server.getsockname()))
client, addr = server.accept()
print("Client connected: {}".format(addr))
while True:
data = client.recv(4096)
if not data:
print("Client has disconnected")
break
client.send(data.upper())
server.close()

4. except Exception:
• We can only handle one client at a time!
• Solutions for handling multiple clients
• Threads
• I/O multiplexing
• Check the C10K problem if you haven’t already!
• http://www.kegel.com/c10k.html

5. try: sockets + threads
from __future__ import print_function
import socket
import thread
def handle_client(client, addr):
print("Client connected: {}".format(addr))
while True:
data = client.recv(4096)
if not data:
print("Client has disconnected")
break
client.send(data.upper())
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)
server.bind(('127.0.0.1', 1234))
server.listen(10)
print("Server listening on: {}".format(server.getsockname()))
while True:
client, addr = server.accept()
thread.start_new_thread(handle_client, (client, addr))

6. except Exception:
• Threads have overhead
• Stack size
• Context switching
• Synchronization
• GIL?
• Not for I/O!

7. I/O multiplexing
• Examine and block for I/O in multiple file
descriptors at the same time
• Single thread
• A file descriptor is ready if the corresponding I/O
operation can be performed without blocking
• File descriptor has to be set to be non-blocking

8. I/O multiplexing (II)
1. Put file descriptors in non-blocking mode
2. Add file descriptors to a I/O multiplexor
3. Block for some time
4. Perform blocking operations on file descriptors
which are ready

9. def set_nonblocking
def set_nonblocking(fdobj):
try:
setblocking = fdobj.setblocking
except AttributeError:
try:
import fcntl
except ImportError:
raise NotImplementedError
try:
fd = fdobj.fileno()
except AttributeError:
fd = fdobj
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
else:
setblocking(False)

10. select
• Lowest common denominator I/O multiplexor
• Takes a list of “readers”, “writers”, “exceptional
conditions” and a timeout
• Limit of FD_SETSIZE file descriptors, usually 1024
• Processing takes O(number of fds)

12. epoll
• “A better select”
• No FD_SETSIZE limit!
• Processing takes O(1) time!
• Linux only

13. Other I/O
multiplexors
• kqueue
• Like epoll but for Max OSX and BSD
• poll
• Like select but without a limit of file descriptors
• O(number of file descriptors) processing time
• Very broken on some systems (OSX 10.3-5)

14. sys.platform == “win32”
• Windows supports select
• 64 file descriptors per thread - WAT
• Starting with Vista WSAPoll (poll) is supported
• It’s broken - http://daniel.haxx.se/blog/
2012/10/10/wsapoll-is-broken/
• IOCP is the good stuff
• Based on completion, not readiness

15. edge/level triggering
• Level triggering (the most common)
• You get notified when the condition is present
• Edge triggering
• You get notified when the condition happens
• epoll and kqueue support both mechanisms
• IOCP is kind-of edge-triggered epoll

17. Event loop libraries
epoll, IOCP on
File I/O
kqueue windows
libev YES NO libeio
libevent YES YES NO
libuv YES YES YES

18. Event loop libraries
(II)
1. Update loop time
2. Process timers
3. Process idle handles
4. Process prepare handles
5. Block for I/O
6. Process check handles

19. libuv: the power
underneath nodeJS

20. nodeJS <= 0.4.x
node standard library
JavaScript
C / C++
node bindings
V8 libev libeio

21. nodeJS >= 0.6.x
node standard library
JavaScript
C / C++
node bindings
V8 libuv

22. libuv
• Originally based on libev + libeio + IOCP
• Now: epoll + kqueue + event ports + IOCP + file I/O
• TCP, UDP, named pipes, TTY
• Nice additions: interface addresses, process title, ...
• Most complete cross platform networking library
• Python bindings - pyuv

23. import pyuv
• Written in C, wraps everything libuv has to offer
• Python >= 2.6, supports Python 3!
• Works on Windows
• https://github.com/saghul/pyuv

24. from __future__ import print_function
import signal
import pyuv
def on_read(client, data, error):
if data is None:
print("Client read error: {}".format(pyuv.errno.strerror(error)))
client.close()
clients.remove(client)
return
client.write(data.upper())
def on_connection(server, error):
client = pyuv.TCP(server.loop)
server.accept(client)
clients.append(client)
client.start_read(on_read)
print("Client connected: {}".format(client.getpeername()))
def signal_cb(handle, signum):
[c.close() for c in clients]
signal_h.close()
server.close()
clients = []
loop = pyuv.Loop.default_loop()
server = pyuv.TCP(loop)
server.bind(("127.0.0.1", 1234))
server.listen(on_connection)
signal_h = pyuv.Signal(loop)
signal_h.start(signal_cb, signal.SIGINT)
loop.run()

25. import pyuv
• Experiments in replacing event loops of common
Python networking frameworks
• Twisted - https://github.com/saghul/twisted-pyuv
• Tornado - https://github.com/saghul/tornado-pyuv
• Gevent - https://github.com/saghul/uvent

26. import pyuv
• gaffer: application deployment and supervisor
• https://github.com/benoitc/gaffer
• ts_analyzer: realtime analysis of multicast video
streams
• https://github.com/tijmenNL/ts_analyzer

27. The Python
async I/O problem
• Each framework uses it’s own event loop
implementation
• Protocols aren’t reusable
• PEP-3156 to the rescue!
• For Python >= 3.3
• Reference implementation (Tulip)
https://code.google.com/p/tulip/

28. import rose
• PEP-3156 EventLoop implementation using pyuv
• pyuv.Poll: high performance level triggered I/O
(works on Windows as well!)
• Uses fairy dust covered unicorns
• https://github.com/saghul/rose

29. import rose
import signal
from rose import events, protocols
class EchoProtocol(protocols.Protocol):
def connection_made(self, transport):
# TODO: Transport should probably expose getsockname/getpeername
print("Client connected: {}".format(transport._sock.getpeername()))
self.transport = transport
def data_received(self, data):
self.transport.write(data.upper())
def eof_received(self):
self.transport.close()
def connection_lost(self, exc):
print("Client closed connection")
reactor = events.new_event_loop()
events.set_event_loop(reactor)
reactor.start_serving(EchoProtocol, '127.0.0.1', 1234)
reactor.add_signal_handler(signal.SIGINT, reactor.stop)
reactor.run()

31. saghul