Async I/O in Python 3 with Tulip

Asychronous I/O in
Python 3
Feihong Hsu
Chicago Python User Group
July 11, 2013

What is PEP 3156?
• "Asynchronous IO Support Rebooted"
• Describes a new event loop API
• Aims to be compatible with current
popular event loops (Twisted,Tornado, etc.)
• Should land in the standard library in 3.4
(early 2014)

The big take-away
We're going to be able to write
asynchronous code without threads or
callbacks!

Threads vs event loop
• I'm not going to talk about this
• Twisted being around for more than a
decade kinda validates the event loop
model
• For a detailed discussion see An Intro to
Asynchronous Programming and Twisted

Twisted vs Tulip
Ugh, how to explain this?
I could just give you a boring list of
differences...
Or I could use an analogy...

Programming with
Twisted is like...
Being a mystical ninja warrior!

Mystical Ninja Warrior
• Can defeat demons and demigods by
deploying an army of shadow clones
• Mental gymnastics required to coordinate
shadow clones requires years of brutal
training
• Ninja magic comes with a terrible price: the
more clones you make, the more insane
you become

Programming with Tulip
is like...
Playing PlantsVs Zombies

Playing PlantsVs
Zombies
• Can defeat a zombie horde by deploying
and managing a large number of
horticultural specimens
• Not particularly mentally taxing
• Somehow end up playing for hours without
eating or sleeping

Alrighty
With the analogies out of the way, let's look
at some examples.

Fetching a web page in
Twisted
https://github.com/feihong/tulip-talk/blob/
master/examples/1-twisted-download.py

Control ﬂow diagram
I was going to draw one but then I got
sleepy.

Problems with callback-
based asynch code
• Can be hard to understand control ﬂow
• Error messages are not friendly
• Debugging is harder

Tulip
master/examples/2-tulip-download.py

Tulip w/ callbacks
master/examples/3-tulip-download-
callbacks.py

Twisted w/ inline
callbacks
master/examples/4-twisted-download-inline-
callbacks.py

The Tulip API
• Basics are documented in PEP 3156
• Not available in Python Package Index
• Includes sockets, ﬁle I/O, etc.
• Includes concurrency data structures like
locks, queues, semaphores
• Includes SSL and HTTP (with support for
websockets!)

Coroutines vs tasks
When I ﬁrst started, the most confusing
thing about Tulip.
Coroutines are executed by "yield from".
Tasks are not executed by "yield from".

Coroutine
• Basically a function that contains at least
one "yield from" statement.
• Not the same thing as a generator
function, which is a function that contains a
yield expression.
• Tulip will barf if you try to make it execute
a generator.

Coroutine
@tulip.coroutine
def download(url):
response = yield from tulip.http.request('GET', url)
return yield from response.read()
Calling "download()" returns a generator
object, but otherwise does nothing!You need
to do "yield from download()" to run the
body of a coroutine.

Task
• An object that manages a coroutine
• Roughly equivalent to the Deferred object
in Twisted

Task
@tulip.task
def download(url):
response = yield from tulip.http.request('GET', url)
return yield from response.read()
Calling "download()" actually does run the
body of the function.The "yield from" part is
done implicitly for you.

Task
In Tulip, there are two ways of creating tasks:
tulip.task is a decorator that produces
a task-wrapping function
tulip.Task is a constructor that accepts
a coroutine

Why bother to use
tasks?
• To interoperate with callback-based
frameworks like Twisted
• To cancel an already-running coroutine
• To start a new coroutine from within
another coroutine

Wait, so what's a
Future?
PEP 3156 makes frequent mention of
Futures.
But talking about Futures is a little confusing,
since there are two Future classes:
tulip.Future and concurrent.futures.Future.

Future is the superclass
of Task
• Future don't necessarily manage a
coroutine
• In practice you never create Future
objects, only Task objects
• Futures are acceptable to yield from
expressions

Methods on Future
• cancel(), cancelled()
• running(), done()
• result(), set_result()
• add_done_callback(),
remove_done_callback()
• ...

Web development with
Tulip
• All the classes you need are in the tulip.http
module
• Make subclass of
tulip.http.ServerHttpProtocol
• Override the handle_request() method

Hello World!
master/examples/5-hello-world-http.py

Some observations
The HTTP API is fairly simple, but a bit low
level for everyday web programming.
Expect a thousand microframeworks to
bloom in the near future.
Speaking of microframeworks...

Introducing viol
• Tiny web framework based on Tulip
• After initial page load, messages between
client and server are exchanged via
websockets
• Makes code demos a bit more visual

Tulip API demos
Now for a bunch of demos...

Serial
http://www.youtube.com/v/jMgRUI7V_mk?
version=3&autoplay=1&&start=1546

Simultaneous

Wait for both
coroutines to ﬁnish

Loop through
coroutine results in
order of ﬁnishing

Execute synchronous
code in another thread

Talk to event loop from
another thread

A regex debugging tool

Chat application

Tulip Links
• PEP 3156
• Guido's PyCon 2013 keynote
• Tulip project page on GoogleCode

Other links
• An Introduction to Asynchronous
Programming and Twisted
• Code for this talk on Github
• Viol project page on Github

Async I/O in Python 3 with Tulip

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (11)

Semelhante a Async I/O in Python 3 with Tulip

Semelhante a Async I/O in Python 3 with Tulip (20)

Último

Último (20)

Async I/O in Python 3 with Tulip