Presentation for OSDC 2014: http://2014.osdc.com.au/presentation/1268-building-300-seat-inflight-entertainment-system-controller-twisted

1. TWISTED AS AN IOT
CONTROLLER.
BUILDING A 300+ SEAT IFE CONTROLLER IN TWISTED.
David P. Novakovic / @dpn / dpn.name

2. QUICK OVERVIEW
WTF is Twisted?
IFE System Devices
IFE System requirements
Code examples

3. TWISTED THE SLAYER OF NOOBS
SUPER POWERFUL
ALSO CONTROVERSIAL
Seems to have a bad rep with people who don't use it.

4. TWISTED
event driven framework in python
particular focus on networking related things
from web down to serial ports, tun/tap and udev
async io - single process can handle thousands of concurrent
open connections on a low end machine
Clean APIs for both callbacks and deferreds
thread pool with deferred interface
locking "primitives" - DeferredSemaphore, DeferredLock,
DeferredQueue

5. MORE TWISTED
Application framework
Async unit testing
Integration with most major event loops
(wx/gtk/qt/gevent/etc)
third party libs
klein - sinatra/flask style web framework
cyclone - tornado port to twisted
treq - "requests" style lib
ampoule - nice enough (if slightly neglected) process pool
implementation

6. IFE STUFF.. YAY

7. TRADITIONAL
"dumb" screens heavy use of streaming content
very heavy requirements on servers
very expensive to upgrade
GLIDE
Heavier clients
system can be upgraded by replacing players
plane doesnt need rewiring etc
server can remain fairly unchanged
more importantly don't need to be upgraded in lockstep

8. GLIDE SOLUTION
players - embedded debian
seat disconnect - custom embedded device
triple redundant power supplies - switch + power
cm controller - dual core atom running ubuntu LTS + lots of
embedded things attached

9. SYSTEM

10. SOME RELEVANT REQUIREMENTS
multicast commands out to embedded devices
300+ seat updates over HTTP every second
listen to audio stream over multicast (PA, pilot etc)
low latency control of players ie. if pilot talks/decomp
telneting into a streaming VLC process.
some legacy - sync code needed to run in threads
respond to and control hardware in the plane (overhead
screens etc)
cabin crew inserting HDD with content (lock down usb)
downloading content from the web (at gate)
kiosk (lock down control keys/usb ports)
manhole for debugging a running process
ssh reverse tunnel for remote access - conch
tftp - firmware updates to players

11. MULTICAST
from twisted.internet import reactor, task, protocol
class MulticastSeatControl(protocol.DatagramProtocol):
def startProtocol(self):
self.transport.joinGroup("228.0.0.5")
def sendSeatControl(self, bytes):
self.transport.write(bytes, ("228.0.0.5", 8005))
def datagramReceived(self, datagram, address):
print "Datagram %s from %s" % (repr(datagram), repr(address))

12. HTTP INTERFACE
import json
from klein import Klein
class DeviceAPI(object):
app = Klein()
def __init__(self, cmc):
self._cmc = {}
@app.route('/stat/<string:name>')
def stat(self, request):
body = json.loads(request.content.read())
self._cmc.logstat(body['something'])
request.setHeader('Content-Type', 'application/json')
return json.dumps({"status": "success"})</string:name>

13. SERIAL PORT
from twisted.internet.serialport import SerialPort
from twisted.internet import reactor
from twisted.protocols.basic import LineReceiver
class SensorProtocol(LineReceiver):
def connectionMade(self):
print "Connected to serial port."
def lineReceived(self, line):
print "Received line:", line
def send_our_command(self, command):
self.sendLine("command:%s" % command)
def connect_serial_port():
return SerialPort(SensorProtocol(), "/dev/ttyACM0", reactor,
115200, rtscts=False, xonxoff=False, timeout=1)

14. DJANGO
WAIT.. WHAT?
Yep, ORM is used a fair bit, as is admin.

15. DJANGO
from django.core.handlers.wsgi import WSGIHandler
from twisted.python import threadpool
from twisted.internet import reactor
from somewhere import Root
def gimme_some_django():
pool = threadpool.ThreadPool()
wsgi_resource = wsgi.WSGIResource(reactor, pool, WSGIHandler())
r = Root(wsgi_resource)
s = server.Site(r)
pool.start()
return internet.TCPServer(8082, s)
eg. django admin available at http://localhost:8082/admin/
https://github.com/clemesha/twisted-wsgi-django

16. YET MORE
class VLCRemoteControlProtocol(LineReceiver):
...
class UDevMonitor(abstract.FileDescriptor):
...

17. AND TELNET INTO YOUR OWN PROCESS..
import twisted.manhole.telnet
from twisted.internet.endpoints import TCP4ServerEndpoint
def start_manhole(service):
f = twisted.manhole.telnet.ShellFactory()
f.username = "b"
f.password = "b"
f.namespace["var1"] = service
endpoint = TCP4ServerEndpoint(reactor, 7777)
endpoint.listen(f)

18. TIE IT ALL TOGETHER
Application framework
Unit Testing

19. APPLICATION FRAMEWORK
TWISTD
Daemonises optionally
pidfile
log files
Error catchall
http://twistedmatrix.com/documents/13.2.0/core/howto/basics.html

20. CREATE SERVICES
class IOTCServer(Service):
def startService(self):
self.serialport = connect_serial_port()
p = MulticastSeatControl()
reactor.listenMulticast(8005, p)
start_manhole(self)
self.mc_looper = task.LoopingCall(p.sendSeatControl, "s1:on,s2:on")
self.mc_looper.start(1)
def stopService(self):
self.serialport.transport.loseConnection()
self.mc_looper.stop()
def get_http_service(iots_instance):
device_api = DeviceAPI(iots_instance)
server.Site(device.api.app.resource())
return internet.TCPServer(8082)

21. TWISTD PLUGIN
class Options(usage.Options):
optParameters = [["serialport", "s", "/dev/ttyACM0", "Serial port."]
class ServiceMaker(object):
tapname = "iotc"
description = "The IOTC Server"
options = Options
def makeService(self, options):
service_parent = service.MultiService()
iots = IOTCServer()
iots.setServiceParent(service_parent)
http_service = get_http_service()
http_service.setServiceParent(service_parent)
django_service = gimme_some_django()
django_service.setServiceParent(service_parent)
return service_parent
serviceMaker = ServiceMaker()
yourproject/twisted/plugins/iotc_plugin.py
$ twistd -r epoll -n iotc --serialport=/dev/ttyACM0

22. UNIT TESTING
Twisted Trial
Wrapper around pyunit "unittest"
Allows tests to return deferred responses
subclass twisted.trial.unittest.TestCase
run tests: trial yourpackage
Some helpers to let you mock wire-level connections
http://twistedmatrix.com/documents/11.1.0/core/howto/trial.html

23. BUILD AWESOME THINGS!
DIGECOR IS HIRING
If you enjoy working on this kind of stuff, let me know and I'll
forward your details onto digEcor.
@dpn

24. THE END
DAVID NOVAKOVIC - DPN.NAME