Event driven consultance

1. Event Driven with LibUV and
ZeroMQ
Phase I Summary
Luke.jf.luo@gmail.com

2. Outline
● Algo Trading Business Flow Review
● Order Module Design: Current Flaw
● Distributed System: Theory Review
● Event Driven System: Review
● Event Driven Design In Algo Trading
● Project Plan

3. Algo Trading Flow
I1 I2 I3 …... In
任务
( 下达 /
取消 /
停止 )
Instrument (Buy/Sell)
下单
撤单
下单
撤单
步长步长
.
.
.
...
...
….........
...
...
….........
...
...
….........
End
…............

4. Current Design Flaw
● Bad Assumptions
– Serial Execution Among Instructments
– Batch Job ( 1 Sec Interval)
– I/O Cost/ Latency: Neglectable
– Computation Cost/Latency: Neglectable
– No Uncertainty

5. Reality
● Parallel execution of Instruments
● Market change in Ms, 1 Sec (or More) fixed reponse interval
is 1000X
● I/O is SLOW comaring to CPU speed (10^9 order)
● Communication Channel will Break
● Only God have ALL TIMELY information for decision making.
We people live in UNCERTAINTY
● Tradeoff: Best Effort Decision versus Perfect Decision

6. Distributed System: Our Reality
● Parallel: Everything is Moving
● Error Prone (Fault): No 99.9999999....%
Gurantee
● Uncertainty: Who/What/Where/When/Why
● Concensus/Consistency with HIGH COST
● Speed limit: Light Travels at (300,000km/Sec)

7. CAP: No Perfect World

8. Case:Two Generals' Problem

9. Case:Two Generals' Problem
● 1975 . E. A. Akkoyunlu, K. Ekanadham, and R.
V. Huber "Some Constraints and Trade-offs in
the Design of Network Communications"
● “A pragmatic approach to dealing with the Two
Generals' Problem is to use schemes that
accept the uncertainty of the communications
channel and not attempt to eliminate it, but
rather mitigate it to an acceptable degree.”

10. TCP: Connection Establishment

11. TCP: Sliding Window

12. Event Driven: History
● 1960s: Mainframe. Batch Job Oriented
● 1970s: Unix. Interactive,Timesharing Terminal
● GUI programming:
– Xerox : mouse
– Apple: Apple II
– Microsoft: Windows
– MIT: X windows
– …...

13. Event Driven: Now
● Libuv: Chrome browser, node.js...
● Libev/libevent
● Boost ASIO
● ACE/ICE
● Qt event loop
● Glib/Gevent

14. Event Driven: Now
● Nginx,apache,Twisted,Comet
● Node.js
● Python Asyncio
● Go channel
● Scala Actor
● C++ 11 Promise/Future,Coroutine,Closure
● …........

15. I/O Modelling
●  blocking I/O
●  nonblocking I/O
●  I/O multiplexing (select and poll)
●  signal driven I/O (SIGIO)
●  asynchronous I/O (the POSIX aio_functions)
Quoted from “UNIX Network Programming Volume
1”

17. Async/Event Driven Design:
Reactor

19. Proactor

21. So we are event-driven ready.
But...
● Components are mostly event driven
● Interface Among components are BLOCKING
INTERFACE! (block/wait/timeout....)
● One blocking will ruin the whole infrastructure!
● “Public Land Tragic”

22. class StringQue : public MsgQueBase

23. StringQueue is everywhere
[luke@rmbp service_multichannel]$ find . -name StringQue.h
./orderservice/OMSInfterfaceDemo/StringQue.h
./optionhedgeservice/optionhedgeservice/StringQue.h
./queryservice/queryservice/StringQue.h
./queryservice/queryservice_auto_qt_profitchart/StringQue.h
./queryservice/queryservice_auto_qt/StringQue.h
./strategyservice_new/hedgefutureservice/StringQue.h
./strategyservice_new/algoorderservice/StringQue.h
./strategyservice_new/algoorderserviceFuture/StringQue.h
./strategyservice/hedgefutureservice/StringQue.h
./strategyservice/optionhedgeservice/optionhedgeservice/StringQue.h
./strategyservice/algoorderservice/StringQue.h
./strategyservice/algoorderserviceFuture/StringQue.h
./quoteservice/quoteservicelevel2/StringQue.h
./quoteservice/quoteservicelevel2datafeed/StringQue.h
./quoteservice/quoteservicelevel2_new/StringQue.h
./quoteservice/quoteservicelevel2datafeed_clb/StringQue.h

24. How we use StringQueue?

25. Vision
● A fault tolerant, responsive trading platorm
– Respond to every marktet tick/heartbeat
● Maintain “fresh” market snapshot
● React as every tick comes
– Async, Parallel order fullfillment
● Handle > 1000 orders one server instance without
pressure
● Milisec level processing delay per order action

26. Vision: Unified Event Loop
Libuv
event Loop
ZeroMq
socket
Normal socket
timer
Other Async events
eventfd
fd
timer_t handle
async_t handle
Parallel Tasks req_t
Thread Pool
Async Tasks
Order Fullfill
Market Snapshot
update
Other
Event Loop
Other business
modules

27. Async Programming
● Callback
● Task(threading)
● Future/Promise
● Async I/O
● Coroutine
●

28. Trading Version 2
I1 I2 I3 …... In
下单
撤单
下单
撤单
timeouttimeout
.
.
.
...
...
...
...
….........
...
...
….........
End
…............
tick
….........
EndEndEnd
tick
tick
tick

29. Project Plan
● Goal: A testable new framwork
– Algotrading as typical business test case
– High throughput, low latency
– Fault Tolerant with sensible tradeoff strategy
Side by Side comparison with current order system

30. Task:
● Project Managment (Wei Song)
● Event Loop, ZeroMQ (Tie Gang/Luke)
● Market snapshot update (Wei Song/Ze Yu)
● Order business module rewrite(Ze Yu/Wei Song)
● Testing environment (Wei Song/Tie Gang)
– Simulation data/replay
– OMS simulator
● Testing
– All
– + Calvin + other business staff

31. Schedule?
● What can we accomplish before Spring
Festival?