1. Implementing distributed
applications with
...and some other bad guys...
Crippa Francesco

2. Scalability vs Complexity

3. Scalability vs Complexity
what we want
Complexity
1 Thread 2 Threads 2 Nodes 4 Nodes

4. Scalability vs Complexity
reality
Complexity
1 Thread 2 Threads 2 Nodes 4 Nodes

5. The Q in ØMQ

9. The Ø in ØMQ

10. The Ø in ØMQ
Zero Broker

11. The Ø in ØMQ
Zero Broker
Zero Latency (as close as possible...)

12. The Ø in ØMQ
Zero Broker
Zero Latency (as close as possible...)
Zero administration

13. The Ø in ØMQ
Zero Broker
Zero Latency (as close as possible...)
Zero administration
Zero cost

14. The Ø in ØMQ
Zero Broker
Zero Latency (as close as possible...)
Zero administration
Zero cost
Zero waste

15. Sockets

16. Sockets
Unicast transports (inproc, ipc, tcp)

17. Sockets
Unicast transports (inproc, ipc, tcp)
Multicast transports (pgm or epgm)

18. Sockets
Unicast transports (inproc, ipc, tcp)
Multicast transports (pgm or epgm)
connect() and bind() are independent

19. Sockets
Unicast transports (inproc, ipc, tcp)
Multicast transports (pgm or epgm)
connect() and bind() are independent
They are asynchronous (with queues)

20. Sockets
Unicast transports (inproc, ipc, tcp)
Multicast transports (pgm or epgm)
connect() and bind() are independent
They are asynchronous (with queues)
They express a certain "messaging pattern"

21. Sockets
Unicast transports (inproc, ipc, tcp)
Multicast transports (pgm or epgm)
connect() and bind() are independent
They are asynchronous (with queues)
They express a certain "messaging pattern"
They are not necessarily one-to-one

22. ...and, of course...
Cross Platform (Linux, Windows, Mac, etc...)
Multiple Languages (c, c++, python, java,
ruby, erlang, php, perl, ada, c#, lua, scala,
objective-c, go, haskell, racket, cl, basic...)
OpenSource

23. if you have a laptop...
http://www.zeromq.org/
http://zguide.zeromq.org/

24. Basic Message Patterns

26. Server

27. Client

28. Demo

29. Basic Message Patterns

31. Server

32. Client

33. Demo

34. Publisher
Subscriber
The PUB-SUB socket pair is asynchronous
when you use a SUB socket you must set a
subscription using zmq_setsockopt and
SUBSCRIBE
"slow joiner" symptom

35. Basic Message Patterns

38. Ventilator

39. Worker

40. Sink

41. Demo

42. Workers
Always synchronize the start of the batch
The ventilator's PUSH socket distributes
tasks to workers (load balancing)
The sink's PULL socket collects results from
workers evenly (fair-queuing)

43. Fair-Queuing

44. Basic Message Patterns

45. Send
messages
back

48. Demo

49. Allowed Patterns
PUB and SUB DEALER and DEALER
REQ and REP ROUTER and ROUTER
REQ and ROUTER PUSH and PULL
DEALER and REP PAIR and PAIR
DEALER and ROUTER

50. Scalability

51. Scalability

52. Scalability

53. Scalability

54. A Publish-Subscribe Proxy

56. Built-in Devices
QUEUE (request-reply broker.)
FORWARDER (pub-sub proxy server)
STREAMER (like FORWARDER but for
pipeline flows)

58. Dynamic Scalability...

59. Dynamic Scalability...
SCA LOW
LAB
ILI
!!!! TY

60. Dynamic Scalability...

61. ØMQ : Sockets = Python : C++