1. Asynchronous Service Servers Based on AMQP for Ruby

2. Versapay Point of sales eCommerce payment processing Partner with JPChase No.1 Growth Company in Canada (Profit50)

3. Prelude: ConcurrencyOr, ouch, it hurts…

4. Nietzsche on Concurrency …if you gaze for long into an abyss, the abyss gazes also into you.

5. Spectrum of Grainedness Parallelism The Multicore Future™ Code structuring (e.g. block/wait) Concurrency Simultaneous tasks Distributed Services Internet/REST REST (all its constraints: stateless, layered, cacheable, uniform interace, …)

6. Lazy Scalability Worry Later? Very hard to fix a wrong architecture Get it right, right now! Architecture (constraints) makes scaling possible Actual scaling when you need it Path from concurrency to SOA Actor Model (e.g. Erlang) Key-value store (no join)

7. Actor Model The Actor model is characterized by inherent concurrency of computation within and among actors, dynamic creation of actors, inclusion of actor addresses in messages, and interaction only through direct asynchronous message passing with no restriction on message arrival order. N.B. From wikipedia

8. Actor Model Identified by address Mobile (doesn’t matter which machine) Transient (doesn’t matter which process) Decouples the “who” of computation Message passing Ask, don’t tell Decouples the “what” of computation Asynchrony Event-based Decouples the “when” of computation

9. Erlang-Style Concurrency Creating and destroying processes is very fast. Sending messages between processes is very fast. Processess behave the same way on all operating systems. We can have very large numbers of processes. Processes share no memory and are completely independent. The only way for processes to interact is through message passing. Joe Armstrong

10. More on Erlang Message passing is asynchronous. Processes can monitor each other. It is possible to selectively receive messages. Remote processes appear largely the same as local processes. Ulf Wiger

11. ASS kicking with RabbitMQ Stealing Ideas from Erlang

12. AMQP Just a protocol specification Open, interoperable Binary Extensible Dynamic creation of entities For enterprise message middleware RabbitMQ is a free implementation Qpid is an Apache project

13. AMQP(<1.0) Details AMQP Entities Exchange (publish messages here) Queue (route messages here) Binding (establishes routing between exchange and queue) AMQP Error Model The sound of success is silence An error just closes connection QOS, prefetch window

14. ASS Inspired by Erlanggen_server Hide concurrency behind generic server Parameterize generic server with modules Familiar sequential programming Pass around Ruby objects by marshalling Switch to JSON? Full access to AMQP if you need it

15. A Simple ASS Actor require 'rubygems' require 'ass' ASS.start do ASS.actor("demo") { def echo(o) o end } end

16. Demo Server/Client Load distribution Store and forward Zero downtime Future & RPC Wait in batches Failure recovery with :ack => true

17. RPC is an Illusion ASS is fundamentally asynchronous Async/event-based programming is unfamiliar RPC client does blocking wait to simulate sequential programming “Future” gives some asynchrony back Timeout as generic exception Usually just give up and die

18. RPC is EVIL when… Hides too much Random failures Doesn’t hide enough Exposing the implementation Leaky abstraction See: http://steve.vinoski.net/blog/2008/07/01/convenience-over-correctness/

19. Experiences So Far

20. Programming Murphy’s Computer Anything can fail at any time… Think in critical sections Retry is the bedrock of reliability :ack => true FSM is a good mental aid Mark progress with a database (ACID state) It’s ok to die… process is transient, data is forever

21. API Message verification Pattern Matching http://github.com/hayeah/matchmaker Fail Fast and Hard at the API Avoid error spreading through the services Share the API spec among services Weak API

22. Processes, Not Threads Avoid threads, avoid horrors The stack is opaque, you don’t know if anything you use might cause blocking Let OS and DB do the hard work Process isolation Anything wrong? Easy, kill it! IPC cost, but you don’t care (most of the time)

23. Testing Concurrent App No established practice… Test the real thing, not mocks Pretty easy with ASS, hard with web-services Test in a single process, deploy to multiple processes, then hosts, then networks Invariants are super important Some kind of property testing Must test with enough load

24. Future Work Like, TODO, right now!

25. System Health Monitoring Event based Metrics Logging Self-management Supervisor hierarchy Zookeeper?

26. Better Testing Stress Test Testing with generated message sequences Some kind of proxy to force message order Some kind of code injection technique System invariant checker Test failures (connection dying, db dying, etc) Continuous integration

27. What Datastore? We are using ActiveRecord at the moment Because it’s there… Some kind of schemaless key-value store Like Friendsfeed

28. Conclusion

29. Three Legged Stool Scalability gives, Distributability gives, Reliability gives scalability http://github.com/hayeah/ASS ZERO documentation Quirks It’s not Rails