Boost Fertility New Invention Ups Success Rates.pdf
Message Queues in Ruby - An Overview
1. Message Queueing in Ruby
An Overview
Pradeep Elankumaran
@pradeep24
pradeep@intridea.com
2. About Us
❖ Intridea is a high-end web consulting shop
❖ Based out of Washington DC, but dispersed all over the United States
❖ Our most popular product is Presently, an enterprise microblogging
platform
❖ Grub.it is our latest product - mobile dish reviewing
❖ Various iPhone & Android apps (Car Finder, Grub.it)
❖ Specialize in web & mobile development using Ruby, Rails, Java, Erlang
and beyond
4. A request is supposed to
return a response really fast.
5. However, there are certain
actions that may take longer
than a few milliseconds
6. A Few Examples
❖ Delivering e-mail
❖ Image processing (resizing, cropping etc.)
❖ Sending out notifications
❖ Geocoding a location
❖ Talking to other web services
7. These actions, if not
extracted out to run
asynchronously, will slow
down your app
21. A Few Notes about DJ
❖ used by Github exhaustively till they moved to Resque
❖ will get you 80% of the way there, has some nice advanced
features (priority, retries etc.)
❖ should be careful not to overload your jobs table
❖ backed up jobs can lead to query slowdown on your site
❖ suggested that you monitor and profile workers before you
launch on production
30. A Few Notes about Starling
❖ persists to RAILS_ROOT/tmp/starling
❖ great for smaller apps, very lightweight
❖ again, will get you 80% of the way there
❖ writing robust, reliable workers is left up to you
❖ polling makes it slow compared to other queues
37. ActiveMessaging plugin
❖ developed by ThoughtWorks
❖ provides a mini-framework to publish and subscribe to
various ActiveMQ queues and topics
❖ mature, well-tested
42. A Few Notes about ActiveMQ
❖ enterprise-grade
❖ can be clustered (edit XML files)
❖ using ActiveMessaging, no need to write your own workers
❖ STOMP gem is great if you do want to write your own
workers
47. Careful with the libraries....
❖ @tmm1’s amqp gem - runs off EventMachine, doesn’t play
well with Passenger (asynchronous)
❖ warren - the best of the bunch for Rails (synchronous)
❖ bunny - lightweight (synchronous)
49. A Few Notes about
RabbitMQ
❖ enterprise-grade
❖ can be clustered using Erlang’s built-in awesome clustering
❖ will have to write your own workers unless you use Workling
❖ very, very fast
52. Redis
superfast data structures store
like memcached, but smarter
in-memory for the most part
persisted to disk asynchronously
sets, lists & corresponding operations
64. ... but writing Ruby workers
that are long-running and
reliable is NOT easy
unless you’re using EventMachine or
really know what you’re doing
65. Why so hard to write Ruby workers?
❖ You will wake up to workers with memory leaks
❖ Workers sometimes lock up
❖ Loading the whole Rails stack for each worker
❖ In general, a pain to manage these worker daemons
66. Use EventMachine
Reactor Pattern
Define callbacks on events
pretty solid & fast
67. fork( )
spin up the worker in the new process
close when done
can kill bothersome workers
easier debugging
71. Llama
❖ A port of Apache Camel to Ruby
❖ based off EventMachine, so it runs ROCK SOLID
❖ early alpha at this point, looking for helpers
❖ github.com/skyfallsin/llama