Message queues provide a way for applications and systems to communicate asynchronously by passing messages. They allow for decoupling of components and offloading of work. Some common uses of message queues include asynchronous processing, communication between applications/systems, auto-scaling, and handling legacy applications. Popular message queue servers include RabbitMQ, ActiveMQ, and Beanstalkd. Code examples are provided for publishing and receiving messages with each server.

1. Mike Willbanks Sr. Software Engineer at CaringBridge Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Talk: http://joind.in/1375 The Art of Message Queues ConFoo.ca 2010

2. “ Message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content.” http://en.wikipedia.org/wiki/Message_queue

3. “ Messaging describes the sending and receiving of data (in the form of messages) between systems. Messages are exchanged between programs or applications, similar to the way people communicate by email but with guarantees on delivery, speed, security and the absence of spam.” http://www.rabbitmq.com/faq.html#what-is-messaging

4. A Basic Message Queue

6. A way to decouple components.

7. A way to offload work.

9. Communication between Applications / Systems

10. Chat

11. Image Resizing

12. Video Processing

13. Sending out Emails

14. Auto-Scaling Virtual Instances

15. Log Analysis

17. You need to communicate between multiple applications / systems.

18. You have a limited number of processing slots.

19. You need to support legacy applications or use multiple programming languages.

20. Message Queue Servers

22. Scalability (Horizontal / Vertical)

23. Synchronous / Asynchronous

24. Batching Policies

25. Delivery Policies

26. Purging Policies

27. Routing Policies

28. Security Policies

29. Standards http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes

31. STOMP – Streaming Text Oriented Messaging Protocol

32. XMPP - Extensible Messaging and Presence Protocol

34. ActiveMQ – Stomp Transport

35. Beanstalkd – Non-standard based

40. Client from: http://github.com/pda/pheanstalk/

41. Some Code

42. A Simple Publish with AMQP

43. A Simple Receive with AMQP

44. A Simple Publish with Stomp

45. A Simple Receive with Stomp

46. A Simple Publish with Beanstalk

47. A Simple Receive with Beanstalk

48. How to Use Your Queues

50. JSON

53. A Simple Architecture Queue Receive Client / Publisher Message Queue Server Worker / Consumer

54. A Complex Architecture Client Message Queue Server Worker Worker Worker Worker Worker Message Queue Server Message Queue Server Client Client

57. Use Cases

60. Then allows us to focus on sending email through a single queue with multiple workers.

62. The worker or consumer then utilizes a service call or logic to boot up a new instance.

64. When a new video or photo comes in, send a message to the queue to do the work. Keep only enough workers / consumers available to not strain your machine.

65. Remember to monitor how many jobs are getting backed up :)

66. Mike Willbanks Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Talk: http://joind.in/1375 Questions?