Internally, some projects at Shopzilla have recently started to leverage the HornetQ messaging system to meet performance and scalability requirements. In this talk, Mark Lui and Josh Long review the basic principles of messaging and distributed communication. They demonstrate how loosely coupled, asynchronous communication can improve performance, scalability and reliability and finally touch on Shopzilla-specific use cases for messaging.

1. Better Living Through Messaging -
Leveraging the HornetQ Message
Broker at Shopzilla
Mark Lui ( Merchant Services)
and
Josh Long (Inventory)
- 1-

2. What is Messaging
- 2-

3. What is Messaging?
 Store and Forward
 Enables asynchronous communication
• Reduces latency for information transfer
- 3-

4. What is Messaging?
 Loosely coupled distributed communication (Data or Command)
 Exchange of messages between software components
 Two forms
• Point-to-Point (Queue)
• Publish/Subscribe (Topic)
- 4-

5. Enterprise Application Integration
 Integration of data and services between applications
 Communication between information silos
 Approaches
• File Transfer
• Shared Database
• Remote Procedure Call
• Messaging
- 5-

6. Integration Patterns
 Patterns
• Data Synchronization
• Workflow
• Portal
- 6-

7. Enterprise Application Integration
• Approaches integration using messaging
• Enables Event Driven Architecture
• Multiple users of information
- 7-

8. Message Queue Features
 Reliable
 Fast
…
 Secure
 Duplicate message detection
 Flow control – Slow/Fast consumer handling
 Wild card destinations
 Durable subscription
 Scheduled Delivery
- 8-

9. Message Queue Features
 Redelivery handling
• DLQ/Discard
• Delay redelivery
 Message Ordering on Topics
 Client side load balancing options
• Random
• Round Robin
 Message prioritization
- 9-

10. Introducing HornetQ
 New ultra high performance enterprise grade messaging system
from JBoss
 Apache V2 License
 HornetQ 2.0.GA obtained scores up to 307% higher than
previously published SPECjms2007 benchmark results
 “Throughput of 800K small non persistent messages per second
on a single 4 x 2.5GHz core server” – Tim Fox
- 10 -

11. HornetQ – Secret Sauce
 LibAIO
• Linux Kernel 2.6 Subsystem
• Able to submit I/O requests without waiting for completion
 HornetQ uses an append-only journal using LibAIO
 ActiveMQ 6 will support LibAIO 
- 11 -

12. ActiveMQ Issues
 Difficult to configuration correctly
 Prone to hang
 Poor persistence options
• KahaDB – better scalability, shorter recovery time
• AMQ Message Store – faster
• JDBC database – too slow
- 12 -

13. High Availability - About Resiliency
 Active/Passive replication
 HornetQ Example
- 13 -

14. High Availability - About Resiliency
 Live Server
- 14 -

15. High Availability - About Resiliency
 Backup Server
- 15 -

16. Clustering - About Scalability
 Client side load balancing
- 16 -

17. About performance
 Offer Bidding Service Performance Data
- 17 -

18. Spring Framework
 Connection Options
- 18 -

19. Spring Framework
 Alternative Connection Options
- 19 -

20. Spring Framework
 JMS Template
- 20 -

21. Spring Framework
 Message Driven POJOs : MessageListener
- 21 -

22. Spring Framework
 Message Driven POJOs : MessageListener
- 22 -

23. Spring Framework
 Message Driven POJOs : SessionAwareMessageListener
- 23 -

24. Spring Framework
 Message Driven POJOs : SessionAwareMessageListener
- 24 -

25. Spring Framework
 Spring Integration
- 25 -

26. Spring Framework
 Spring Integration
- 26 -

27. Messaging at Shopzilla
 Merchant Service
• Offer Bidding Service – Using to scale
 Inventory
• Offer Service
 Dampening request to Coherence
 Dampening request to Oracle for write behind
- 27 -

28. Merchant Service’s Order Bidding Service
 Demo
- 28 -

29. Inventory’s Offer Service
 Demo
- 29 -

30. HornetQ – Lessons Learned
 Use the right Connection Factory
 Use Spring 2.5.6 or 3.0
 For High Availability disable Spring reconnection logic
 Be aware of the High Availability restart sequence
 Paging is turned off by default
 JMX is your friend
- 30 -

31. Possible Future Opportunities
 SLA compliance
 Unified messaging
 Real-time, event driven architectures
- 31 -

32. Conclusion
 Q and A?
 Don't forget to tip your waiter!
- 32 -