1. Beyond Horizontal Scalability:
Concurrency & Messaging Using Spring
Bruce Snyder, Senior Software Engineer, SpringSource/VMware
Friday, July 8, 2011

2. Problem Statement
• Software systems are growing larger than ever before
– Virtual Machines
– Physical Machines
– More CPUs
– Multi-core CPUs
– More requests
– More data
• With so many hardware changes, shouldn’t our software
practices change too?
Friday, July 8, 2011

3. Typical Application Layers
3
Friday, July 8, 2011

4. Typical Code Construction
• Command-and-control scheme
– One method calls another
– Blocking calls
• Assumptions
– One task takes place at a time
– The order of operations in known
– The provider of a particular function is known
– All execution happens in a single JVM
4
Friday, July 8, 2011

5. Typical Component Interaction
• Single memory space
• Using the call stack
• Sequential execution
• Assumptions
– No separation between:
• Knowing what needs to happen next
• Knowing which method to invoke
– Interactions are always known at compile time
5
Friday, July 8, 2011

6. Typical Assumptions
• All work must take place in a single call stack
• Example:
– Receive request
– Verify data
– Save data
– Query data
– Generate PDF
– Send email
– Render response
6
Friday, July 8, 2011

7. • The higher the number of assumptions, the more tightly
coupled the system
7
Friday, July 8, 2011

8. Typical Client Request
8
Friday, July 8, 2011

9. Typical Request Growth
9
Friday, July 8, 2011

10. Typical Strategy For Horizontal Scaling
10
Friday, July 8, 2011

11. Typical Assumptions
• All work must take place on a single machine
• Example:
– Receive request
– Verify data
– Save data
– Query data
– Generate PDF
– Send email
– Render response
11
Friday, July 8, 2011

12. Treating the Symptoms
• Horizontal scale is good to a point
– Treat the symptoms instead of fixing the cause of the illness
• Throughput of functions should not equal response time!
• This is not the case at forward-thinking companies
– Amazon.com
12
Friday, July 8, 2011

13. The Problem With Distributed Systems
• Most developers don’t understand them
– Call stack provides specific interaction style
– Most object-oriented systems focus on structure vs.
interaction
• Interaction becomes more important than structure
– Mediator pattern - encapsulates how objects interact
– Observer pattern - notifies dependent objects of state change
• The real point of service-orientated design
– Composition rules are rather simple
– Loosely coupled interaction becomes very important
13
Friday, July 8, 2011

14. Is There a Better Solution?
• Insert a level of indirection
– Remove direct interaction between components
– Extract the interaction into a separate element
– Enterprise Integration Patterns (EIP)
• http://enterpriseintegrationpatterns.com/
• Simplify the rules of component interaction
– Remove the coordination and continuation style of interaction
– Just send data
14
Friday, July 8, 2011

15. Concurrency
• Concurrency is a style of asynchronous execution
15
Friday, July 8, 2011

16. Concurrency With Spring
• Spring provides the TaskExecutor interface
– Abstracts the execution of a Runnable, i.e., asynchronous work
– SimpleAsyncTaskExecutor
• Creates a new thread for each task
– ThreadPoolTaskExecutor
• Uses a JDK 1.5 ThreadPoolExecutor to create a pool of threads
– org.springframework.scheduling.concurrent package
• Many scheduling related utility classes
– WorkManagerTaskExecutor
• Delegates to a JCA 1.5 CommonJ WorkManager
16
Friday, July 8, 2011

17. JDK ExecutorService
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(task);
executorService.shutdown();
executorService = Executors.newFixedThreadPool(10);
executorService.execute(task);
executorService.shutdown();
int corePoolSize = 5;
int maxPoolSize = 10;
long keepAliveTime = 5000;
executorService = new ThreadPoolExecutor(corePoolSize, maxPoolSize,
keepAliveTime, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
executorService.execute(task);
executorService.shutdown();
17
Friday, July 8, 2011

18. Spring TaskExecutor
<bean id="myTask" class="org.bsnyder.spring.concurrency.MyTask" />
<bean id="springConcurrencyExample"
class="org.bsnyder.spring.concurrency.SpringConcurrencyExample" />
<bean id="simpleAsyncTaskExecutor"
class="org.springframework.core.task.SimpleAsyncTaskExecutor"
p:daemon="false" />
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"
p:corePoolSize="5" p:maxPoolSize="10" p:queueCapacity="30" />
18
Friday, July 8, 2011

19. Spring TaskExecutor
public class SpringConcurrencyExample {
! @Autowired
! private Runnable task;
! @Autowired
! private SimpleAsyncTaskExecutor simpleAsyncTaskExecutor;
! @Autowired
! private ThreadPoolTaskExecutor threadPoolTaskExecutor;
!
! public void runExamples() {
! ! simpleAsyncTaskExecutor.execute(task);
! ! threadPoolTaskExecutor.execute(task);
! }
!
! public static void main(String[] args) {
! ! ApplicationContext context =
! ! ! new ClassPathXmlApplicationContext("/META-INF/spring/executor-context.xml",
SpringConcurrencyExample.class);
! ! SpringConcurrencyExample example =
context.getBean(SpringConcurrencyExample.class);
! ! example.runExamples();
! }
}
19
Friday, July 8, 2011

20. Messaging
• Messaging is a style of communication
• Often used for integration purposes
20
Friday, July 8, 2011

21. Messaging With Spring
• JMS Support
– JmsTemplate for sync send and receive
– DefaultMessageListenerContainer and
SimpleMessageListenerContainer for async receive
• javax.jms.MessageListener
• org.springframework.jms.listener.SessionAwareMessageListener
• org.springframework.jms.listener.adapter.MessageListenerAdapter
• AMQP Support
– RabbitTemplate for sync send and receive
– SimpleMessageListenerContainer for async receive
• org.springframework.amqp.core.MessageListener
• org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter
21
Friday, July 8, 2011

22. Spring JmsTemplate
Synchronous
<bean id="connectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory"
p:brokerURL="tcp://localhost:61616" />
<bean id="destination" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="FOO.TEST" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate"
p:connectionFactory-ref="connectionFactory"
p:defaultDestination-ref="destination" />
<bean id="messageProducer"
class="org.bsnyder.spring.jms.producer.SimpleMessageProducer"
p:jmsTemplate-ref="jmsTemplate" />
22
Friday, July 8, 2011

23. Spring JmsTemplate
Synchronous
// Use the default destination
jmsTemplate.convertAndSend("Hello World!");
// Use a different destination
jmsTemplate.convertAndSend(“TEST.BAR”, “Hello World!”);
// Use a different destination
String textMessage1 = (String) jmsTemplate.receiveAndConvert();
// Use a different destination
String textMessage2 = (String) jmsTemplate.receiveAndConvert(“TEST.BAR”);
23
Friday, July 8, 2011

24. DefaultMessageListenerContainer
Asynchronous
• XML configuration
<bean id="connectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory"
p:brokerURL="tcp://localhost:61616" />
<bean id="messageListener"
class="org.bsnyder.spring.jms.listener.SimpleMessageListener" />
<jms:listener-container concurrency="5-10">
<jms:listener destination="FOO.TEST" ref="messageListener"/>
</jms:listener-container>
24
Friday, July 8, 2011

25. Spring RabbitTemplate
Synchronous
<bean id="rabbitAdmin"
class="org.springframework.amqp.rabbit.core.RabbitAdmin">
<constructor-arg ref="connectionFactory"/>
</bean>
<bean id="connectionFactory"
class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory"
p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/>
<bean id="rabbitTemplate"
class="org.springframework.amqp.rabbit.core.RabbitTemplate"
p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO"
p:queue="TEST.FOO" />
<bean id="messageProducer"
class="org.bsnyder.spring.amqp.producer.SimpleMessageProducer"
p:rabbitAdmin-ref="rabbitAdmin" p:rabbitTemplate-ref="rabbitTemplate"/>
25
Friday, July 8, 2011

26. Spring RabbitTemplate
Synchronous
// Use the default destination
rabbitTemplate.convertAndSend("Hello World!");
// Use a different destination
rabbitTemplate.send(“TEST.FOO”, “TEST.FOO”, message);
// Use a different destination
Message message1 = rabbitTemplate.receiveAndConvert();
// Use a different destination
String textMessage2 = (String) rabbitTemplate.receive(“TEST.FOO”);
26
Friday, July 8, 2011

27. SimpleMessageListenerContainer
Asynchronous
<bean id="connectionFactory"
class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory"
p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/>
<bean id="rabbitAdmin"
class="org.springframework.amqp.rabbit.core.RabbitAdmin">
<constructor-arg ref="connectionFactory"/>
</bean>
<bean id="rabbitTemplate"
class="org.springframework.amqp.rabbit.core.RabbitTemplate"
p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO"
p:queue="TEST.FOO" />
<bean id="messageListener" class="org.bsnyder.spring.amqp.listener.SimpleMessageListener"
p:rabbitTemplate-ref="rabbitTemplate"/>
<bean id="listenerContainer"
class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer"
p:connectionFactory-ref="connectionFactory" p:queueName="TEST.FOO"
p:concurrentConsumers="5" p:listener="messageListener" />
27
Friday, July 8, 2011

28. Best of Both Worlds
• What if concurrency and messaging was provided
together?
28
Friday, July 8, 2011

29. Spring Integration
• A framework for integration
• Styles of integration
– Intra-application integration
– Inter-application integration
– External system integration
29
Friday, July 8, 2011

30. Enterprise Integration Patterns (EIP)
http://enterpriseintegrationpatterns.com/
30
Friday, July 8, 2011

31. Spring Integration
• Provides both concurrency and messaging
– Message Endpoints
• Connections between services
– Channel Adapters
• Adapter between application and message broker
– Messaging Gateways
• Provides uni-directional or bi-directional messaging
– Service Activators
• Invokes a services based on an incoming message
– Routers
• Determines where to dispatch a message
– Splitters and Aggregators
• Breaks up a message and reassembles it after processing
31
Friday, July 8, 2011

32. Spring Integration
• Supports
– AMQP
– Email
– File system
– Gemfire
– JMS
– JMX
– MongoDB
– Redis
– Spring Batch
– Testing
– Web Services
32
Friday, July 8, 2011

33. Spring Integration
33
Friday, July 8, 2011

34. Spring Integration
34
Friday, July 8, 2011

35. Spring Integration
35
Friday, July 8, 2011

36. Spring Integration
36
Friday, July 8, 2011

37. Spring Integration
37
Friday, July 8, 2011

38. Spring Integration
38
Friday, July 8, 2011

39. Spring Integration
39
Friday, July 8, 2011

40. Spring Integration
40
Friday, July 8, 2011

41. Spring Integration
41
Friday, July 8, 2011

42. Spring Integration
42
Friday, July 8, 2011

43. Your Coffee Shop Does Not Use 2PC
• Order is accepted
• Cup is labeled and placed in the queue
• Money is exchanged
• Coffee drink is processed
• Multiple baristas = competing consumers
• Drinks are processed out of order = correlation id (label)
• Cannot pay for drink = discard (write-off)
• Erroneous drink = retry
• Drink machine fails = compensating action (refund, etc.)
43
Friday, July 8, 2011

44. Conversation Pattern
• Interaction between two parties
– Short synchronous interaction
– Longer asynchronous interaction
• Other examples
– Amazon.com
44
Friday, July 8, 2011

45. Cafe Demo
45
Friday, July 8, 2011

46. Thank You!
Q&A
Friday, July 8, 2011