1. JAVA EE CONCURRENCY MISCONCEPTIONS
Haim Yadid, Performize-IT Ltd.
Daniel Pfeifer, Whistler AB
2. Daniel Pfeifer
‣ Founder of
Performize-IT Ltd.
‣ 18+ years in
technology
‣ Primary expertise in
performance optimization
‣ Founder of
Whistler AB
‣ 10+ years in IT
‣ Primary expertise in missioncritical business applications
WHO ARE THOSE GUYS?
Haim Yadid
12. You found yourself in need of sharing some data between calls so you
‣ Create a static constant pointing to a List (or perhaps your own
singleton)
‣ Add a synchronization block for your constant
‣ You operate on your instance inside the synchronize-block
SYNCHRONIZATION IN ENTERPRISE BEANS
Use Case
13. @Stateless
public class SharedStateEJB {
private static final List usedNumbers = new ArrayList();
public double fetchUnusedRandomNumber() {
double number = 0d;
synchronized (usedNumbers) {
while (true) {
number = Math.random();
if (!usedNumbers.contains(number)) {
usedNumbers.add(number);
break;
}
}
}
return number;
}
}
SYNCHRONIZATION IN ENTERPRISE BEANS
Returning unused random numbers
14. Sharing state like this will not work, because …
‣ Static constants are not shared among class loaders
‣ A lock will not is not propagated across machines
SYNCHRONIZATION IN ENTERPRISE BEANS
Problems
16. @Singleton
public class UsedNumbersBean {
private final List usedNumbers = new ArrayList();
@Lock(READ)
public boolean numberExists(double num) {
return usedNumbers.contains(num);
}
@Lock(WRITE)
public void addNumber(double num) {
usedNumbers.add(num);
}
}
SYNCHRONIZATION IN ENTERPRISE BEANS
@Singleton
17. “Backup” solutions
‣ Use the database and use locking => Potentially slow
‣ Use a cluster-aware cache (i.e. EHCache or JBoss Cache)
‣ Read the chapters on clusters and consistency!
‣ Terracotta
“Creative” solutions
‣ Use a single EJB
‣ Use H/A Singleton (available on JBoss)
‣ Wait for the future…
SYNCHRONIZATION IN ENTERPRISE BEANS
Suggestions
19. • You processed an order
• You ask the bank for settlement confirmation
• You can present the customer with a result without waiting for the
bank
ASYNCHRONICITY THROUGH THREADS
Use Case
20. @Stateless
public class OrderHandlingBean {
public String process(final Order order) {
String orderId = ...//save order to database and get id
final Thread t = new Thread() {
@Override public void run() {
// submit amount to bank
}
};
t.start(); // start “background” process
return orderId;
}
}
ASYNCHRONICITY THROUGH THREADS
Creating a thread for async tasks
21. ‣ Enough stuck Threads will eventually make your server go OOM.
‣ Difficult to get transaction management right.
‣ Some servers will tell you it can’t check the status, some don’t.
‣ Failed tasks in the thread need difficult manual rollback.
‣ Threads do not scale out
‣ Can’t be load-balanced to other machines.
‣ Server’s automatic resource management doesn’t cover your own
threads.
‣ Debugging and monitoring isn’t the joy it used to be.
ASYNCHRONICITY THROUGH THREADS
Problems
23. ‣ Until Java EE 6 Message Driven Beans are used for asynchronous
tasks.
‣ And if you are lucky enough to use a Java EE 6 container, you can
use the new @Asynchronous.
ASYNCHRONICITY THROUGH THREADS
Best Practice Alternatives
24. @Stateless(name = ”QueueingEJB")
public class QueueingBean {
@Resource private Queue q;
@Resource private ConnectionFactory cf;
public String process(final Order order) {
// ... saving the Order to the database and fetching id
Connection cn = connectionFactory.createConnection();
Session s = cn.createSession(true, AUTO_ACKNOWLEDGE);
MessageProducer producer = s.createProducer(queue);
Message msg = s.createTextMessage(payment);
producer.send(msg);
return orderId;
}
}
Don’t forget clean-up!
We just want to
conserve screen estate!
ASYNCHRONICITY THROUGH THREADS
Using Queues
25. @MessageDriven(name = “PayEJB”,
activationProperties = {
/* activation properties for your server */
})
public class PayBean implements MessageListener {
public void onMessage(Message m) {
// Calling bank...
}
}
ASYNCHRONICITY THROUGH THREADS
Using Queues
26. @Stateless(name = ”PayEJB")
public class PayBean{
@Asynchronous
public void callBank(final String payment) {
// Hello, ... Bank!
}
}
ASYNCHRONICITY THROUGH THREADS
Using @Asynchronous
27. @Stateless(name = ”AsyncCallerEJB")
public class AsyncCallerBean {
@EJB private PayBean payBean;
public String process(final Order order) {
// ... saving the Order to the database and fetching id
payBean.callBank(payment);
return orderId; // <- Returns + transaction OK
}
}
That’s All!!!
ASYNCHRONICITY THROUGH THREADS
Using @Asynchronous
28. We mentioned transactions…
‣ True, our examples won’t use the same Transaction, but…
‣ @Asynchronous will by spec create a Transaction
(REQUIRES_NEW)
‣ MDBs will also create a transaction
ASYNCHRONICITY THROUGH THREADS
A note for the observant
30. Your back-office staff fires off a bunch of invoices and you want the
total as fast as possible, so you…
‣ Create a thread-pool
‣ Push all invoices for calculation to the thread-pool
‣ Wait for the answer and return it
PARALLELISM WITH THREADPOOLS
Use Case
31. private static class OrderCallable implements Callable<Double> {
private final Order order;
public OrderCallable(Order order) {
this.order = order;
}
@Override
public Double call() throws Exception {
// return the total based on the order that’s been
// provided.
}
}
PARALLELISM WITH THREADPOOLS
Parallelizing with a ThreadPool (our Callable)
32. public double processOrders(List<Order> orders) {
ExecutorService executorService =
Executors.newFixedThreadPool(10);
List<Future<Double>> futures = new ArrayList<>();
for (Order order : orders) {
OrderCallable callable = new OrderCallable (order);
callables.add(callable);
futures.add(executorService.invoke (callable));
}
double totalValue = 0;
try {
for (Future<Double> future : futures) {
totalValue += future.get(10, SECONDS);
}
} catch (TimeoutException e) {
return -1;
}
return totalValue;
}
PARALLELISM WITH THREADPOOLS
Parallelizing with a ThreadPool (using the Callable in an ExecutorService)
33. Essentially we got the same problems as previously, but…
‣ Potential for much larger thread leaks
‣ Still not covered by transaction
‣ Executors, just like Threads, won’t run tasks on other machines.
‣ There is nothing H/A about this.
‣ On crash, part or all of the workload is gone.
‣ Can’t be resumed on server restart, it’s just gone…
PARALLELISM WITH THREADPOOLS
Problems
34. ‣ Up to and including Java EE 5 you can use MDBs
‣ Queues can be clustered, so other servers can take some of the
load.
‣ @Asynchronous-annotated method returning Future
PARALLELISM WITH THREADPOOLS
Pure Java EE alternatives
42. ‣ New to Java 7 java.util.concurrent
‣ Helps you write a parallel recursive algorithm
‣ Break a long task to Fine grained tasks
‣ Very efficient for relatively short tasks
FORK JOIN
Introduction
44. class PartialTask extends RecursiveAction {
protected void compute() {
PartialTask pt1 = new PartialTask(firstHalf);
PartialTask pt2 = new PartialTask(secondHalf);
pt1.fork(); // fork
pt2.exec(); // execute in new thread
pt1.join(); // wait for completion
}
}
fjPool = new ForkJoinPool(para);
Para.invoke(task)
FORK JOIN
Example
45. ‣ Divide n points into k clusters
‣ Based on proximity
FORK JOIN
A Benchmark K-Means Clustering Algorithm
46. ‣ This is not a formal benchmark
‣ I tried to do my best but it is not community criticized
‣ Ran on 8 core machine (no HT)
‣ Windows OS
‣ Warm-up was taken into account J
‣ 120K points to 50 clusters
FORK JOIN
Disclaimer
47. FORK JOIN
Response Time – by amount of parallelism
6000"
5000"
Time%(sec)%
4000"
3000"
Time"
2000"
1000"
0"
1"
2"
3"
4"
5"
#cores%used%
6"
7"
8"
50. ‣ Does your server need to do anything else ?
‣ Selfish Task Deplete all compute resources slows other task
‣ What about throughput
FORK JOIN
FJ on a JEE Server
57. You have a backend machine that can only handle 20 concurrent
requests and you have a cluster of four servers, so you…
‣ Create a semaphore with five permits.
‣ Acquire lock on semaphore
‣ Call server
‣ Release lock
LOAD MANAGEMENT FOR BACKEND ACCESS
Use Case
58. ‣ The backend server can be under-utilized across the cluster.
‣ First server could be full of locks while second is free.
‣ One server can stall unnecessarily long time.
‣ Recompile necessary if some parameter changes.
LOAD MANAGEMENT FOR BACKEND ACCESS
Problems With This Approach
59. Using the Request-Reply Pattern (EIP), using:
‣ Clustered queues (request + response)
‣ A caller (stateless session bean)
‣ A backend invoker bean (message-driven bean)
‣ Configuration to limit in-process messages
For the advanced:
‣ Write a resource adapter (JCA)
‣ Support load balancing
‣ Support cluster
LOAD MANAGEMENT FOR BACKEND ACCESS
A Better Approach
60. @Stateless public class DispatcherBean {
@Resource(mappedName = “ConnectionFactory”) private ConnectionFactory cf;
@Resource(mappedName = “RequestQ”)
private Destination sendQueue;
@Resource(mappedName = “ResponseQ”)
private Destination responseQueue;
@TransactionAttribute(NOT_SUPPORTED) // Alternatively UserTransaction
public double checkStockRate(String stockName) {
try {
//… create producer
Message outgoing = s.createMessage();
outgoing.setStringProperty(”StockName", stockName);
mp.send(outgoing);
Send
String correlationId = outgoing.getJMSMessageID();
cn.start(); // must be started, otherwise we can't receive
MessageConsumer mc = s.createConsumer(responseQueue,
"JMSCorrelationID = '" + correlationId + "'");
Message incoming = mc.receive(10000L); // wait no more than 10 seconds
if (incoming != null) {
return incoming.getDoubleProperty(”StockPrice");
} else {
return Double.MIN_VALUE;
}
} catch (JMSException e) {
throw new EJBException(e);
}
}
}
Receive
LOAD MANAGEMENT FOR BACKEND ACCESS
Request-Reply Pattern using clustered queues (Caller SLSB)
61. @MessageDriven(mappedName = “RequestQ”)
public class BackendInvokerBean implements MessageListener {
@Resource(mappedName = “ConnectionFactory")
private ConnectionFactory cf;
@Resource(mappedName = “ResponseQ")
private Queue responseQueue;
public void onMessage(Message message) {
double stockPrice = // ... Fetch from backend service
try {
// … create producer
Message msg = s.createMessage();
msg.setJMSCorrelationID(message.getJMSMessageID());
msg.setDoubleProperty(”StockPrice”, stockPrice);
mp.send(msg);
} catch (JMSException e) {
throw new EJBException(e);
}
}
}
LOAD MANAGEMENT FOR BACKEND ACCESS
Request-Reply Pattern using clustered queues (Invoker MDB)
62. Vendor-specific, but in general:
‣ You may set an upper limit on consumers
‣ Glassfish has maxNumActiveConsumers
‣ You may set an upper limit of MDBs in Pool
‣ Works on all servers regardless of vendor (i.e. set max-poolsize to 5 in glassfish-ejb-jar.xml)
LOAD MANAGEMENT FOR BACKEND ACCESS
Configuration for in-process limitation
66. You occasionally suffer from resource depletion so to protect your
server you:
‣ Create an EJB Interceptor that
‣ checks current beans in use (i.e. through counter or MBeans)
‣ throws an exception at chosen maximum size.
‣ Or wait until level go down
‣ Assign it to your EJB.
INTERFERING WITH BEAN POOL
Use Case
67. ‣ Is there always equal load distribution across all tasks?
‣ Intelligent load balancers may continue to shoot at the wrong target
‣ It’s just an exception… not an overload
‣ DeadLock
INTERFERING WITH BEAN POOL
Problems
68. Leave pool management to the server, pool sizes:
‣ Can be tuned on a global level
‣ Right strategy (infinite thread pools, incremental, fixed)
‣ Global default for min/max/timeout
‣ Can be set for each EJB
‣ XML
‣ Annotations (i.e. JBoss)
INTERFERING WITH BEAN POOL
Better approach
69. It can be configured through the GUI…(Glassfish example)
INTERFERING WITH BEAN POOL
Leave Management to the app server
72. CONCLUSION
‣ Most mistakes stem from misunderstanding the app server, try to
understand your app server’s OOTB-capabilities…
‣ …and the Java EE specification.
‣ If you have a valid case for your own concurrency-related code,
make sure it plays well with the container.
‣ Your app server doesn’t always know best, but it often does. And if
not, you can often help it understand!
‣ Do not expect your environment to always look the same, it changes
quicker than you might think.