1. # 1
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Diagnostic & Audit system for Java EE
applications
Secure your Java EE project with the performance diagnostic tool
provided by OW2 JOnAS
Florent Benoit, BULL/OW2 [ @florentbenoit ]

2. # 2
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Summary
● Context
● Environment : OW2 Java EE JOnAS Application server
● Diagnostic tool
● Presentation
● Demo
● Audit tool
● Presentation
● Demo
● Conclusion

3. # 3
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Context

4. # 4
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Why these tools ?
● Java EE specification:
● Ensure portability of applications
● Nothing about performance
● Application performance / Reliability ?
● Applications can be Java EE compliant without being reliable
● Finding performance problems ?
● Not so easy to find the problem with all components that are
linked together.
● Traceability
● Get a log for each executed operation
● «Cost» of services
● For example, to know the memory used for a request

5. # 5
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Environment : OW2 Java EE JOnAS
Application server

6. # 6
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
JOnAS: Java EE Application server
● Java EE 5 certified
● Java EE services:
● Web Container: Tomcat (6 & 7) / Jetty
● EJB3 persistence / JPA 1 & 2: EasyBeans (EclipseLink,
Hibernate, OpenJPA)
● Transactions: JOTM
● Clustering: CMI
● Web Services: CXF/Axis2
● Asynchronous Messages: JORAM
● OSGi: Felix et IPOJO
● Administration: web console, commands, API, JASMINe
(Advanced management tool)

7. # 7
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
JOnAS : Open Source Server
● Developed as an open source server (LGPL) within
OW2: http://jonas.ow2.org
● OW2: independent industry consortium dedicated to
developing open source code middleware
● Major contributors for JOnAS :Bull, France Telecom,
Peking University, INRIA, UJF, UNIFOR, SERLI
● Linked OW2 projects : EasyBeans, JASMINe, JORAM,
JOTM, CMI

8. # 8
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
OSGi native Architecture
● Dynamically adaptable
platform
● OSGi based services
● Modularity / Extensibility
● Profiles
● Enhanced application server
life cycle
● On-Demand services
● Dynamic configuration
● Adaptable

9. # 9
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Diagnostic tool

10. # 10
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Diagnostic tool
JDBC Connection leak detector

11. # 11
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
« Pool » of JDBC connections
● Limit the number of physical connections to the database
● Optimize the time to provide a JDBC connection to the
application
datasource.getConnection();
connection.createStatement();
....
....
connection.close();
DataSource Pool

12. # 12
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Forgot to call connection.close() ?
● Problem :
No more available connections for new clients
● → Connections never closed
– → don't go back in the pool
● → Other clients are waiting
– No free connections in the pool !
Busy connections (used by
applications) or not yet closed
Empty PoolDataSource Pool

13. # 13
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Handling the connection leak ?
● Avoid these connection leaks in production ?
● Automatic close of JDBC Connections by JOnAS
– At the end of a method call (EJB stateless / HTTP request),
remove() on stateful EJB beans.
● Life-time of JDBC connections
– If no calls are done on a JDBC connection for a given amount of
time, this connection is released and go back in the pool
● These solutions are only patches
● Goal: Fix the problem in the application's code
– Help provided by the JOnAS web console
● Track the root of the problem

14. # 14
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Servlet using JDBC connections
55 protected void doGet(....) {
56 response.setContentType("text/html");
57 PrintWriter out = response.getWriter();
58 out.println("<html><body>");
59
60 DataSource ds = null;
61 try {
62 ds = (DataSource) new InitialContext().lookup("jdbc_1");
63 ds.getConnection();
64 } catch (NamingException e) {
65 e.printStackTrace();
66 } catch (SQLException e) {
67 e.printStackTrace();
68 } finally {
69 out.println("</body></html>");
70 out.close();
71 }
72
73 }

15. # 15
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Screenshot of JOnAS Admin console
Line to analyze

16. # 16
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Servlet with the JDBC error
55 protected void doGet(....) {
56 response.setContentType("text/html");
57 PrintWriter out = response.getWriter();
58 out.println("<html><body>");
59
60 DataSource ds = null;
61 try {
62 ds = (DataSource) new InitialContext().lookup("jdbc_1");
63 ds.getConnection();
64 } catch (NamingException e) {
65 e.printStackTrace();
66 } catch (SQLException e) {
67 e.printStackTrace();
68 } finally {
69 out.println("</body></html>");
70 out.close();
71 }
72
73 }

17. # 17
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Demo
Tracking JDBC connection leaks

18. # 18
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Diagnostic tool
Monitoring/displaying JVM Threads

19. # 19
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Information about JVM threads

20. # 20
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Demo
Threads monitoring

21. # 21
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Audit tools

22. # 22
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Goals of the audit system [1/2]
● Development
● Discovery of the software architecture of applications and calls
between the Java EE modules
→ Difficult to track (complex/distributed applications )
● Tracking the performance problems:
→ Enhance the performance
→ Identify the component that is causing the problem
● Qualifying
● Statistics on features/services that are used (top 10, ...)
● Adapt applications to their usage
● Trends on applications/services
– Response time, ...

23. # 23
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
● Production
● Audit
● Traceability
● Log of services that have been used
● Billing (You pay what you're using)
– (Google App Engine)
Goals of the audit system [2/2]

24. # 24
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Commercial Tools
● Commercial tools
● CA Wily Introscope®
● dynaTrace
● BMC AppSight
● Compuware Vantage Analyzer

25. # 25
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Solution based on interceptors
● Different level of interceptors
● Enabling/disabling on demand
● EJB 3
● Invocation (Business service calls)
● Lifecycle (Start/Stop)
● HTTP requests
● Servlet filter
● JNDI access
● Each call on the context returned by the command
new InitialContext() »: lookup, bind, etc.

26. # 26
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Architecture of the Audit System
EasyBeans
Tomcat
JNDI Audit log
JOnAS Admin (Audit module)
JMX
Notifications
Jconsole / JMX Client
Audit System
JASMINe

27. # 27
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Collected data [1/2]
● EJB3
● Invocation
– Bean's name
– Identity (name + roles)
– Called method
● @Local
● @Remote
● OnMessage
– Size of method parameters
– Result
– Elapsed time in the method
– Exceptions

28. # 28
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
● HTTP
● URL
● Encoding
● Client (protocol,host, port)
● SessionId
● Query
● Status HTTP
● JNDI
● Method that is called on the InitialContext
– bind, lookup, ...
– Parameters (if any)
● Elapsed time
Collected data [2/2]

29. # 29
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Traceability / Logger
● Client of Audit MBeans
● Collecting data
● Storage in a log file
● Human readable format
[10/03/04 22:05:35] class org.ow2.util.auditreport.impl.InvocationAuditReport
requestStart = 1267736735591573000
requestStop = 1267736735591630000
requestDuration = 0.057
businessMethod = getCalculator@Local
BeanName = Calculator
target = /easybeans/audit-sample.ear/audit-sample-ejb.jar/SessionFacade/getCalculator@Local
paramSize = 5
returnSize = 0
freeMemoryBefore = 25623392
totalMemoryBefore = 64126976
freeMemoryAfter = 25617704
totalMemoryAfter = 64126976
sweepMarkTime = 873
scavengeTime = 5170
user = ANONYMOUS
roles = [JOnAS]
requestTimeStamp = 1267736735580
methodStackTrace = [java.lang.Thread.getStackTrace(Thread.java:1409) - ..... ]
methodParameters = null
Elapsed time
Called method
Identity
Parameters

30. # 30
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Screenshot of the tool

31. # 31
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Screenshot of a method's graph

32. # 32
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Advanced mode
● Tracking a request on several servers
● Tracking asynchronous calls
● Sending to JMS queue / Receiving from a JMS queue
JMS
Servlet
Server 1
Servlet
EJB
Server 2
MDB
Server 3
IDID
IDID
IDID
EJB
Server 4
IDID
Collecting
Events

33. # 33
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Demonstration

34. # 34
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Demo
● Goal of the demonstration
● Enhancing the performances of an application
– Discovering problems
– Solving problems
– Checking this with the audit console
● Traceability of calls in an application

35. # 35
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Conclusion

36. # 36
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Conclusion [1/2]
● Preventing performance problems
→ Secure a project
● Tools can be used in designing/integrating/production
● In production, an other Java EE server may be used
● Tool bundled with JOnAS
● Key feature comparing to other Java EE servers
● Ready to use
● Open Source / LGPL
● Integrated in JOnAS 5.2

37. # 37
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
● Supervising OSGi service
● Available OSGi services
● Links between components/services
● …
● Supervising JPA
● Life cycle of “Entities”
● Other metrics
● SQL request
– Number of requests
– Elapsed time of requests
● ...
Conclusion: what's next ? [2/2]

38. # 38
OW2 Annual Conference 2010, November 24-25, La Cantine, Paris.
www.ow2.org.
Q & A
Florent Benoit, BULL/OW2 [ @florentbenoit ]