Making communications land - Are they received and understood as intended? we...
Second review presentation
1. Design and Implementation of a Service
Monitoring Console within a Service Oriented
Architecture Framework
Arvind Krishnaa .J
31508104017
Guided By
Dr. Chitra Babu
HOD/CSE
SSN College of Engineering
Second Review - 30th March, 2012
2. Turmeric Runtime Platform
1 Client Runtime - Client application code written by the
consumers of the service. Service Invocation
Framework(SIF)
3. Turmeric Runtime Platform
1 Client Runtime - Client application code written by the
consumers of the service. Service Invocation
Framework(SIF)
2 Server Runtime - Deployment platform for developers to
publish web services. Service Provider Framework (SPF)
4. Turmeric Runtime Platform
1 Client Runtime - Client application code written by the
consumers of the service. Service Invocation
Framework(SIF)
2 Server Runtime - Deployment platform for developers to
publish web services. Service Provider Framework (SPF)
3 CodeGen - Generate instrumentation code for deploying
services onto Turmeric.
6. Central Application Logging Framework
Centralized collection of application server
logs
Base for collecting data for the
monitoring console
7. Central Application Logging Framework
Centralized collection of application server
logs
Base for collecting data for the
monitoring console
Ability to generate custom reports from
this data
8. Central Application Logging Framework
Centralized collection of application server
logs
Base for collecting data for the
monitoring console
Ability to generate custom reports from
this data
TMC is a specific instance of a report
created using CAL
9. Architecture of the Monitoring Console
1 Can be abstracted to two frameworks running in parallel -
metrics logging framework and metrics handler framework
10. Architecture of the Monitoring Console
1 Can be abstracted to two frameworks running in parallel -
metrics logging framework and metrics handler framework
2 Both frameworks are web services deployed on an application
server
11. Architecture of the Monitoring Console
1 Can be abstracted to two frameworks running in parallel -
metrics logging framework and metrics handler framework
2 Both frameworks are web services deployed on an application
server
3 SOA Query Metrics Service (sqms) - Logging logging handler
framework
12. Architecture of the Monitoring Console
1 Can be abstracted to two frameworks running in parallel -
metrics logging framework and metrics handler framework
2 Both frameworks are web services deployed on an application
server
3 SOA Query Metrics Service (sqms) - Logging logging handler
framework
4 Monitoring Console (console) - Metrics handler framework
14. Working of SQMS
1 SPFServlet is the main servlet which handles the requests to
the Turmeric runtime.
2 Initializes the CAL logger for logging metrics published by
Turmeric services.
3 Metric data logged in a format called heartbeats.
4 Log message casted into a JMX MBean like object.
5 The MBean object is dispatched to the JMX servlet and
published to a controller interface (Agent Service)
16. Metrics Storage Provider
Java interface for defining the underlying storage mechanism
Provides the general contract for defining the queries to store
metrics into the persistence mechanism
17. Metrics Storage Provider
Java interface for defining the underlying storage mechanism
Provides the general contract for defining the queries to store
metrics into the persistence mechanism
Can be extended to provide custom implementations, for
various Hibernate- compliant databases.
18. Metrics Storage Provider
Java interface for defining the underlying storage mechanism
Provides the general contract for defining the queries to store
metrics into the persistence mechanism
Can be extended to provide custom implementations, for
various Hibernate- compliant databases.
Currently two implementations are provided - DAO(based on
a MySql storage) and Cassandra (partially implemented)
19. Metrics Logging
1 saveMetricValue(): It receives the timestamp, metric name,
and a key-value pair to store the metric value, and converts it
to row oriented format for MySQL storage (or name-value pair
for Cassandra)
2 saveMetricSnapshot(): Encapsulates the above received
metrics objects into a MBean instance and sends it to the
JMX server for publication.
20. Logging helper utilities
1 Snapshot File Logger: Logs the aggregated metrics into a
.log file
2 Differential Snapshot View Logger: Calculating the
percentage changes between the metrics collected on two
different days.
3 Monitoring System: Schedules the metrics logging process
to be performed at regular intervals. Presently it is set at 60
seconds.
21. Logging Handlers
1 Metrics Storage Provider Interface
Defines a framework which runs two
MetricsSnapshotSchedulers in separate background threads,
one for the client-side metrics and the other for the server-side
metrics
Calls the saveMetricsSnapshot() method to store the metric
data.
2 DAO Metrics Provider : The provider implementation for
MySQL persistence mechanism
3 Cassandra Storage Provider: Implementation for Cassandra
storage.
Any custom persistence mechanism can also be integrated.
23. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
24. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
Retrieves the encapsulated data in the form of a MetricsData
instance
25. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
Retrieves the encapsulated data in the form of a MetricsData
instance
This object is serialized into JSON format and sent to the
Google Web Toolkit’s charting API
26. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
Retrieves the encapsulated data in the form of a MetricsData
instance
This object is serialized into JSON format and sent to the
Google Web Toolkit’s charting API
GWT API constructs the trends chart based on the JSON data
27. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
Retrieves the encapsulated data in the form of a MetricsData
instance
This object is serialized into JSON format and sent to the
Google Web Toolkit’s charting API
GWT API constructs the trends chart based on the JSON data
A REST URI is created for every JSON object, and a link is
created in the dashboard.
28. Implementation of TMC service
Responsible for constructing user interface of monitoring
dashboard
Retrieves the encapsulated data in the form of a MetricsData
instance
This object is serialized into JSON format and sent to the
Google Web Toolkit’s charting API
GWT API constructs the trends chart based on the JSON data
A REST URI is created for every JSON object, and a link is
created in the dashboard.
The metrics detail can be exported into a CSV file as well.
29. Offline Aggregator - A Partial Implementation
Objective of collecting the distributed metrics across cluster of
nodes, by utilizing Cassandra storage.
Only the storage into Cassandra keyspace has been
implemented
Hooks are provided to read data across distributed nodes, to
make this feature full-fledged.
Three main classes for implementing offline-aggregation
1 Readers
2 Writers
3 Erasers
30. Overview of Readers, Writers and Erasers
Readers: Read the data from the offline cluster, de-serialize
the MetricsData objects and display it in the dashboard.
Different reader classes are available to customize metrics
reading, in the future. (Classes need to be extended with
custom implementation)
31. Overview of Readers, Writers and Erasers
Readers: Read the data from the offline cluster, de-serialize
the MetricsData objects and display it in the dashboard.
Different reader classes are available to customize metrics
reading, in the future. (Classes need to be extended with
custom implementation)
Writers: Write data into the offline cluster, by serializing the
metrics into MetricsData objects. Different writer base classes
available for extending with custom implementations.
32. Overview of Readers, Writers and Erasers
Readers: Read the data from the offline cluster, de-serialize
the MetricsData objects and display it in the dashboard.
Different reader classes are available to customize metrics
reading, in the future. (Classes need to be extended with
custom implementation)
Writers: Write data into the offline cluster, by serializing the
metrics into MetricsData objects. Different writer base classes
available for extending with custom implementations.
Erasers: Erase the data from the offline cluster.
33. Overview of Readers, Writers and Erasers
Figure 3: Overview of Reader, Writers and Erasers
34. Partial Demo
1 Nuts and bolts of Turmeric SOA services
2 Sample REST URLs for metrics
3 Walkthrough of database schema and significant
implementation classes
4 Overview of data aggregator tool
5 Tour of the user interface
35. Screenshots - SMC Dashboard for an eBay service
Figure 4: ShortURI Service
41. References
[1]Jeffrey Dean, Sanjay Ghemawat, MapReduce: Simplified Data
Processing on Large Clusters, In Sixth Symposium on Operating
System Design and Implementation(OSDI’04), San Francisco, CA,
December, 2004
[2] Eben Hewitt, Cassandra: The Definitive Guide, O’Reilly
Publications, November 2010.
[3] eBay Open Source Project, Turmeric SOA platform,
http:
//www.ebayopensource.org/index.php/Turmeric/HomePage
42. References
[4] eBay Open Source Project, Documentation of Turmeric SOA
platform,
https://www.ebayopensource.org/wiki/display/
TURMERICDOC110GA/Turmeric+Documentation+Overview
[5] eBay Open Source Project, Turmeric Source Code,
http://www.github.com/ebayopensource
[6] Internal eBay documentation
[7] Google Web Toolkit,
http://code.google.com/webtoolkit
[8] Apache Cassandra,
http://cassandra.apache.org/
