Introducing WSO2 Analytics Platform 3.0, on WSO2Con 2015

1. An Introduction to the WSO2
Analytics Platform
Sriskandarajah Suhothayan (Suho)
Technical Lead
WSO2

2. Why Analytics?

3. Why Analytics?
Reduce cost and increase revenue
Improve customer satisfaction
Improve process efficiency
Track progress over time
Act immediately
Predict future opportunities and threats
...

4. WSO2 Data Analytics Platform

5. Collect Data

6. Collecting Data
* Supports custom event receivers via its pluggable architecture!

7. Store Data
Data Abstraction Layer enabling pluggable data
connectors
● Cassandra
● RDBMS
● HBase
● …
With a common API

8. Analysing

9. Interactive Analytics
What has happened & Why ?
● On demand analysis of data
○ Full text search support
● Selective deep analysis
○ Drill down search
● Distributed data indexing.
● Near real time data indexing and retrieval
message: “Error” AND
date: [2015-11-01 TO
2015-11-03]

10. Batch Analytics
Summarize & Tack progress over time …
● Powered by Apache Spark
○ Up to 30x higher performance than Hadoop
● Optimized in-memory processing
● Spark SQL SQL-like query language
● Schedulable Scripts
○ Simple Configs & Cron Expressions

11. Realtime Analytics
Detect and act immediately …
● Streaming Complex Event Processing
○ Uses WSO2 Siddhi engine
○ Process more than 100k Events/Sec
● SiddhiQL SQL-like Query Language
● Extension Support (Geo, NLP, Math, String, Regex, … )
● Distributed Processing with Apache Storm
Siddhi

12. Predictive Analytics
Predict future & act upon it !
● Explore and build models
○ Via Spark MlLib & R (from WSO2 ML 1.1)
○ Export as PMML (from WSO2 ML 1.1)
● Run models with CEP, DAS and ESB.
● Run R Scripts, Regression and Anomaly
Detection on realtime
● Deep learning & NLP (In roadmap)

13. Machine Learner

14. Communicate

15. Visualizing Results
Overall idea of what's going on …
● Dashboard using
○ Google Gadget
○ HTML5 + javascripts
● Support custom gadgets and
dashboards
● Gadget generation with
○ D3 and Vega

16. Gadget Generation Tool
Customize with few clicks …
● Support realtime and
stored data visualization
● How ?
○ Start with data in tabular format
○ Select chart type (Bar, Line, Map, ...)
○ Map columns to dimensions (X Y, color, point size, … )
○ Also do drill downs

17. Communicating Results
* Supports custom event publishers via its pluggable architecture!

18. Beyond Boundaries
● Expose analytics results
as API
○ Mobile Apps, Third Party
● Provides
○ Security, Billing,
○ Throttling, Quotas & SLA
● How ?
○ Write data to database from DAS
○ Build Services via WSO2 Data Services Server or use Analytics REST API
○ Expose them as APIs via WSO2 API Manager

19. Developer Tools & Deployment

20. Developer Tools
● Event Flow

21. Developer Tools ...
● Event Flow
● Event Tracing

22. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics

23. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It

24. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator

25. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console

26. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console
● Data Explorer

27. Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console
● Data Explorer
● Analytics Tooling

28. ● One Analytics Archive format for all analytics
○ Based to Carbon Archive
● Minimum HA Deployment (2 Nodes)
● Fully Distributed Deployment (8+ Nodes)
● Realtime Distributed Deployment
○ Siddhi runs on top of Apache Storm
Deployment

29. Solutions

30. Solutions
● Fraud Detection
○ Alerts for known and
unknown frauds
○ Deep Search Analytics
○ https://goo.gl/TWV5C1
● IoT Analytics (WIP)
○ With configurable Alerting &
Monitoring capabilities.
● Log Analytics (WIP)
● Analytics for Products (WIP)
○ Each WSO2 Product will have
analytics of its own kind.

31. Custom Solutions
UI to configure predefined
analysis with Execution Manager.
Good for Business Users

32. Use Cases & Case studies

33. Use Cases
● Smart Parking solutions
● Health & Financial Monitoring
● Smart City project, Vehicle tracking,
Building monitoring
● Railway monitoring
● Throttling and Anomaly Detection
● API Analytics
● Connected Car

34. Case Studies
● Realtime Soccer Analytics (DEBS 2013)
○ https://goo.gl/c2gPrQ
● TFL Traffic Analysis
○ Using Transport for London open data feeds.
○ http://goo.gl/04tX6k, http://goo.gl/9xNiCm
● Smart Meters (DEBS 2014)
○ Using smart home electricity data : 2000
sensors, 40 houses, 4 Billion events
○ Processes 400K events/sec, one of the four
finalists at DEBS 2014

35. How to get started ?

36. How to get started ?
● (WSO2 DAS) 3.0
○ Combines all types of analytics.
● (WSO2 CEP) 4.0
○ For who need to analyze realtime event streaming.
● (WSO2 ML) 1.0
○ For building Predictive Models
http://wso2.com/analytics

37. WSO2 DAS vs WSO2 BAM
What's New ?
● Interactive & predictive analytics
● Data Abstraction Layer supporting
○ RDBMS, Cassandra, HBase
● Improved performance with Spark & Siddhi
● Better Dashboard
● Distributed Realtime Processing
● Simple deployment model
● ...

38. “The price of light is less than the cost of darkness”
~ Arthur C. Nielsen, Market Researcher & Founder of ACNielsen

39. Thank You