1. How to Act on Big Data in Real Time – Part 2
Name · Title · Dunn SolutionsJanani Eshwaran
Renata Simanjuntak

2. Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next

3. Dunn Solutions is a Full-Service IT Consulting Firm
Founded in 1988
Raleigh, NC
Delivery  Training
Bangalore, India
Delivery
Minneapolis
Delivery  Training
Chicago
Delivery

4. Practice Areas
Application
Development
• Portals
• eCommerce &
Content Managed
Websites
• Mobile App
Development
• Custom App
Development
Training
• Certified
SAP/Liferay
• Classroom, On-
site, Computer
Based & Virtual
• Mentoring &
Custom Training
Frameworks
• Accountable Care
Orgs (ACO’s)
• Corporate Legal
• Higher Education
• Optical Shop
Solutions
Analytics
• Analytics & BI
Platforms
• Data Warehouse
& Data
Integration
Big Data
Predictive
Analytics

5. Selected Clients

6. PartnershipsPartnerships

7. Analytics Practice
Business Intelligence
Big Data
Data IntegrationBusiness Analytics
Data Warehousing
• KPI’s and Metrics
• Dashboards
• Data Exploration and
Visualization
• Ad Hoc Analysis &
Reporting
• Data Mining
• Predictive Analytics
• Prescriptive Analytics
• R, AzureML
• Hadoop, MapReduce
• AWS and Azure
• Hive, Sqoop, Spark
• NoSQL
• Data Lake
• Columnar
• In-memory
• EIM (Data Integration
and Data Quality
• Dimensional Modeling

8. Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next

9. Real Time Big Data Analytics
• It is not only to store and analyze streaming
BIG data
• It is more about making better decision and
taking meaningful action at the right time
Traditional Enterprise Data Warehouse plus analytics are no longer enough

10. • Fraud detection while a
credit card is swiped
• Triggering an offer while a
shopper is standing on a
checkout line
• Placing an ad on a website
while someone is reading a
specific article
Real Time Business Benefit

11. • Service improvement
• Cost savings
• Fraud detection
• Keep up with customer trends
• Sales insights enhancement
• Instantly errors detection
• Immediate new strategies of your competition
notification
Power of Real Time Analytics

12. • Financial Loss
• External Confidence
• Company Morale
• Increased Audit Costs
How Fraud Hurts You & Your Organization

13. Impact of Fraud in Telecom Industry
Communications Fraud Control Association (CFCA) Global Fraud Loss Survey
Telecom Fraud cost the industry 2015 over 38 Billion USD annually

14. Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next

15. Azure Event Hub
Customer Name / 16
• Benefits:
• Stream millions of events per second
• Process events with variable load profiles
• Connect millions of devices across platforms
• How much data? Throughput units
• To scale the traffic coming in or out
• Key pricing parameter
• In (Publisher): 1mb or 1000 events/sec
• Out (Consumer): 2mb/sec

16. Azure HDInsight - Ecosystem
• Big Data with No
Hassle
• Open and
Flexible
• Insight in MS
Excel
• Build Big Data
Apps your Way

17. • Scalable
• High-throughput
• Fault-tolerant
• Stream processing of live data streams
• Data collected can be later post-processed
• Code and business logic can be shared and reused
Spark Streaming for Real Time Analytics
Spark
Streaming
Spark
Engine
Input Data
Stream
Batches of
input data
Batches of
processed data
Less time learning, implementing, and maintaining different frameworks
More focus on developing smarter applications

18. Hive
Customer Name / 19
• Data warehouse in Hadoop
• Project structure on largely unstructured data
• Work with structured and semi-structured data
• Hive QL
• Low cost data storage

19. • Empower user
• Q&A function
• Dashboard visualization
• Innovative technology
• In memory engine
• Columnar database
• You own your data
• Faster turn around
• Lower cost
Power BI
Customer Name / 20
Enterprise-level data is yours for free or at a very low monthly cost

20. • Email
• Link
• Website
• Phone: text or call
• Application
Real Time Alert
Customer Name / 21
You are informed in real time when errors or frauds or anomalies exist
Take action in real time for real results

21. Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next

22. Big Data Real Time Use Case for Telecom (1)
• Real-time fraud prevention
• Can be passed on to customer bills
• Prevent revenue loss and additional expense to correct
• Visibility of service performance, costs and
discounts to the customer
• Cannot monitor customer bills to provide services
• Analyze and offer products and discounts
• Optimization of Least Cost Routing (LCR)
• Choose lost cost network in real time
• Select optimized and high performing network quickly

23. Big Data Real Time Use Case for Telecom (2)
• Call performance monitoring
• Cannot prevent dropped calls and issues
• Can identify issues to resolve immediately
• Real-time profitability analysis
• Make use of long term trend data offline
• Can learn service provided to customer for
understanding gross margin

24. What if I wanted to…
• Capture data from any application in real time
• Store the data
• Perform analysis on the streamed data
• Visualize the information interactively
Demonstration: Setting the Stage

25. Big Data, Real time project checklist
• Azure Event Hub
• Azure HDInsight cluster
• Spark Streaming
• Hive
• Azure SQL database
• Power BI
• Real Time Notification (email)
What Do I Need?

26. Our Demo Architecture
Streaming data
sources
(Call Records)
Azure
Event Hub
Consume Store
Streaming
data
Enrichment
data

27. Step 1: Start The Event Hub Event To Collect The Events
Streaming data sources
(Call Records)
Azure Event Hub

28. Step 2: Prepare Your Receiver To Receive Events
Streaming data sources
(Call Records)
Azure Event Hub
Consume

29. Step 3: Persist The Events In Hive Table
Streaming data sources
(Call Records)
Azure Event Hub
Consume Store

30. Step 4: Alert /Notify the anomaly
Streaming data sources
(Call Records)
Azure Event Hub
Consume Store

31. Step 5: Visualization in Power BI
Streaming data
sources
(Call Records)
Azure
Event Hub
Consume Store
Streaming
data
Enrichment
data

32. Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next

33. Recap
Streaming data
sources
(Call Records)
Azure
Event Hub
Consume Store
Streaming
data
Enrichment
data
1. Capture streaming
data
2. Process and
store data
4. Analyze
3. Alert by
email

34. • What is your Big Data strategy?
• Do you have a Big Data project in mind?
• Are you wondering how you can use Big Data for
real time data analysis to benefit your company?
• Should you do it on premise or in the cloud?
• Contact us and we’ll help you execute!
• info@dunnsolutions.com
Let’s Get You Started with Real-Time Big Data

35. Thank You
Janani Eshwaran· Analytics Consultant · Dunn Solutions
jeshwaran@dunnsolutions.com
Renata Simanjuntak· Analytics Manager· Dunn Solutions
renatas@dunnsolutions.com

36. • http://cfca.org/fraudlosssurvey/2015.pdf
• http://www.cfca.org/pdf/survey/CFCA2013Glob
alFraudLossSurvey-pressrelease.pdf
• Microsoft blogs and tutorials
• Other analytics webinars:
• http://www.dunnsolutions.com/content/webinar-
white-paper
Reference