BigData Visualization and Usecase@TDGA-Stelligence-11july2019-share
1. Data Visualization and Big Data
Sharing Experience
Santisook Limpeeticharoenchot
Managing Director
Santisook.l@stelligence.com
0827049888
11 July 2019
3. Copyright STelligence Co.,Ltd. 2018 All rights reserved.
Predictive Analytic &
DataScience Workshop 2016
Tableau,Alteryx
Workshop March 2017
• BigData MBA TU. 2017
• Fraud Analytic 2016
2015-2017 Events in Thailand
MBA TU 2016
• BigData CSA ASEAN2015 • DataScience Meetup 2015 Hosting Bigdata Hackaton2015
4. Copyright STelligence Co.,Ltd. 2018 All rights reserved.
Stock Exchange of Thailand, 18 July 2018
Tableau,Alteryx
Workshop March 2017
DataCube, Tableau&Alteryx, 29-30 June
2018 Public Speakers in Thailand
IMC Big Data Event, 10-11 July
Government Saving BankMBA Thammasart 11 August
6. Agenda for Today
• Data Visualization and Visual Analytic Sharing Experience
• Big Data Usecases
• Workshop
• Q&A
7. Data Sources
Umstructured
Social Media,
Website
Semi-structured
IoT,Sensor,
Logs,GPS
Structured
Internal Data
Access &
Store Data
Hadoop
NoSQL
DataWareHouse
Traditional
Database (SQL)
Analysis Data &
Process
Predictive
Social Analytic
Prep & Blend
Descriptive
Visualize &
Decision
Predictive API
Self-Service
Analysis
Visual Analytic
Traditional BI/
Dashboard
Reporting
User of
Analysis
Business Analyst &
Data Scientist
Top-Mgmt
BU-Manager
IT-Manager
Recap : Big Data Analytic Framework
8. Brief History of Data Visualization
http://www.datavis.ca/papers/hbook.pdf, A Brief History of Data Visualization, Michael Friendly,2006
12. I II III IV
x y x y x y x y
10 8.04 10 9.14 10 7.46 8 6.58
8 6.95 8 8.14 8 6.77 8 5.76
13 7.58 13 8.74 13 12.74 8 7.71
9 8.81 9 8.77 9 7.11 8 8.84
11 8.33 11 9.26 11 7.81 8 8.47
14 9.96 14 8.1 14 8.84 8 7.04
6 7.24 6 6.13 6 6.08 8 5.25
4 4.26 4 3.1 4 5.39 19 12.5
12 10.84 12 9.13 12 8.15 8 5.56
7 4.82 7 7.26 7 6.42 8 7.91
5 5.68 5 4.74 5 5.73 8 6.89
Can we trust statistical properties?
Property Value
Mean of x in each case 9 (exact)
Variance of x in each case 11 (exact)
Mean of y in each case 7.50 (to 2 decimal places)
Variance of y in each case 4.122 or 4.127 (to 3 decimal places)
Correlation between x and y in each
case
0.816 (to 3 decimal places)
Linear regression line in each case
y = 3.00 + 0.500x (to 2 and 3 decimal
places, respectively)
“Anscombe’s Quartet” Source: Wikipedia
13. But they are different in visualization
“Anscombe’s Quartet” Source: Wikipedia
26. COLOR VISION DEFICIENCY (COLOR BLINDNESS)
1.Protanopia is the lack of long-wave cones (red weak).
2.Deuteranopia is the lack of medium-wave cones (green weak).
3.Tritanopia is the lack of short-wave cones (blue). (This is very rare, affecting less than 0.5
percent of the population.
35. Good vs Poor Design
Highlight One, Tell Story Direct Label Proper axis ratio
https://www.eea.europa.eu/data-and-maps/daviz/learn-more/chart-dos-and-donts
40. Visual Analytics Data Visualization
Unknown Unknowns Known Unknowns
Uncertainty ClarityPattern Insight Focus
41. Visual Analytic
What questions we are going to
ask?
How can I use data to introduce
measurable business benefit?
Transition from insight to action
Let’s brainstorm
Data Visualization
What chart to visualize
our data?
How to create ABC
reports?
World of wizard
Let’s gather business
requirement
42. Explanation
• It’s a process of telling stories with data. You actually create a narrative to lead your viewers through your
analysis. Your job here is to facilitate a conversation between your data and your readers.
• Interactivity can be very powerful, it allows your reader explore the data themselves. Check out
this https://public.tableau.com/en-us/s/gallery/tale-100. Play with it a bit, think about how the interactivity
enhances the visualization.
43. Exploration
• Once the data’s in good shape, you’ll explore it to gain an understanding of the data. You’ll want to
look at how data is distributed, if some variables are correlated, and how records are split between
categories. This process is usually called EDA for exploratory data analysis.
• Data visualization comes in handy here as you can plot out distributions of your data, and create
things like scatter plots to reveal correlations. It’ll help you look for interesting patterns in the data,
and other things that will help guide decisions.
51. References
1. The Big Book of Dashboards: Visualizing Your Data Using Real-World Business Scenarios
Steve Wexler, Jeffrey Shaffer, Andy Cotgreave
2. Tableau’s visualization from partner’s presentation.
3. https://www.udacity.com/course/data-visualization-and-d3js--ud507
4. Storytelling with Data: A Data Visualization Guide for Business Professionals, Wiley, Cole,Numssbaumer,Knaflic.
5. Diagram based on “The Design Process Squiggle” by Damien Newman. http://cargocollective.com/central/ The-Design-
Squiggle
6. http://www.visual-analytics.eu/
7. http://www.vismaster.eu/wp-content/uploads/2010/11/VisMaster-book-lowres.pdf
8. https://www.eea.europa.eu/data-and-maps/daviz/learn-more/chart-dos-and-donts
9. http://www.datavis.ca/papers/hbook.pdf, A Brief History of Data Visualization, Michael Friendly,2006.
10. http://www.gartner.com/reprints/datarobot?id=1-53T7OAP&ct=180618&st=sb
53. Data Sources
Umstructured
Social Media,
Website
Semi-structured
IoT,Sensor,
Logs,GPS
Structured
Internal Data
Access &
Store Data
Hadoop
NoSQL
DataWareHouse
Traditional
Database (SQL)
Analysis Data &
Process
Predictive
Social Analytic
Prep & Blend
Descriptive
Visualize &
Decision
Predictive API
Self-Service
Analysis
Visual Analytic
Traditional BI/
Dashboard
Reporting
User of
Analysis
Business Analyst &
Data Scientist
Top-Mgmt
BU-Manager
IT-Manager
Data Analytic Framework
55. Big Data Enhances Public Sector Efficiency
Historical
Archives
Cyber Threat
MetadataVehicle
Telemetry Data
Disease
Outbreaks
Natural
Disasters
PUBLIC
TRANSPORTATION
INFRASTUCTURE
MAINTENANCE
PUBLIC
HEALTH
NATIONAL
DEFENSE
HOMELAND
SECURITY
Social
MediaWork
Orders
Meeting
Notes
Voter Rolls
Public Benefits
Claims
Financial
Audits
Extreme
Weather Alerts
76. Data Sources
Umstructured
Social Media,
Website
Semi-structured
IoT,Sensor,
Logs,GPS
Structured
Internal Data
Access &
Store Data
Hadoop
NoSQL
DataWareHouse
Traditional
Database (SQL)
Analysis Data &
Process
Predictive
Social Analytic
Prep & Blend
Descriptive
Visualize &
Decision
Predictive API
Self-Service
Analysis
Visual Analytic
Traditional BI/
Dashboard
Reporting
User of
Analysis
Business Analyst &
Data Scientist
Top-Mgmt
BU-Manager
IT-Manager
Data Analytic Framework
77. Big Data Analytic Usecase (1Y quick win) :_ลดค่าใช้จ่ายในการติดตามค่าไฟฟ้า_________
Data Source
1. 3Vs
2. Internal/External
3. Data Availability
Transformation &
Storing Required for
Analysis
Analytic
Method
(Descriptive,
Predictive)
Analytic
Consumption
(Dashboard/API)
Who are Data
Consumers
1.ข้อมูลชำระตรง กับไม่ตรง คำนวณ ระยะระหว่ำงบ้ำนถึง
ศูนย์
สถิติ ควำมสัมพันธ์ ใช้
predictive model
หำ factors ที่มี
impact สูง
dashboard หน่วยงำนกลยุทธ์
2.ช่องทำงชำระ List ของลูกค้ำที่มีควำม
เสี่ยง
กำรทดสอบ A/B Testing,
Pilot group.
3.ระยะทำง
Pain Points -> Benefit or Value:____100 ล้านบาท ต่อปี__________________
78. Big Data Analytic Usecase (1Y quick win) :_การแตกรั่วของท่อประปา_________
Data Source
1. 3Vs
2. Internal/External
3. Data Availability
Transformation &
Storing Required for
Analysis
Analytic
Method
(Descriptive,
Predictive)
Analytic
Consumption
(Dashboard/API)
Who are Data
Consumers
1.ปริมำณ สำเหตุของกำรแตกรั่ว สถิติ correlation
factor analysis
กำรใช้ model ในกำรหำ
โอกำสที่จะมีจุดรั่ว
dashboard ผ่ำย operation
2.แรงดัน Alert เมื่อควำมดันตก
เพื่อส่งทีมไปค้นหำปัญหำ
3.จุดแตกรั่ว
Pain Points -> Benefit or Value:____250 ล้านบาทต่อปี__________________
80. Actionable Intelligence Powers Today’s
Financial Services
OFAC
Lists
Credit
Records
ATM
Streams Transactions
& Wires
Stock
Tickers
Trade
Settlements
DIGITAL
CUSTOMER 360
RISK DATA
AGGREGATION
ANTI-MONEY
LAUNDERING
FRAUD
DETECTION
TRADE
SURVEILLANCE
Mobile
App Data
Trade
Data
Web
Logs
Banker
Notes
Demographic
Data
Customer
Transactio
n Data
81. Actionable Intelligence Transforms
Energy & Utilities
Asset
Data
Customer
Surveys
Weather &
Environmental
Service Fleet
GPS Data
Smart Meter
Streams
Commodity
Prices
REVENUE
PROTECTION
SINGLE VIEW
OF CUSTOMER
PREDICTIVE EQUIPMENT
MAINTENANCE
CONSERVATION
VOLTAGE REDUCTION
COMMODITY
TRADING
Social
Media
GIS
Data
SCADA Outage
Histories
CIS
Records
EDW
82. Actionable Intelligence Drives the New
Automotive Industry
ERP Data
Warranty
Data
Geo
Tracking
Infotainment
Metadata
SCADA
Systems
Social Media
Streams
PREVENTATIVE
MAINTENANCE
SUPPLY CHAIN
OPTIMIZATION
MANUFACTURING YIELDS
MAXIMIZATION
QUALITY
CONTROL
NEW PRODUCT
PLANNING
ERP
Systems
Defect
Testing
Data
Machine
Data Data
Historians Product
Design Docs
Service
Records
83. Actionable Intelligence Makes Healthcare
Precise and Personal
Patient
Records
Lab Data
Pharmacy
Data
Patient
Locations
Wearables
Intra-Network
Data
Sensor
Data
Claims
Data
Social
Media Physician
Notes Patient
Satisfaction Data
Clinical
(EMR)
Data
SINGLE VIEW OF
PATIENT
REAL-TIME VITAL
SIGN MONITORING
BILLING &
REIMBURSEMENTS
EMR
OPTIMIZATION
SUPPLY CHAIN
OPTIMIZATION
84. Actionable Intelligence Fuels Oil & Gas
Industry Renovation
ERP Data
Engineering
Notes
IoT
Gateway
Data Video
WITSML
Data
Weather &
Environment
REAL-TIME
MONITORING
SINGLE VIEW OF
OPERATIONS
PREDICTIVE
MAINTENANCE
LAS ARCHIVE
& ANALYTICS
UNSTRUCTURED DATA
CLASSIFICATION
Vehicle
GPS Data
GIS Data
SCADA
Systems Field
Comments
Production
Histories
G&G
Data
85. Actionable Intelligence Is Shaping the Modern
Insurance Industry
Catastrophic
Event Data
Customer
Onboarding Data
Seismic
Data
Biometrics
Data
Usage-Based
Driver Data
Cyber Threat
Metadata
RISK & UNDERWRITING
ANALYSIS
USAGE-BASED
INSURANCE
CLAIMS
ANALYTICS
NEW PRODUCT
DEVELOPMENT
CYBER RISK
ANALYTICS
Drones &
Aerial Imagery
Claims Docs,
Notes &
Diaries
Weather &
Environment
Underwritin
g Analysis
Policy
Histories
Photos
86. Actionable Intelligence Powers Modern
Manufacturing
Defect
Testing Data
Product
Designs
MES
System
s
RFID
Streams
SCADA
Systems
Shop Floor
Sensors
PREVENTATIVE
MAINTENANCE
SUPPLY CHAIN
OPTIMIZATION
YIELD
MAXIMIZATION
QUALITY
CONTROL
RECALL
AVOIDANCE
ERP
Systems
Supplier
Receipts
Machine
Data
Assembly
Line Sensors
Data
Historians
Work
Orders
87. Actionable Intelligence Drives
Retail Sales Growth
Product
Catalogs
Sales
Forecasts
Beacons &
RFID Server
Logs
In-Store
WiFi Logs
Store
Communication
s
SINGLE VIEW OF
THE CUSTOMER
PRODUCT
RECOMMENDATIONS
INVENTORY &
SUPPLY CHAIN
PRICING
OPTIMIZATION
TARGETED
PROMOTIONS
Clickstrea
m
ERP
Data
Social
Media
Staffing
Plans
Store
Reporting
CRM
Records
88. Actionable Intelligence Personalizes Digital Advertising
Market
Research
Studies
CRM
Records
Online
Transactions
Social Media
Streams
Impressions
Video
Consumption Logs
CUSTOMER
SEGMENTATION
ONLINE AD
PLACEMENT
PRODUCT
RECOMMENDATIONS
TARGETED
PROMOTIONS
VIDEO
SYNDICATION
Sensor
Data
Product
Catalogs
Server Logs
Clickstreams Customer
Surveys
Sales
Reports
89. Actionable Intelligence Transforms
the Software Industry
Cyber Security
Metadata
Sales
Forecasts
Mobile Device
Geo-Location Server
Logs
User Activity
Events
Network
Logs
NEW PRODUCT
DEVELOPMENT
QUALITY ASSURANCE CUSTOMIZATION &
PERSONALIZATION
CYBER
SECURITY
REAL-TIME USAGE
MONITORING
Clickstreams
CRM
Records
Social Media
Streams
Sprints &
Backlogs
User
Testing
Historical
Audit Trails
90. Connected Data Drives Success in
Telecommunications
Call Detail
Records
Product
Catalogs
Cyber Threat
Metadata
Sensor
Data
Server
Logs
Voice-to-Text
SINGLE VIEW OF
THE CUSTOMER
CHURN
REDUCTION
CDR
ANALYSIS
NETWORK
OPTIMIZATION
DYNAMIC BANDWIDTH
ALLOCATION
Clickstrea
m
ERP System
Data
Social
Media Billing
Data
Subscriber
Profiles
CRM
Records