The Briefing Room with David Loshin and Datawatch
Live Webcast Feb. 17, 2015
Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=4a053043c45cf0c2f6453dfb8577c72a
Patience may be a virtue, but when it comes to streaming analytics, waiting is no option. Between Big Data and the Internet of Things, businesses are faced with more data and greater complexity than ever before. Traditional information architectures simply cannot support the kind of processing necessary to make use of this fast-moving resource. The modern context requires a shorter path to analytics, one that narrows the gap between governance and discovery
Register for this episode of The Briefing Room to hear veteran Analyst David Loshin as he explains how the prevalence of streaming data is changing business pace and processes. He’ll be briefed by Dan Potter of Datawatch, who will tout his company’s real-time data discovery platform for data in motion. He will show how self-service data preparation can lead to faster insights, ultimately fostering the ability to make precise decisions at the right time.
Visit InsideAnalysis.com for more information.
3. Twitter Tag: #briefr The Briefing Room
Welcome
Host:
Eric Kavanagh
eric.kavanagh@bloorgroup.com
@eric_kavanagh
4. Twitter Tag: #briefr The Briefing Room
Reveal the essential characteristics of enterprise
software, good and bad
Provide a forum for detailed analysis of today s innovative
technologies
Give vendors a chance to explain their product to savvy
analysts
Allow audience members to pose serious questions... and
get answers!
Mission
5. Twitter Tag: #briefr The Briefing Room
Topics
February: DATA IN MOTION
March: BI/ANALYTICS
April: BIG DATA
6. Twitter Tag: #briefr The Briefing Room
Parmenides and the Truth of Now
There is no tomorrow
There is no yesterday
There is only today
There is only now
7. Twitter Tag: #briefr The Briefing Room
Analyst: David Loshin
David Loshin, president of Knowledge
Integrity, Inc, is a thought leader and
expert consultant in the areas of data
quality, master data management, and
business intelligence. David is the
author of numerous books and papers
on data management, including the
“Practitioner’s Guide to Data Quality
Improvement.” David is a frequent
speaker at conferences and in web
seminars. His best-selling book, “Master
Data Management,” has been endorsed
by data management industry leaders.
David can be reached at
loshin@knowledge-integrity.com, or at
(301) 754-6350.
8. Twitter Tag: #briefr The Briefing Room
Datawatch
Datawatch began as a BI tool and has developed into a
visual analytics platform
The platform provides visual data analytics and discovery on
any type of data, including streaming data
The suite of products are Datawatch Desktop, Datawatch
Server, Datawatch Report Mining Server and Datawatch
Modeler
9. Twitter Tag: #briefr The Briefing Room
Guest: Dan Potter
Dan Potter is the Vice President of
Product Marketing at Datawatch
Corporation. In this role, Dan leads
the product marketing and go-to-
market strategy for Datawatch. Prior
to Datawatch, Dan held senior roles at
IBM, Oracle, Progress Software and
Attunity where he was responsible for
identifying and launching solutions
across a variety of emerging markets,
including cloud computing, visual data
discovery, real-time data streaming,
federated data and e-commerce.
10. VISUAL DATA DISCOVERY
& STREAMING DATA
New Technologies for Real-Time Analytics
Dan Potter
Vice President, Product Marketing
11. NASDAQ: DWCH
Pioneer in real-time visual data discovery and self-service data
preparation
Global operations and support
§ US, UK, Germany, France, Australia, Singapore, Philippines
Extensive global customer base
§ 93 of the Fortune 100
§ 12 of the 15 largest financial institutions
Embedded and resold by leading vendors
About Datawatch
13. Where Do Real-Time Streams Come From?
• Internet of Things
• Machine data / log files
• Web clickstreams
• Enterprise applications
• Human generated
• Commercial data
14. Streaming Visualization Examples
Capital
Markets
§ Transac'on
Cost
Analysis
§ Analyze
market
data
at
ultra-‐low
latencies
§ Momentum
Calculator
Fraud
preven2on
§ Detec'ng
mul'-‐party
fraud
§ Real
'me
fraud
preven'on
e-‐Science
§ Space
weather
predic'on
§ Detec'on
of
transient
events
§ Synchrotron
atomic
research
§ Genomic
Research
Transporta2on
§ Intelligent
traffic
management
§ Automo've
Telema'cs
Energy
&
U2li2es
§ Transac've
control
§ Phasor
Monitoring
Unit
§ Down
hole
sensor
monitoring
Natural
Systems
§ Wildfire
management
§ Water
management
Other
§ Manufacturing
§ ERP
for
Commodi'es
§ Real-‐'me
mul'modal
surveillance
§ Situa'onal
awareness
§ Cyber
security
detec'on
§ Emergency
Evacua'on
Law
Enforcement,
Defense
&
Cyber
Security
Health
&
Life
Sciences
§ ICU
monitoring
§ Epidemic
early
warning
§ Remote
healthcare
monitoring
Telephony
§ CDR
processing
§ Social
analysis
§ Churn
predic'on
§ Geomapping
15. Visual Data Discovery
• Easy for users to author,
customize and share
• Interactive exploration &
visually filter results
• Quickly identify
anomalies and outliers
with large or in-motion
datasets
• Rich palette of
visualizations for static
and time series data
16. Visualize Any Data at Any Speed
Stream
Rela2onal
NoSQL
OLAP
Warehouse
Hadoop
Content
Connect,
Federate,
Visualize
18. Data
at
Rest
Limitations of Traditional BI
Database
Distributed
or
Hybrid
Database
In-‐Memory
Database
Streaming
Analy'cs
19. Data
at
Rest
Streaming Data Visualization
Database
Distributed
or
Hybrid
Database
In-‐Memory
Database
Streaming
Analy'cs
20. Datawatch Streaming Data Visualization
• Connect directly to data in motion
• CEP (IBM Streams, Informatica Rulepoint, Tibco Streambase)
• Hosted IoT platforms (Amazon Kinesis, PTC ThingWorx)
• Message Bus (Informatica UltraMessaging, WebSphere MQ)
• Operational Intelligence Systems (OSIsoft Pi)
• Purpose built data model optimized for both caching
and persistence
• High density visuals with rendering in milliseconds
Monitor
Analyze
Take
Ac2on
21. Time Series Data
• Traditional BI only looks at buckets of
time
• Day, week, month, year
• Streaming data is a continuous and has
different requirements
• Second, millisecond, nanosecond
• Time windows
• Time slices
• Playback
• Complete situational awareness
• Now (streaming)
• Intra-day
• Historic
22. Predictive & Advanced Analytics
• Connect to R (Rserv) and
Python (Pyro) servers
• Transform using R and
Python
• Many use cases in IoT (e.g.
predictive maintenance,
smart logistics, clinical
pattern detection etc.)
23. Modeled
and
transformed
for
analysis
Complex File Formats
• Sensor and machine data often in multi-structured format
• Need to transform, enrich and prepare data
• Almost no metadata
• For example, wave form visualization from JSON arrays
stored in MongoDB and streaming
23
Log
Files
HTML,
XML
JSON
PDFs
25. Customer Challenge
Dozens of risk management
systems generating data silos of
operational information
Server based solution to
visualize integrated risk
information in real-time to
identify trends and anomalies
Analyze patterns in physiological
data that may detect and
eventually to predict deadly
clinical events
Visualize large volumes of
streaming, unstructured data
from multiple devices in real-time
Improve yield production and
enhance machine reliability in
contact lens manufacturing
process
Flexible visualization solution
highlighting production line
yield, leading to a 2% yield
increase and 750,000 additional
units produced
Real-World, Real-Time Examples
Process and visualize billions of
streaming trades per day for
leading surveillance and
compliance platform
Fully embedded visual data
discovery solution that delivers a
single consolidated real-time
view of trading across venues
26.
27. Twitter Tag: #briefr The Briefing Room
Perceptions & Questions
Analyst:
David Loshin