Why Big Data Analytics Needs Business Intelligence Too

Copyright © 2014 9sight Consulting, All Rights Reserved
Dr Barry Devlin
Founder & Principal
9sight Consulting
Why Big Data Analytics Needs
Business Intelligence Too
BrightTALK Webinar
9 April 2014

Dr. Barry Devlin
2 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
Founder and Principal
9sight Consulting, www.9sight.com
Dr. Barry Devlin is a founder of the data warehousing industry
and among the foremost authorities worldwide on business
intelligence (BI) and beyond. He is a widely respected
consultant, lecturer and author of the seminal “Data
Warehouse—from Architecture to Implementation”. His new
book, “Business unIntelligence—Insight and Innovation
Beyond Analytics and Big Data” (http://bit.ly/BunI-Technics)
was published in October 2013.
Barry has 30 years of experience in IT, previously with IBM, as
an architect, consultant, manager and software evangelist.
As founder and principal of 9sight Consulting (www.9sight.com),
Barry provides strategic consulting and thought-leadership to
buyers and vendors of BI solutions. He is currently developing
new architectural models for fully consistent business support—
from informational to operational and collaborative work.
Based in Cape Town, South Africa, Barry’s knowledge and
expertise are in demand both locally and internationally.
Email: barry@9sight.com
Twitter: @BarryDevlin

Big data analytics began with social media
and web logs
 Understanding and tracking sentiment
– What do you think? How do you react?
– Basic analytics and BI activity on a new
data source
 Real-time insight into and influence
on website activities
– Why did you abandon your cart?
– What would you most likely buy
on getting a cross-sell?
– Deep, real-time analytics and BI
with operational integration

Add the Internet of Things to big data analytics
and reinvent businesses
 Significant new considerations
– Micro-management of supply chains and
extension all the way to the consumer
– Sourcing and delivery
– Completely new business models (usually depending on big
data analytics)
– Motor insurance
– Health monitoring

But wait… it’s not just big data
…we also need traditional business data
 Traditional business processes
– Data created, managed and used in a
structured and regulated way
– “Process-mediated data”
– The legal basis of business
 Big data analytics
– Data gathered from unreliable sources,
often designed for unrelated purposes
 Business value of big data depends on
linking it to traditional business processes

 Characteristics
– Tactical decision making
based on reconciled data
– Consistency and truth
– Separation of
operational and
informational needs
– Vertical and horizontal
segmentation of data
– Unidirectional data flow
 Note: key business needs and
technology limitations of the ’80s and ’90s
6
Process-mediated data is the core of BI and layered
Data Warehouse since the early ’90s
Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
Data marts
Enterprise data warehouse
Metadata
Data
warehouse
Operational systems
“An architecture for a business
and information system”,
B. A. Devlin, P. T. Murphy,
IBM Systems Journal, (1988)

The tri-domain model shows two new types of data /
information
 Process-mediated data
– “Traditional” operational
& informational data
– Via data entry and
cleansing processes
 Machine-generated data
– Output of machines
and sensors
– The Internet of Things
 Human-sourced information
– Subjectively interpreted
record of personal
experiences
– From Tweets to Videos
Human-sourced information
Machine-
generated
data
Process-mediated
data
Structure/Context
Timeliness/
Consistency
HistoricalReconciledStableLiveIn-flight
[In the context of these domains, “data” signifies well-structured and/or
modeled and “information” is more loosely structured and human-centric.]

The modern, REAL logical architecture
 Realistic, Extensible,
Actionable, Labile
 Three interconnected pillars
of information
– Messages, events, measures
and transactions from real
world
– Metadata is context-setting
information
 Adaptive process
– Business and IT
– Information processing
– Instantiation, assimilation and
reification – ETL, ELT,
Virtualization
– Workflows and activities
– Choreography
EventsMeasures Messages
Transactions
Reification
Utilization
Choreography
Organization
Instantiation
Human-
sourced
(information)
Machine-
generated
(data)
Process-
mediated
(data)
Context-setting (information)
Assimilation
Transactional
(data)

Key characteristics of information pillars
 Single architecture includes all
types of data/information
– Mix/match technology as needed
– Relational, NoSQL, CEP, Graph,
etc.
 Integration of sources and stores
– Operational processes gather
measures, events, messages and
transactions
– Assimilation integrates stored
information
 Data flows as fast as needed and
reconciled when necessary
– No unnecessary storage or
transformations
Transactions
Human-
sourced
(information)
Machine-
generated
(data)
Process-
mediated
(data)
Assimilation
Transactional
(data)
Operational
Processes

Process-mediated data: Relational databases evolve
to allow de-layering and reintegration
 Drivers: Stability, Consistency and Reliability
 Relational databases remain core technology
– “New” approaches to storage and processing
– Columnar (and compressed) to hybrid
– Solid-state disk and in-memory
– Massively parallel processing
– Advantages:
– Reduced physical modelling
– Faster read and write
 Sample offerings:
– Upgraded databases: e.g. IBM DB2 BLU, etc.
– Appliances: e.g. Actian ParAccel, HP Vertica,
SAP HANA, etc.
– BI Tools: e.g. Tableau, Qlikview, etc.
Multi-core
MPP

Machine-generated data: NoSQL and streaming take
on relational at the extremes
 Drivers: Speed, Size and Flexible Structure
 NoSQL is the current darling,
especially at the extreme of
all three drivers
 CEP (complex event processing) /
Streaming at extreme speed
 Relational can address many
of these drivers
– Even flexible structure (see my
relational vs. Hadoop session)

Human-sourced information: Hadoop and/or
enterprise content management
 Drivers: Soft, Large and Ill-defined data
 Hadoop , Hadoop and more Hadoop
– Hadoop 2.0 enables more real-time
processing
 Traditional ECM tools should
not be forgotten
– Enterprise content management
– Soft information needs
to be managed

Information processing creates, maintains and
mediates access to all information.
 Instantiation
– Turns measures, events and
messages into info. instances
– File access, ETL, change capture…
 Assimilation
– Creation of reconciled and consistent
info. sets prior to business use
– Key to big data – BI linkage
– With context-setting information
– ETL, ELT and virtualization
 Reification (making the abstract real)
– Providing a real-time, consistent,
cross-pillar access to info. according
to an overarching model
– Virtualization
Transactions
Reification
Instantiation
Human-
sourced
(information
)
Machine-
generated
(data)
Process-
mediated
(data)
Assimilation
Transactional
(data)
Organization

Context-setting information (metadata) is key.
 Metadata is two four-letter words!
– Information (not data)
– Describes all “stuff” (not just data)
– Indistinguishable from “business information”
by non-IT people (and some IT people)
 Context-setting information (CSI)
– New image: describes what it is and does
– Context-setting information provides the background to each
piece of information, to every process component and to all
the people that constitute the business
– All information is actually context-setting for something else
 How to create CSI
– Modeling up-front combined with Text Mining on the fly
Mars Climate Orbiter,
lost in 1999, $325M:
metadata error

From BI to Business unIntelligence
 Rationality of thought and far beyond it
 Logic of process, predefined and emergent
 Information, knowledge and meaning
 The confluence of
– Reason and inspiration
– Emotion and intention
– Collaboration and competition
– All that comprises the human and
social milieu that is business
 Not business intelligence
 Business unIntelligence
 http://bit.ly/BunI-Technics: 25% discount with code “BIInsights25”

Conclusions
 Big data and the Internet of Things only
offer background to “real” business
 Reconciled and consistent data built via
Data Warehouse and BI contains the
reality of the business – legally-binding
actions and transactions
 The emerging architecture consists of
three interconnected information pillars
based on appropriate technologies

Why Big Data Analytics Needs Business Intelligence Too

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (18)

Destaque

Destaque (6)

Semelhante a Why Big Data Analytics Needs Business Intelligence Too

Semelhante a Why Big Data Analytics Needs Business Intelligence Too (20)

Último

Último (20)

Why Big Data Analytics Needs Business Intelligence Too