In this document, the five disruptive trends shaping the corporate IT landscape today are layed out. Out of the five, Big Data has the biggest potential to generate new sustainable competitive advantages. But the benefits will remain out of reach of many organizations as they struggle to adopt the technology, develop new capabilities, and manage the cultural change associated with the use of big data. This document offers a pragmatic approach to generating business value.
Boost Fertility New Invention Ups Success Rates.pdf
Starting small with big data
1. Is Big Data hype or ready for prime time? Once again CIOs are tasked with
making sense of this new technology while charting a pragmatic course for
generating real value. There are five disruptive trends shaping the corporate
IT landscape today (Figure 1), and of the five, Big Data has the biggest
potential to generate new sustainable competitive advantages. But the benefits
will remain out of reach of many organizations as they struggle to adopt
the technology, develop new capabilities, and manage the cultural change
associated with the use of big data. Industry literature is usually focused on
discussing the technical characteristics of big data—Volume, Variety, and
Velocity (the “3 V’s”)—without an adequate emphasis on the challenges
associated with generating value from big data—Capacity, Capability, and
Culture (the “3 C’s”). We believe an evolutionary approach utilizing a series
of pilot projects supported by a network of key partners, with strong business
collaboration and positive feedback mechanisms, are necessary to address these
challenges and will adequately hedge investment risk while generating quick
returns. Additionally, there may be network effects and first mover advantages;
therefore, it is imperative that organizations begin this process now.
Drive Your Business
Starting Small with Big Data
A Pragmatic Approach to Generating Business Value
Strategy Brief | Big Data
1
There are five
disruptive trends
shaping the corporate
IT landscape today, and
of the five, Big Data has
the biggest potential to
generate new sustainable
competitive advantages.
Figure 1: Five Disruptive Trends
Responsibility for IT is Moving to the Business
Convergence of BPO and ITO
Big Data, Mobility and Analytics
Commoditization of IT and Global Delivery
Consumerization of IT
1
2
3
4
5
Source: WGroup
2. “While the market for
Big Data is real, it is
important to remember
the fundamental
differences between
Big Data Analytics
(BDA) versus traditional
analytics and how they
complement each
other. Both are needed
for a comprehensive
analytics strategy.”
Eric Liang
Principal Consultant
WGroup
Unique Characteristics of Big Data Analytics (BDA)
The market for Big Data is real; it is projected to be growing to $16B by 20151
,
equal to or more than a third the size of the ‘traditional’ BI (Business Intelligence)
and analytics market and at twice the speed. However, it is important to
remember the fundamental differences between Big Data Analytics (BDA) versus
traditional analytics, how they complement each other, and why both are needed.
Much has been said about the Volume,
Variety and Velocity of Big Data. While
a plurality of companies nowadays
typically deal with data sets exceeding
10’s of terabytes, analyses using
large data sets do not necessarily
leverage the uniqueness of Big Data
Analytics. We have segregated data-
driven analytics into three categories:
‘Simple’ problems use relatively simple
algorithms to manipulate (e.g., slice
and dice) small to moderately large,
structured data records (see Figure
2). This is the domain of traditional
BI. ‘Quant’ problems are those that
require highly specialized numerical
analysis to operate complex algorithms,
use intensive computational power
for a single solution, and the algorithms
will grab as little or as much data as
needed in the course of computation. Examples include DNA sequencing,
protein folding, nuclear physics and computational fluid dynamics in aerospace.
By contrast, true ‘Big Data Problems’ are those that use relatively simple
algorithms to mine, associate, or discover patterns from huge data sets that
include lots of unstructured data. Some problems, like security monitoring,
are inherently Big Data Problems because patterns can only be discovered by
examining voluminous data sets. Others, like Google Search, choose to use
large datasets and incremental learning algorithms in lieu of approaches that
use less data and more complex algorithms. Thus, BI focuses on individual
transactions, whereas BDA seeks to predict trends and anticipate opportunities.
Recognizing this distinction, the proper architecture for a company expanding
into the world of BDA would probably look like the diagram in Figure 3.
Big data analytical techniques include, for example, content analysis,
sentiment analysis, text analysis and natural language processing,
associative analysis, plus the ability to do predictive simulations
(predictive analysis) and return recommendations with low enough
latency to enable interactive decision-support (real-time analytics.)
1 D. Vesset et al, “Worldwide Business Analytics Software 2012-2016 – Forecast and 2011 Vendor Shares”, IDC
WGroup
2
Big Data
Problems
‘Simple’
Problems
Quant
Problems
Algorithmic Complexity
DataVolume
Figure 2:
The Three Categories of
Data-Driven Analytics
Source: Chris Swan, ”Big Data – a little
analysis,” Chris Swan Weblog, Apr 2012.
3. Implementing Big Data
Launching into Big Data is difficult given the large number of projects
and initiatives typically underway in most IT organizations coupled
with the perceived lack of benefits experienced from traditional BI and
Data Warehousing efforts. The challenges to successfully implementing
big data revolve around Capacity, Capability and Culture:
Capacity—Most IT organizations are operating under resource constraints.
Additionally, traditional BI and Data Warehousing efforts continue to drain
resources due to data issues, integration problems, etc., reducing their effectiveness
and perceived value.The appetite for adding new capacity is very limited, and
where approved directed towards addressing problems with current systems.
Capability—Even where there is excess capacity, the resident skills do not
match what is necessary to implement Big Data. Existing IT resources are
often unable to effectively assimilate leading-edge technical capabilities.
Brand new capabilities are also required—a combination of strong data
modeling, statistical analysis with business domain and process knowledge—
the so called “data scientists.” These resources are in short supply.
Culture—Lastly, cultural issues play a big factor in the ultimate success of
using Big Data. Managers are typically used to making decisions based on past
successes and subconsciously carry an intuitive model of how the business operates.
Intuitive insights may not be transferrable from one experience to another, or at
“The ‘Three C’s’ of
Capacity, Capability
and Culture are key
to understanding a
framework for a successful
implementation of big
data technologies.”
Eric Liang
Principal Consultant,
WGroup
Starting Small With Big Data
3
Figure 3: A Reference Architecture for Big Data Analytics
Source: WGroup
4. The real challenge
is to transform an
organization from an
intuition-driven decision-
making culture into one
that is data-driven.
the very least be only partially informed, especially in today’s hyper-competitive
environment subject to the interplay of macro-level trends. Intuition must be
backed up with data and facts. This requires a sea-change in behavior to get to
a more data-driven culture—using data to balance intuition and vice versa.
Given this understanding, we recommend an evolutionary approach to the
deployment of big data capabilities that targets investments, minimizes
risk, and captures value while building longer-term capabilities:
WGroup Approach to Big Data
1. Pick an analytics-friendly cross-functional team; challenge team to
identify top business opportunities based on Big Data concepts
2. Evaluate and prioritize top opportunities for piloting
3. Develop prototype process for implementation
4. Measure results; share experience and data sets
with other teams for process replication
Picking the right team to delve into BDA is important, because a key to
extracting Big Data insights is the ability to pair analytical skills with detailed
business knowledge in order to address relevant business problems in context.
Furthermore, the real challenge is to transform an organization from an
intuition-driven decision-making culture into one that is data-driven. An
analytics-friendly team would at least already have the beginnings of using
hypotheses-testing-feedback loops as a habit. To the extent that strong
statistical and/or data modeling skills are not available in-house (but are
almost certainly required for a BDA initiative,) an analytics-friendly team
would also be in the best position to acquire and assimilate such talents.
WGroup
4
Assess Indentify
Opportunities
Prioritize Select
Pilot Opportunity
Develop
Implementation
Plan
Implement
Assess Results
Prove Business
Value
Assess Current
Capability
Identify Opportunities
Evaluate ROI
Potential
Prioritize
Recommend
Identify Partners
Develop
Infrastructure
Develop BD
Analytics
Feedback
Learning
Figure 4: Evolutionary Approach to Building BDA Capabilities
Source: WGroup
5. Traditional BI3
Big Data Analytics
Rationalizing and reducing
operational costs
Trend sensing and operational log analyses from
numerous sensors enables predictive analytics
Improving the customer
management process
Customer profiling, segmentation sentiment
analysis based on more dimensions, leverages
unstructured data. Analysis is more granular and
fluid
Maximizing operational agility
Embedded BA with minimal latency enables
CEP4
(Complex Event Processing) and real time
responses
Enhancing business
performance alignment across
the enterprise
Variety of data sources for analysis present holistic
view for enterprise-wide strategic decision support
Avoiding unnecessary risk
exposure and ensuring
adherence to regulatory
compliance
Sophisticated pattern and anomaly detection
no longer hampered by small sampling sizes or
sample bias
3 Helena Schwenk, “Business intelligence and analytics fundamentals,” Ovum, July 2010
4 Nenshad Bardoliwall, “The Top 10 Trends for 2010 in Analytics, BI Performance Management,” Dec 2009
Being creative in coming up with business opportunities based on Big Data
concepts should not be hard, but to evaluate and project the value of such initiatives
would be. Here an important concept is the so-called ‘Return-On-Data.”
Traditional analytics, using structured, cleansed and carefully sampled data, are able
to extract useful insights out of a relatively small data set (insight per byte), yet the
cost of acquiring that pre-processed data is high per byte. By contrast, Big Data
tends to require large quantities of data to extract one insight, which is why it must
use techniques, algorithms and infrastructure (Hadoop-based distributed storage
and massively parallel processing) with low cost per byte to justify the economics.
Thus the key metric is to maximize the ratio of these two “per byte” numbers.
Once a pilot project from among the creative Big Data business opportunities is
selected, the next step is to select technology partners for both the infrastructure
and analytics tools. Typically, Big Data infrastructure choices are different than
traditional BI; e.g., direct-attached storage (DAS) and high capacity SATA
disks sitting inside massively parallel processing nodes are preferred over slower
shared storage. Simultaneously, a new process needs to be defined, starting with
how the data from varied sources are to be gathered, integrated and governed.
Finally, in the fourth step, the analytic models need to be deployed and
the business benefits measured. It has been reported that companies using
data-driven decision-making enjoy 5-6% boost in productivity2
. More
importantly, BDA initiatives should focus on extracting insights not
discoverable using smaller, structured data sets. Some typical BDA benefits
are contrasted with those obtainable from traditional BI in the table below:
2 Erik Brynjolfsson et al, “Strength in Numbers: How Does Data-Driven Decision Making Affect Firm Performance?” MIT, Apr
2011
Starting Small With Big Data
5
Akey criterion to the
evaluation of big data
initiatives is the so-called
‘Return-On-Data’ metric.
Cloud-based, massively
parallel processing and
storage have brought
this metric into viable
territory for Big Data sets.