The ability to rapidly analyze Big Data has become a key competitive advantage to manage the
exponential growth of globally available information. Early adopters have already proven benefits both
on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to
leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has
coordinated several hundred Big Data projects for a major solution supplier, always supporting the
fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big
Data and to disclose its full potential by giving strategic guidelines as well as execution support.
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Big Data - Big Benefit or Big Waste?
1. PERSPECTIVE
2012
1
Big Data - Big Benefit or Big Waste?
Your Big Data strategy will determine your success
Big Data – Blessing or Curse?
Google, Amazon and Facebook, three of today’s
most powerful companies, have been built around
Big Data. The information that flows through their
veins is directly linked to their ability to analyze vast
amounts of data. Exactly this capability makes them
market leaders with a position that is hard to attack
for any competitor. No surprise that by now, other
companies across all industries have likewise started
to gather information in hope to boost their business.
T-Mobile USA is only one of many examples for
which these efforts have already paid off. By
analyzing huge amounts of data in real time with
SAP’s new in-memory database HANA, it is now
possible to provide targeted offers to more than 21
million customers. This results in $10-25 savings for
each won back subscriber and potentially in billions
of additional revenue per year. In the saturated
telecommunications market this will be a crucial
advantage against the competitors.
On the flipside, companies missing this analytical
competence will see their market position significantly
weakened with the threat of an attack of a much
better prepared competitor always lingering around
the corner. Since the amount of digital data doubles
every five years according to recent calculations, this
effect will further increase. By 2012 we have already
been faced with about 2.6 Zettabytes of global data.
Assuming that a human being is able to memorize
100 GB of data, it would need four times of the entire
global population to store this information. With a
CAGR of almost 45%, in 2013 already six times of
the world population would be needed.
This trend is boosted by the increasing distribution of
networked devices and sensors in our daily life, smart
phones, internet and the social media platforms. The
key challenge is that the amount of data companies
need to evaluate in acceptable time is nowadays
growing faster than the performance of established
database technologies and analytical tools (compare
Figure 1). In addition, available data becomes more
and more semi-structured (XML) and unstructured
(documents, e-mails, videos, pictures, etc.) and
hence more difficult to analyze.
Figure 1: Growing “amnesia” of enterprises
1960 1970 1980 1990 2000 2010 2020 2030
1,000,000
1,000
1
1,000,000,000
Analytical
Speed1)
1) # of Data Units that can be evaluated in acceptable time and money
2) # of company’s Data Units to be evaluated to stay competitive
Source: xCon Partners analysis
Evaluation
Requirements2)
World Wide
Web
Email
PC
today
Enterprise Social
Media
Internet of
Things
# of Data
Units
Data amount
exceeds analytical
performance
resulting in amnesia
The ability to rapidly analyze Big Data has become a key competitive advantage to manage the
exponential growth of globally available information. Early adopters have already proven benefits both
on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to
leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has
coordinated several hundred Big Data projects for a major solution supplier, always supporting the
fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big
Data and to disclose its full potential by giving strategic guidelines as well as execution support.
2. PERSPECTIVE
2
To avoid resulting amnesia, companies have to either
dispose part of potentially valuable data untouched or
switch to new innovative analytics techniques.
Therefore it is essential for executives to define the
optimal Big Data strategy tailored towards their
company’s needs.
Big Data for Big Benefits
Even though data intense industries such as
telecommunication, (multi-)media as well as the
banking and insurance sector face the highest
obvious demand for Big Data solutions, basically all
industries and companies can take advantage of the
new and improved analytic possibilities. One
compelling success story is for example written by
the retail industry which is characterized by a lot of
customer interactions. After putting new in-memory
analytics technology in place, the luxury label
Burberry is now able to access millions of records
from multiple sources (customer data, stock
information, social media, etc.) nearly in real time.
This was made possible due to a speed increase
factor of 14,000 – requests now take one second
instead of previously nearly five hours. The real-time
analytics system is used to profile customers,
allowing making tailored customer offerings on the
fly. Hence, it allowed Burberry to implement a very
efficient customer-oriented sales strategy and to build
up a consistent global brand.
Certain cross-industry effects related to Big Data and
new analytics technologies are affecting all types of
companies:
Fact based decisions instead of relying on
assumptions: For business critical decisions, all
historical facts and current events can be
considered instead of relying on estimates and
assumptions.
Interactive scenario simulation: Calculate the
impact of alternative scenarios in almost real time
to assess risks and fine tune the chosen model.
This is especially interesting for price
optimizations which can now be based on all
available historical data.
Real time customer interaction support: Always
having an up-to-date customer behavior profile
available will create a very personal customer
experience and enable you to react in an optimal
way.
Micro personalization: Tap directly into your
customer base by defining very precise customer
segments to run individual micro campaigns and
to measure their success.
New business models: Real-time analytical
capabilities can also be used to access emerging
markets. New sensors like smart phones or smart
meters will for example enable new business
models around location based services or home
automation.
Even in the field of crime reduction, Big Data is
continuously gaining importance: Fraud Mining
applications are used by international finance
institutions with high transaction rates for systematic
identification and prevention of fraud. JP Morgan is
for example successfully deploying this technology in
numerous projects for identifying potential fraud
among traders.
Another example in this field is the joint venture of
Paymint AG and Fraunhofer IAIS that successfully
developed and implemented the application MINTIfy.
This solution protects millions of European credit card
accounts from fraud by analyzing thousands of
attributes in the transaction history and identifying
conspicuous patterns.
Different branch, similar problem: TomTom Business
Solutions, a commercial vehicle fleet specialist, is
Are you faced with Big Data?
Benefiting from Big Data requires analyzing huge
amounts of data from different sources at very
high speed. An exact quantitative definition of Big
Data is difficult since computing power is
constantly growing. For example the amount of
data used by the NASA in 1969 for the moon
landing operation can nowadays easily be
handled by a modern smart phone. Therefore we
recommend this pragmatic quantification: Big
Data cannot be captured, stored, managed and
analyzed by the commonly used software and
hardware within a tolerable period of time. In this
sense the magnitude is specific for your business
model and your Big Data capabilities. A good
indication on the criticality for your business can
be derived by assessing the following three V’s:
Volume: What amount are you faced with?
Variety: How diverse is your data?
Velocity: How fast do you need to analyze it?
3. PERSPECTIVE
3
facing over 1.5 billion real-time notifications from
175,000 vehicles and more than 1 billion requests per
month. Fujitsu accepted the challenge and
implemented an Oracle based solution which is now
able to handle 200,000 input-output-operations per
second with response times of under one millisecond.
This grants fleet managers and transport planners
access to real time information to optimize the routing
of their vehicles and to reach a higher fleet utilization.
In-Memory, Scale Out or NoSQL?
Once the need to handle Big Data is evident, the
question of the best IT solution(s) arises. To make
use of the company’s Big Data, many different
technologies (e.g. in-memory computing) and
methodologies (e.g. visual analytics) need to be
evaluated to define a comprehensive Big Data
strategy. Depending on the business problem, the
currently most promising technologies to significantly
speed up and broaden data evaluations are in-
memory computing, a horizontal scale-out approach,
NoSQL databases, or a combination of those. The
best fitting technology always depends on the
company’s specific business context.
The “trick” of in-memory technology is to avoid slow
hard disk access by constantly keeping all relevant
data in RAM. As the average access speed between
RAM and disk differs roughly by a factor of 100 to
1,000, data operations can be performed significantly
faster. Obviously, the speed increase depends on the
specific business context and the chosen
implementation. No wonder that currently almost all
of the leading global enterprises evaluate this
technology. In addition, in-memory solutions are often
combined with other powerful concepts to further
boost their performance. Some database vendors for
example use their know-how of specific applications
to optimize the underlying data structure: by using
columnar architectures, only columns that contain the
necessary data to determine the answer are being
processed which leads to significantly faster
response times. Additionally, advanced data
compression capabilities are applied to reduce the
size of information that needs to be stored and
analyzed. In-memory technology “standalone”
currently works best with relational databases and
structured data, providing instant results.
Speed improvements with even higher factors than
1,000 are possible: Automotive Resources
International (ARI) is now able to analyze millions of
data points collected from approx. 923,000 vehicles
3,600 times faster than with traditional technologies.
After a three week implementation of the SAP HANA
data mart solution, the company now benefits from
5% cost reduction in total overhead expense and
from increased contact center performance.
Currently, the in-memory market is still dominated by
data warehouse projects aiming to provide optimal
management decision support (see Figure 2). But
almost as many companies gather their own
experience by experimenting with custom
developments. We expect that some of the most
radical game changing innovations will arise out of
these initiatives.
Figure 2: Market split of current in-memory activities
Complementary or as an alternative to in-memory
computing, Hadoop is one of the most prominent
frameworks to implement scale-out scenarios.
Developed by the Apache group, it allows for the
distributed processing of large data sets across
clusters of computers using simple programming
models. It is designed to scale out from single
servers to thousands of machines, each offering local
computation and storage. At the core of Hadoop is an
implementation of the “MapReduce” algorithm: first,
the “Map” function divides the original query into sub-
segments and calculates their results on any number
of distributed nodes. Second, the “Reduce” function
centrally aggregates these intermediate results and
returns the answer. A whole set of additional Hadoop
components supports the seamless integration into
the enterprise environment: Flume and Sqoop help
with data population, Mahout encapsulates data
mining capabilities, Hive and Pig assist with query
generation. This framework is already used by most
of the leading online companies like Google,
Amazon, and Facebook for searching and analyzing
On-Demand
Solutions
Data
Warehouse
Custom
Development
Applications
Standardized
Configurations
Side-by-Side
Acceleration
Source: xCon Partners analysis
4. PERSPECTIVE
4
their data. Scale-out technology works especially well
with unstructured and semi-structured data.
NoSQL (Not Only SQL) databases are a good
alternative if Big Data performance and scalability is
most important and 100% consistency is not
required. Compared to the sophisticated relational
databases, they are better suited to handle large
volumes of multi-structured data. The biggest
disadvantage is that the majority of NoSQL
implementations no longer required transactions to
be ACID (atomic, consistent, isolated, durable). This
is one major difference to the in-memory databases
and disqualifies this technology for online-
transaction-processing that does not compromise on
data accuracy. Besides the most prominent NoSQL
implementations BigTable (Google) and Dynamo
(Amazon), it is to mention that with HBase, a version
exists which is especially meant to be deployed on
top of HDFS, the Hadoop Distributed File System. By
allowing low-latency lookups in Hadoop, it combines
these two Big Data technologies.
In certain cases, also Hadoop and in-memory
technologies need to be combined to achieve the
desired computing power. One example is a new
cancer research solution realized by Charité. The
university medical center has proven that it is
possible to reduce the time required to analyze a
tumor by a factor of 1,000, reducing it from hours
down to a few seconds. This offers the possibility to
adjust cancer treatments before the patient leaves
the hospital.
Increasingly, new players position themselves
successfully and offer their own Big Data solutions:
the online retailer OTTO was seeking for a possibility
to improve its warehouse management and its sales
forecast system. The predictive-analytics software
NeuroBayes by Blue Yonder was applied here,
handling over one billion forecasts in a year. The self-
learning system is able to process over 135 GB or
300 million of daily new data sets, boosting the
forecast efficiency up by 40%.
At the same time, established vendors like SAP or
Oracle develop a whole set of new in-memory based
applications that will set the industry standard in the
mid future. Some early adopters and co-innovators
have already migrated.
Additionally to the described major Big Data
technologies, there are various new and innovative
analytic solutions being developed, tailored
specifically towards Big Data. Those analytic
solutions are usually combined and sit on top of one
or several of the described Big Data technologies.
Due to the high number of different approaches,
describing them would go beyond the purpose of this
Perspective and needs to be evaluated individually
for the specific business purpose.
Big Data Market Overview
The Big Data market size strongly depends on the
market definition. Sticking to the Big Data definition
provided above, we size the Big Data market
(Software, Hardware and Services related to and
used by In-Memory, Scale-Out, and NoSQL
technologies) for 2012 at about 10 billion Euros.
Looking in the past and the years to come, we predict
a CAGR of 40% for the upcoming five years which
brings us to a market size above 50 billion Euros in
2017. Major driver for this significant market growth is
the explosion of data due to increasing use of social
media, mobile devices and the internet of things.
There are more than 100 players active in the Big
Data market with solutions tailored specifically
towards Big Data. Figure 3 provides xCon Partners’
view on the most important players in the Big Data
market. Naturally, this matrix is frequently subject to
change due to the fast moving and dynamic market.
Figure 3: Big Data Market Overview
Big Data Capability2)
Challengers Leaders
High Potentials Innovators
SAP
Oracle
Microsoft
Google
Amazon
IBM Netezza
HP Vertica
Teradata Aster
EMC Greenplum
CurrentMarketPosition1)
Cloudera
1010Data
SAS
10gen
MapR
Hortonworks
VMware
1) Besides Big Data market share considers also market share in
DBMS, Data Warehouse, BI, Enterprise Process Management
2) Considers completeness (In-memory, Hadoop, NoSQL, etc.),
maturity and vision of Big Data solution
Source: xCon Partners analysis
FacebookKognitio
ParAccel
MarkLogic
5. PERSPECTIVE
5
To determine the “Current Market Position”, besides
the still very volatile Big Data revenue, also market
shares in related and established markets like
Database Management Systems (DMS), Data
Warehouses (DW), Business Intelligence (BI), and
Enterprise Process Management (EPM) have been
considered. The “Big Data Capability” is not only
determined by the completeness of the offered
solution, but also by its maturity and the vision of the
vendor.
The seven biggest vendors with complete offerings
for In-Memory or Scale-Out Solutions in the Big Data
market are currently IBM Netezza, Oracle, Microsoft,
SAP, HP Vertica, Teradata Aster, and EMC
Greenplum.
The market position of those seven vendors is on the
one hand threatened by the innovators which are
mature internet companies that have developed their
own and very sophisticated Big Data solutions like
Google, Amazon or Facebook. On the other hand,
there are many emerging players entering the market
with new and innovate Big Data technologies and
solutions.
Companies need to select the most suitable solutions
and best positioned vendors carefully to not run into
the trouble of discontinued products.
Rely on xCon Partners’ Experience
Since 2011, xCon Partners has been involved in and
coordinated several hundred Big Data Projects,
always supporting the fast and smooth project
execution. Therefore, we are your ideal partner for
tackling the involved challenges and for disclosing
the full potential. Besides strategic recommendations,
we also support our clients during the execution to
ensure successful implementations. From having
analyzed hundreds of slipped projects we can give
indications on common implementation risks and help
to set realistic project targets and roadmaps.
From our experience, the Big Data topic is best
addressed with the following 5-step approach (see
also Figure 4):
1) Conduct a Big Data readiness check to
derive a clear picture of available information,
data sources and own analytic capabilities
2) Calculate the business potential for
enhancements to the current business model
and for new business opportunities including
a cost-benefit check
3) Understand which technology and solution is
best suited to implement the Big Data
strategy
Figure 4: xCon Partners Big Data Services
xCon Partners Big Data Services
Big
Data
Strategy
Implement
and Track Under-
stand
Technology
and
Market
Big Data
Readiness
Check Calculate
Business
Potential
Plan
Implementation
Define roadmap and project plan
System & integrator selection
Technology assessment
Overview on Big Data market
(vendors & service providers)
Transparency on current
market adoption of different
solutions
Support proof of concept
activities
Cost-Benefit analysis
Analyze available information
and additional data sources
Evaluate current capabilities for
Big Data analytics
Benchmarking
Analyze potential
enhancements to current
business and identify new
business opportunities
Define additional
information requirements
and analytics capabilities
Estimate effort and
business potential
Project management
Expert insights based on
400+ tracked active
Big Data projects
Avoid project delays
Measure success
1
2
3
4
5