1. 1
I
n today’s digital oilfield, big data is gath-
ered from many points throughout explo-
ration, discovery, and processing. This data
often is described by its volume, velocity, and
variety; all three attributes may be found
within the oil and gas industry. The industry
increasingly relies upon it for data analytics
used to drive strategic decisions. Such analy-
sis demands massive parallel and accelerated
processing capabilities enabled by advanced
hardware and software solutions.
IT organizations must be fully prepared
to deal with big data in order to compete
successfully. This report explores how Dell
solutions help meet evolving big data chal-
lenges throughout the energy industry now
and into the future. Technological advances,
including density-optimized servers with
leading edge co-processors, accelerators,
and open source software, can be used to
solve global challenges in E&P all the way
through to distribution in a highly competi-
tive international market.
The big data challenge for oil
and gas
The capture and analysis of big data needed
to drive decisions can be challenging. Some
of the challenges include:
Data utilization
When the research firm IDC Energy Insights
researched the topic of big data in the oil and
gas industry, 69.9 %[1]
were unaware of the
term. A better understanding of big data’s uses
and applications will help oil and gas industry
leaders build a successful competitive strategy.
Intelligent modeling
Resource discovery and processing relies on
big data for many valuable functions. For ex-
ample, it may be used to intelligently model
and image the Earth’s structure and layers
1,524 m to 10,668 m (5,000 ft to 35,000 ft)
below the earth's surface for exploration pur-
poses, by computing seismic wave data. Big
data also helps characterize activity around
existing wells, such as machinery perform-
ance, oil flow rates, and well pressures. With
approximately one million wells currently pro-
ducing oil and gas in the US, this dataset is
extensive, rapidly growing, and valuable for
maximizing the returns from these reserves.
Precise computation
Informed decisions are best made with care-
fully selected data. Accurate data capture and
presentation of information related to energy
discovery and E&P accelerates sound deci-
sions in the industry. Specifically, IT and the
right High Performance Computing (HPC) ar-
chitecture enable big data processing, speed
up the time-to-oil, and ultimately provide a
clear competitive advantage[2]
.
Exploration speed and focus
Faster and more accurate analysis of big data
promotes the speed of exploration and dis-
covery of new energy resources without com-
promising safety. This provides a more
focused path to exploration, discovery, and
production with lowest environmental
impact. Through the utility of remote-sensor-
ing devices and monitoring technology, infor-
mation may be captured by personnel and
equipment to drive better business decisions.
While the processing of big data has
made great strides in the past few years,
advances in the specific tools and architec-
ture needed to process big data are expected
in the years ahead. IDC Energy Insights ex-
pects the big data technology and services
September 2013
Big Data and the Digital Oilfield: Using Advanced Technology
to Gain a Competitive Edge
The age of big data is upon us.
Contributed by Dell
1
IT organizations must
be fully prepared to deal
with big data in order
to compete successfully. IDC Energy Insights
expects the big data
technology and services
market to grow to US
$16.9 billion by 2015.

2. market to grow to US $16.9 billion by 2015,
which would represent a compound annual
growth rate of 39.4% or about seven times
that of the overall information and commu-
nication technology market. [3]
Keeping up
with technology advancements, including
the storing and processing of big data, will
continue to be a necessary ingredient in stay-
ing competitive[4]
.
Enabling big data means enabling
time-to-oil
By building a network infrastructure with
the right servers and processors, the bene-
fit of big data can be realized and will
expedite energy resource discovery and E&P
in many ways, including:
• Throughput: Maintain high data
throughput to get results faster;
• Scalability: Accommodate the com-
plexities of seismic imaging and allow
the use of rapidly growing data sets in
data acquisition, processing, or inter-
pretation phases;
• Consolidation: Enable a common set of
management tools to consolidate
petabytes of data onto a single platform;
• Supportability: Provide standard proto-
cols to accommodate applications run-
ning on heterogeneous systems when
accessing the same data set; and
• Speed: Shorten and reduce the discov-
ery process by days and even weeks
through rapid analysis of data by the
petabytes.
HPC and the big data environment
Many oil and gas workloads take advantage
of massive parallel-processing technology to
speed up the time required to complete
complicated analyses. Leading-edge compo-
nent technology such as Intel’s Xeon Phi
co-processor and NVIDIA’s GPU accelerators
provide faster performance and operate eco-
nomically and efficiently with resources such
as open source software.
Co-processors supplement the primary
processors by offloading processor-inten-
sive tasks from the main processor to ac-
celerate system performance. Operations
performed by the co-processor may include
floating point arithmetic, graphics, signal
processing, string processing, encryption, or
I/O interfacing with peripheral devices. This
enables faster processing of data character-
ized by high velocity and volume, such as
real-time streaming data from drill heads
and equipment sensors.
Graphics processing units have highly
parallel architectures that make them more
effective than general-purpose CPUs for
algorithms where processing of large
blocks of data is done in parallel. They are
efficient at rendering the data required by
oil geophysical surveying, including seismic
geological modeling, improving resolution,
and processing seismic data in complex
surfaces or low signal/noise ratios. High-
density optimized infrastructure provides
the required performance and scalability
needed to keep up with these demanding
workloads. Servers, networking, and stor-
age are purpose-built and tuned for maxi-
mum performance and efficiency in these
data-intensive environments. I
Case study one: Enabling big
data volume[5]
Background: Network infrastructure is a criti-
cal capability for accommodating big data
volume.
The data volumes for seismic image pro-
cessing are very large. Single files can range
from 500 gigabytes to several terabytes (TB)
in size. Projects may range from 4 TB to 24
TB. The movement and storage of such data
creates congestion or gridlock that can cause
geoscientists and seismic engineers to wait
weeks or even months to access the results.
Problem: A company that processes
geophysical data for companies engaged in
oil and gas exploration faced a challenge in
their network infrastructure to make a high
volume of big data available to their clients.
The company routinely processes massive
amounts of seismic data so their clients can
determine where to drill the next well.
Infrastructure bandwidth is core to this
company’s offering. Network switching ca-
pability is key to moving data quickly
enough for end-users to rapidly process
seismic data for decision-making purposes.
However, the firm’s legacy switches did not
provide sufficient bandwidth to move data
fast enough.
Solution: With two different data centers
in disparate geographic locations, the com-
pany used one Dell Force10 C300 resilient
switch at each location. With up to 384 line-
rate 10/100/1000Base-T ports, coupled
with five microsecond switching latency
under full load for 64-byte frames, the chas-
sis-based switches provided the port den-
sity and throughput needed to ensure the
data processing performance demanded by
their client's end-users. With nearly 1,000
nodes at one data center, and more than
250 nodes at the other, the company was
able to leverage the switch’s backplane
capacity of more than 1.5 terabits and more
than 950 megabits per second of L2/L3
packet forwarding capacity.
Result: The company became more agile
and was able to offer its current and poten-
tial clients the capability to handle explo-
ration data. They were able to reduce their
cost-per-port and achieve a robust end-to-
end line speed with an attractive total cost of
ownership. In addition, the company had
planned for a 10GbE backbone in the future.
The new Dell Force 10 enabled such a mi-
gration seamlessly and averted any addi-
tional challenges to support it.
Case study two: A three-fold
increase in time-to-oil[6]
Background: Seismic data demands the right
tools and the right capability to handle the
huge volume and velocity of data generated
by research.
Problem: A company providing seismic
imaging and analysis service had a key
business opportunity to partner with an off-
shore vessel operator and collect seismic
offshore data. The company’s client sought
to improve its time-to-oil.
With this growing demand, the company
needed to shift its analysis of seismic data
to an offshore research vessel. This allowed
the company to process data quickly while
onboard without waiting until the vessel
returned to port. Its clients could then have
the information sooner, providing them a
significant competitive advantage.
2
Advancing HPC in the digital oilfield: four case studies

3. 3
The company’s proprietary software
required a high performance hardware plat-
form to operate, which was beyond the
capabilities of the ship’s existing infrastruc-
ture. The server capacity needed to conduct
analyses and operations was limited by the
physical space, power, and available cool-
ing aboard the vessel.
Solution: To maximize the onboard sys-
tem’s density and power efficiency, the
company chose Dell M1000 blade servers
with Intel Xeon Processor 5400 series. This
provided eight cores per blade and enabled
them to run their software while minimiz-
ing space requirements. The Intel Xeon
processors are designed to be energy effi-
cient and enabled the vessel’s engineers to
support the system without disrupting other
operations.
Result: The combined Intel Xeon quad-
core processors were more than twice as
fast as the legacy onboard dual-core
processors, allowing them to handle the
CPU-intensive demand easily. In addition,
the design of the Intel Xeon processor 5400
series incorporates a low-voltage power
supply but provides an increased process-
ing capability for facility with a reduced
power capacity.
A significant outcome of the platform
was an accelerated project timeline. The
new hardware configuration reduced the
total project time from nine months down
to six. This not only reduced cost for their
clients, but also provided these clients
a substantial head start on developing en-
ergy sources.
Case study three: Double the
process power while reducing cost[7]
Background: In geophysical surveying used
for oil and gas exploration, complex, com-
puter-generated subsurface maps can depict
geological surfaces in 3-D with detailed con-
tours and cross-sections. These visualizations
require increasingly complex and sophisti-
cated algorithms to generate accurate results
that can be used to assess the development
potential of claims.
Problem: China’s leading engineering
and technical services company BGP spe-
cializes in geophysical exploration and is
top-ranked globally for land-based seismic
exploration in its quest to identify energy
sources. This includes discovery of 260 oil
and gas fields of different sizes, more than
80 billion tons of geological oil reserves,
and 2 Tcm of gas reserves.
The company employs an advanced seis-
mic data analysis platform with many appli-
cations including seismic geological
modeling, pre-stack migration, 3-D visuali-
zation, and advanced resolution for pro-
cessing complex seismic data. Their
architecture and software enable sophisti-
cated model processing and data interpre-
tation. This platform provides more than
300 batch and interactive processing seis-
mic application modules in 20 categories.
In addition, it has the capability to support
onshore and offshore 2-D and 3-D seismic
data processing and advanced 3-D seismic
visualization.
The company’s previous UNIX-based
processing system struggled to provide the
needed performance, cost, and hardware
expandability and software compatibility
with new and advanced applications. It
needed the very best configuration to meet
the domestic demand for oil and gas explo-
ration in China and to compete in the inter-
national marketplace.
Solution: To meet its next-generation pro-
cessing requirements, the company moved
to an x86-based platform that implemented
the Intel Xeon E7 processor series. The
Xeon-based servers delivered strong pro-
cessing performance, supporting 80 cores
and 160 threads, and memory capacity as
high as 4 TB, a top choice for geological
data analysis systems.
Result: The new architecture cost 46%
less than the existing Unix system and in-
creased process time by as much as 49%.
The company was able to double its pro-
cessing capability, reduce its cost to achieve
this, and create a technical capability to be
much more competitive in an increasingly
demanding and complex exploration envi-
ronment. (Intel Inc., 2012)
Case study four: A hundred-fold
increase in HPC performance[8]
Background: Complex seismic algorithms
and acquisition techniques have acceler-
ated the demand for HPC capacity and per-
formance that can meet the demands of big
data. The challenge is to strategically maxi-
mize the value of HPC to deliver the most
business value.
Problem: A multinational oil company
relies heavily on HPC for the success of its
operations and technical functions. The
company set a long-term goal to achieve a
hundred-fold improvement in technology
performance, meaning if the technology de-
livers a hundred-fold gain in performance,
the cost must also be less than 100 times
the expense.
Solution: The company formed and de-
veloped an HPC Center of Excellence fo-
cused on collaboration with internal IT and
operations teams and the suppliers and ven-
dors who help to serve the company’s HPC
needs. This was a strategy to find and build
solutions to their needs of applications, data,
storage, network capability, and visualization
technology to deliver the business value and
meet market demands into the future. More
accurate seismic imaging, data center con-
solidation, customized engineering models,
software optimization, and the deployment
of advanced computing hardware are all
part of the collaborative efforts put in place
to reach this goal. All of these delivered sub-
stantial performance improvements that
have put the company well on its way to
achieving its hundred-fold goal.
Result: The company’s custom-devel-
oped reservoir engineering model tripled its
HPC computing performance, with work-
flows achieving significantly higher increases.
Its software optimization improved by as
much as twelve-fold and its HPC capacity
doubled. In addition, internal teams have de-
veloped and optimized higher performing
versions of critical software applications, pro-
ducing sophisticated and cutting-edge algo-
rithms. In the company’s Gulf of Mexico
(GoM) exploration, its optimized processes
for data storage and processing enabled the
company to make its projects larger and
denser. This reduced the cost by a multiple of
two-and-a-half times what it was 10 years
earlier. (Intel, Inc., 2012)
Open-source software provides
flexible options to manage big data
The growing availability of high-quality open
source software enables developers to use
code that can be tailored to fit specific com-
puting needs without incurring the expense
of proprietary licensing costs. Open-source
software is flexible in its use and can be mod-
ified to work with homegrown applications
and commercial applications. The oil and gas
industry is using open source applications,
particularly those used for data analysis.

4. 4
Hadoop is one example of an open-
source application that is gaining popularity
as a cost-effective tool to process and store
large data sets across commodity server clus-
ters. According to analysts at TechNavio
research, the global Hadoop market is set to
grow 55.63% from 2012 to 2016. According
to the research, the main driver of this market
is the growing demand for big data analytics.
[9]
Many oil and gas companies have used
Hadoop as part of their HPC operations.
Case study five: Chevron uses
open-source software for energy
exploration
Problem: Chevron’s oil exploration efforts
are comprehensive and include the move-
ment of drillships into areas such as the Gulf
of Mexico to survey targeted areas where oil
or gas deposits may be located. A research
ship can cost the company nearly $1 million
per day to be on task. The technology chal-
lenge is to capture and process the highest
quality of data possible while controlling the
cost of exploration.
The drillship crew sends seismic waves
to the ocean floor and measures the re-
flected waves as they bounce back to the
ship’s array of receivers. The array measures
the amplitude and arrival times of the re-
flected waves to build a time series that can
then be analyzed to create a simulated
graphic model of the ocean floor that indi-
cates potential locations of energy re-
sources. The amount of data generated via
the seismic waves is overwhelming.
Chevron collects and analyzes seismic in-
formation that contains five dimensions.
Processing this level of complex data and
producing an accurate visual simulation re-
quires a supercomputer with extreme com-
puting horsepower.
Solution: Chevron takes 25 steps in pro-
cessing the seismic data to create a visual
model for engineers to locate potential oil
reservoirs. Hadoop is used to sort the data
and create models and simulations of the
underground environment.
The company claims that Hadoop costs
one-tenth of the commercial software solu-
tion previously used. The use of Hadoop
reduced the total cost of exploration, and
the company was able to achieve its objec-
tives seamlessly with its use. Open-source
software was a key advantage for Chevron
and is an option for other scientists and
developers in the industry.
The oil and gas industry works within an
environment increasingly populated by big
data. The digital oilfield will increasingly
rely on efficient processing of such data,
allowing firms to stay competitive by
improving time-to-oil. The challenge of
successfully processing big data for discov-
ery, exploration, and development pur-
poses is formidable and demands
precise-fit, high-performance computing
solutions. The careful combination of these
advanced technology solutions, which
includes hardware infrastructure based on
a new generation of density-optimized
servers, accelerators, and co-processors, as
well as open-source software, will allow oil
and gas firms to best compete and stream-
line time-to-oil.[10]
I
F
or more than 26 years, Dell has empowered countries, com-
munities, customers, and people everywhere to use tech-
nology to realize their dreams. Customers trust Dell to deliver
technology solutions that help them do and achieve more,
whether they're at home, work, school, or anywhere in the world.
Seismic processing, simulations, and other data-intensive
computing require exceptional server performance, bandwidth,
and efficiency. To support the massive volume of computations
required, Dell PowerEdge servers incorporate the latest Intel
Xeon series processors. With expanded memory bandwidth and
channels and integrated I/O to help reduce latency by up to
30%, these processors deliver more than 80% more perform-
ance than previous-generation Intel series processors.
The PowerEdge C-Series servers offer a streamlined approach
for targeted hyperscale environments. For these servers, Dell has
removed the redundant hardware, broad OS support, and same-
day parts replacement that these organizations do not need,
helping provide the requisite performance levels in dense,
energy-efficient configurations. These servers also allow organi-
zations to gain fast access to emerging technology instead of
waiting for customized solutions or traditional general-purpose
servers (with their additional features and extensive OS support).
IT managers can gain further cost efficiencies by moving to
computing platforms based on open source applications, shared
infrastructures, and today’s most advanced processors, network
connectors, and serviceability.
Dell’s HPC and big data solutions are supported by Dell’s
broad portfolio of planning, implementation, and maintenance
services. Those professional services can help oil and gas firms
accelerate IT initiatives, manage the complexities of an HPC
environment, and accelerate time-to-oil in their vital exploration
activities. Dell services can be tailored to meet specific require-
ments and can include data center consulting, custom rack
integration, online solution support, and other solutions.
About Dell’s advanced technology solutions for HPC environments
To learn more, contact Dell’s Data Center Solutions and/or High Performance Computing specialists at Dell.
www.dellhpcsolutions.com • www.dell.com/poweredgec
References:
[1] Rick Nicholson, Big Data in the Oil and Gas Industry, IDC Energy Insights, (https://www-950.ibm.com/events/wwe/grp/grp037.nsf/vLookupPDFs/RICK%20-%20IDC_Calgary_Big_Data_Oil_and-
Gas/$file/RICK%20-%20IDC_Calgary_Big_Data_Oil_and-Gas.pdf).
[2] Adam Farris, “How Big Data is Changing the Oil & Gas Industry,” Analytics Magazine (http://www.analytics-magazine.org/november-december-2011/695-how-big-data-is-changing-the-oil-a-gas-industry, (Nov-
Dec 2011).
[3] Jill Feblowitz, The Big Deal About Big Data in Upstream Oil and Gas, (http://www.hds.com/assets/pdf/the-big-deal-about-big-data-in-upstream-oil-and-gas.pdf).
[4] Steven Croce, Brandon Draeger, and Buck Avey, “Designing for Hyperscale Computing,” http://www.dell.com/downloads/global/power/ps2q10-20100360-cloud.pdf
[5] Dell Customer Stories: Tricon Geophysics, Inc., http://www.dell.com/Learn/us/en/uscorp1/corporate~case-studies~en/Documents~2011-tricon-10010209.pdf
[6] Oceangoing Multicore, http://www.geotrace.com/news/pdf/Intel-ESS-GeotraceCaseStudyLR-Web.pdf
[7] Cutting Edge Tool For Exploring the Earth, http://www.geotrace.com/news/pdf/Intel-ESS-GeotraceCaseStudyLR-Web.pdf
[8] Shell Drills Down In Improvements in High Performance Computing, http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-shell-paper.html?wapkw=oil+and+gas
[9] Big Data Goes Hadoop, March 21, 2013, (http://www.giiresearch.com/press/7737.shtml).
[10] Rachel King, Chevron Explores Open Source Using Hadoop, CIO Journal, June 5, 2012 (http://mobile.blogs.wsj.com/cio/2012/06/05/chevron-explores-open-source-using-hadoop/).