Does Big Data Spell Big Costs- Impetus Webinar

Impetus Technologies Inc.
Does Big Data Spell Big Costs?
1 © 2014 Impetus Technologies
Recorded version available at
http://www.impetus.com/webinar_registration?event=archived&eid=56

Outline
• Big Data – Current Scenario
• Cost components in a Big Data Warehouse
• Best Practices - Reducing the cost of Big Data solutions
– Cost of storage
– Technologies- What and Where?
– Big Data strategies
– Our recommendations to reduce TCO

Big Data – Current Scenario
2.5 Quintillion Bytes produced every day
$6 Trillion Big data cost IDC/EMC
$650 Billion
Cost of wasted productivity because of information
overload
1ZB Estimated Internet Traffic by 2015
1800EB Size of the digital universe in 2011
90%
90% of the data in the world today has been created in
the last two years alone
18 Months Estimated time for the digital universe to double

Age of Data
Age of Software Age of Data

Existing State of Big Data
Solution

Using Commodity H/w for Big Data
Commodity Hardware
• Pros
– Build your own
– The promise of innovation
• Cons
– Building reliable storage – $1 per GB
– Add the cost of managing / monitoring / hosting

Using Open Source & Cloud Computing
Open Source
– Pros
• Software is free !! Glory to the Elephant
– Cons
• Cost of Training – thinking parallel is not intuitive
• Cost of Support – support is not free
Cloud Computing
– Pros
• Rent what you need
– Cons
• $14,000 a month for 100 TB data – storage only

Big Data Warehouse- Cost Components
• Initial entry costs- Cost of experimentation
• Cost of integration and moving data - Cost of ETL
• Query and analytics capability
• Manageability
• On-going maintenance - Monitoring and tuning
• Changing capacity - Additional hardware
• Cost of compliance

Lowering TCO of Big Data
• Hardware
– Lower cost of storage
– Lower cost of computation
• Software
– Make things faster
– Do more with less

How to reduce the cost of storage?
– Compress – RainStor and similar solutions
• Just make sure your ‘Read Throughput’ is high
– Retain all v/s load & process
• Setup “data pipelines” or use ILM Principles
• Creation and Receipt
• Distribution
• Use
• Maintenance
• Disposition
– Focus on Big Data but don’t forget the “Small Data”

Technologies: What and Where?
What?
• Open Source vs. Commercial software?
• Specialized hardware/appliances vs. commodity
hardware?
• Vendor lock-in vs. vendor independence?
• Cost of latencies?
• Cloud?
Where?
• OLTP - NoSQL v/s OLAP - DW (MapReduce & MPP)

OLAP: Big Data Scenarios

Data Tapping Point, Cost & Latency

Indirect Analytics over Hadoop

Direct Analytics over Hadoop

Analytics over Hadoop with MPP DW

Selecting the Right Technology
Key considerations
– $ per TB
– Business Continuity/ Cost/ Vendor Lock-in
– Latency Needs

Choosing MPP
$ per TB Driven
– EMC Greenplum
– Teradata, Aster
– HP Vertica
– Oracle Exadata
– Netezza
– ParAccel
– Others

Faster Map Reduce & Hadoop
Business Continuity/ Cost/ Vendor Lock-in
– MapR
– HPCC
– Hadapt
– Pervasive DataRush, HStreaming
– Cloud Map Reduce
– DataStax
– Platform Computing
– MARS, GPMR
– ParStream

OLTP: NoSQL Solutions
Latency Needs
• Column stores
– HBase, Cassandra
• Documents stores
– MongoDB, CouchDB
• Key stores
– Redis, Riak etc.; Kyoto Cabinet/Tokyo Tyrant, Berkley
• GraphDB
– Neo4j
• Cloud stores
– SimpleDB

OLTP: New Era RDBMS Version
• Postgres, InfiniDB, Infobright
• MySQL Cluster
• GridSQL, EnterpriseDB
• MS SQL
• Sybase IQ
• Specialized stores
– VoltDB, MarkLogic, Clustrix
• Xeround
• ParStream
• Oracle NoSQL

Recommendations- Cost Components of Big
Data Warehouse
• Initial Entry Costs - Cost of Experimentation
We recommend – Follow Best Practices , Learn or Hire
• Cost of Integration and Moving Data- Cost of ETL
We recommend - Remove costly licensed tools, switch to Map Reduce
for ETL or ELT
• Manageability - Provisioning, management tools
We recommend – Opt for multi-vendor management toolsets,
e.g. Impetus Ankush
• On-Going Maintenance- Monitoring and Tuning
We recommend – Automate! Automate! Automate!
• Changing Capacity - Additional Hardware
Do you know the GPU?

Recommendations- Hardware & Software
• Cost of Storage- Compress Data
We recommend – Opting for RainStor/ similar solutions
• Do More with Less - Faster MR
We recommend – MapR/ similar solutions
– Acunu and related solutions for NoSQL

About Impetus

• Strategic partners for software product engineering and
R&D
• Thought leaders in cutting-edge technologies
• Mature processes and practices that are methodical, yet
flexible
• Diverse domain expertise
Our Services in Big Data and Analytics
Expert Consulting
Proof of Concept & Implementation
Support Services

Big Data Quick Start Program
Three Modules
• Gear up (1 day session)
• Base Camp (4 day session)
• Summit (5 day session)

Q & A

Thank You
Write to us at inquiry@impetus.com
Follow us on Twitter @impetustech

Does Big Data Spell Big Costs- Impetus Webinar

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (19)

Destaque

Destaque (6)

Semelhante a Does Big Data Spell Big Costs- Impetus Webinar

Semelhante a Does Big Data Spell Big Costs- Impetus Webinar (20)

Mais de Impetus Technologies

Mais de Impetus Technologies (20)

Último

Último (20)

Does Big Data Spell Big Costs- Impetus Webinar