Hadoop: What's Next? Mike Olson CEO, Cloudera

1. Hadoop: What’s Next?
Mike Olson

2. Reflections On You
12+ months using on average
114.5TB average size
66 average nodes in Use
500+ certified on Hadoop in 1 year
60+PB Total
Data from pre-conference survey

4. Immutable Law of Data
RDBMS
Hadoop
Volume, Variety, Velocity increase

5. Immutable Law of Data
RDBMS
Hadoop
Volume, Variety, Velocity increase
Geopbytes
Brontobytes
Yottabytes
Zettabytes
Exabytes
Terabytes

6. Linked
Complex
Unstructured
Pre-relational
Raw
Detailed
Heterogeneous
Dirty
Graphs
Large
Schemaless

7. Hadoop Was Built for Data.

8. Proven at Scale

9. Room to Grow

10. Open Source Wins.

11. Hadoop: The Core of a Platform

12. A Platform Built by You
Hue Hue SDK
OozieOozie
HBaseFlume, Sqoop
Zookeeper / Avro
Hive
Pig/
Hive

13. The Vendor Ecosystem

14. A Platform Enabling Applications…
Query &
Reporting
Complex
ETL
Trade
Compliance
POS
Analysis
Search
Quality
Click Stream
Analysis
Machine
Learning
Graph Analysis
And
More…
Fraud
Detection
Archive
Scientific Security

15. Solving Critical Business Problems
• Modeling true risk
• Customer churn
analysis
• Recommendation
engine
• Ad targeting
• PoS transaction analysis
• Analyzing network data
to predict failure
• Threat analysis
• Trade surveillance
• Search quality
• Data “sandbox”

16. • Capture critical IT data
• Monitoring usage
• Driving bottom line value

17. • Risk analysis
• Customer insight
• Drive growth

18. • Customer intimacy
• Precision targeting
• Driving top line growth

19. So Much To See Today!
• Optimizing search
• Advanced analytics in the Army
• Using Flume &Hive for log data
• Analyzing VOIP data with R

20. What’s Next?
Market
• Adoption
• Agility
• Flexibility
Technology
• Accelerated innovation
from community
• More tools e.g., monitoring
• More automation
• More stability
• More interfaces

21. • At the core of the open source platform for
data
• Four years old and going strong!

23. Organizational Impact
• More knobs and dials
• Fine grain control
• Achieve previously impossible /
impractical
• Save money
• Save time
• Greater flexibility with data
Copyright 2010 Cloudera Inc. All rights reserved

24. Hadoop World Keynote (NOTES)
• Themes
– Hadoop is already a big deal
• Keep in mind the why
• Solving real problems now
– It is about the platform with Hadoop at the
core
• Why
• Helps you profit
• More accessible now than ever, real people with
enterprise ops and enterprise skills, no longer the
exclusive demand of the PhDs
– What’s on the Horizon for Hadoop
Copyright 2010 Cloudera Inc. All rights reserved

25. Hadoop is Having a
Transformative Impact (notes)
• Continued growth and excitement
• Transformative to your career, your enterprise, your market
– Star maker
– Get ready for Hadoop being a big deal for your companies
– Your market – hyper personalization
– Use data to interact in a more customized fashion
– “It’s hard not to have a TB of data” – Mike
– Operability and SLAs for a critical enterprise platform
– Education and training
– A new stack for analytics (CEP (flume) CDH (Sqoop) dbms/BI)
• Future is now
– Use cases now and impact it is having and where it will be, look at
Facebook, Yahoo, eBay etc.
Copyright 2010 Cloudera Inc. All rights reserved

26. What is on the Horizon for
Hadoop (notes)
• Continued growth and excitement
• Transformative to your career, your enterprise, your market
– Star maker –
• good for your career, help make critical changes in the way customers are supported, major new business opportunities etc.
• Pull cloudera certification #’s
– Get ready for Hadoop being a big deal for your companies
• Enterprise will be more agile and able capture and analyze more data to better target ads, find fraud, etc.
• Agility – impacts the things that matter to you
• What’s happened before the transaction
– Your market – hyper personalization
• 100s’s of vertical apps to be created (developers are you listening?)
• Trend that crosses? Any other trend we can compare to? DBMS growth? Improvements in operations,
• How detailed sources have changed
• Devices, understanding how people interact with your business – retail, online entertainment, fin serv, government
– Use data to interact in a more customized fashion
– “It’s hard not to have a TB of data” – Mike
– Operability and SLAs for a critical enterprise platform
– Education and training
– A new stack for analytics (CEP (flume) CDH (sqoop) dbms/BI)
• Future is now
– Use cases now and impact it is having and where it will be, look at Facebook, Yahoo, eBay etc.
Copyright 2010 Cloudera Inc. All rights reserved

27. Emerging Importance
of Data Scientist
• Able to impact business at many
levels
• New conference focused data and
data related roles — O’Reilly
Strata Conference
Copyright 2010 Cloudera Inc. All rights reserved

28. Unprecedented Data Volume,
Velocity and Variety
Data Growth
Out Pacing
Processing Power
Organizations
Swamped and
Turning to Hadoop
61% CAGR
42% CAGR
Data
Transistors
Copyright 2010 Cloudera Inc. All rights reserved

29. Transforming Analytic
Requirements
• Insight into this data needs more than simple
tabular analysis
– More is needed for meaningful answers
• You can and will do deeper and more
introspective analysis
– Machine learning, natural language processing, clustering,
sophisticated statistical analysis, modeling and back testing
• Looking for patterns
– You can see patterns in lots of data that are invisible in less
data. You need pattern discovery tools
Copyright 2010 Cloudera Inc. All rights reserved

30. Hadoop: Already a Big Deal!!
Massive Adoption
Vibrant & Growing Community
100’s of PB Under Management
1000’s of Implementations

31. Benefitting From a Dynamic
OS Community
• Community around
Hadoop is proliferating
and expanding
• > ½ Hadoop sub-projects
promoted to TLPs
• Dozens of related projects
• 100’s of developers
& growing
Copyright 2010 Cloudera Inc. All rights reserved

32. Interest in Hadoop Has Exploded
More are looking for it
Leading analysts report
significant growth
in inquiries
Major increase
in coverage
Copyright 2010 Cloudera Inc. All rights reserved

33. A Data Management Platform
Applications
Copyright 2010 Cloudera Inc. All rights reserved

34. Market Impact
• Hyper personalization
• Extreme targeting
• Expand competitive advantages
• Better retention of customers
• Improved risk analysis