Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Hadoop in the Cloud
1. The Elephant in the Room
Jim O’Neil
Developer Evangelist, Microsoft
jim.oneil@microsoft.com @jimoneil
2. DIY
Roll your own Hadoop cluster….
welcome to DevOps
“Isotope”
Pallet
3. Appliances
Oracle Big Data Appliance
– 18 server / 12 core each / 40Gb Infiniband
– Partnering with Cloudera on the distribution
Greenplum HD Data Computing Appliance
– 18 nodes, 12 core each
– Straight up Apache Hadoop
NetApp Open Solution for Hadoop
– Storage arrays only (E2660 and FAS2040)
– Partnership with Cloudera
4. The Elephant in the Cloud
Jim O’Neil
Developer Evangelist, Microsoft
jim.oneil@microsoft.com @jimoneil
5. Cloud: a Notional Definition
Private Cloud
Deployment Models
Hybrid Cloud
Community Cloud
Public Cloud
Essential Characteristics
Infrastructure as a Service
Platform as a Service Broad network Rapid Elasticity
access
Software as
Resource Pooling
a Service
On-demand self-service Measured service
6. Hadoop in the Cloud
Google App Engine
appengine-mapreduce API (not really Hadoop)
Amazon Web Services
66 Public AMIs (including Cloudera)
Elastic Map Reduce
Windows Azure
Hadoop on Azure
IBM SmartCloud
Infosphere BigInsights
7. Google App Engine
MapreducePipeline Class
Experimental!
Mapreduce is an experimental, innovative, and rapidly
changing new feature for App Engine.
Unfortunately, being on the bleeding edge means that
we may make backwards-incompatible changes to
Mapreduce. We will inform the community when this
feature is no longer experimental.
9. Windows Azure
http://HadoopOnAzure.com
Currently in Customer Technology Preview
Partnership with Hortonworks
Windows updates to Apache
JavaScript framework
Hive ODBC connector
10. IBM SmartCloud
InfoSphere BigInsights
IBM distribution of Hadoop (0.20.2)
Jaql query language
BigSheets
BigInsight Scheduler
“Hadoop ecosystem”
Hive, Avro, Hbase, Pig, Oozie, Flume
11. I meant what I said, and I said what I meant.
An elephant's faithful, one hundred percent.
Jim O’Neil
Developer Evangelist, Microsoft
jim.oneil@microsoft.com @jimoneil