Windows Azure HDInsight Service provides a fully-managed Apache Hadoop service in the cloud. It is elastic, simple to use, and secure. It is built on Apache Hadoop Distribution including Pig, Hive, Oozie, and other popular Hadoop components. Users can leverage the scalability and geo-redundancy of Azure storage. It also supports various programming languages and tools including .NET, JavaScript, Hive, Pig, and Mahout.

1. WINDOWS AZURE Matt Winkler
Azure Data Platform
HDINSIGHT

2. Windows Azure

3. cloud services
application
building blocks

4. Windows Azure
HDInsight Service
 elastic
 simple
 secure

5. Built on HDP
Core
Pig
Hive
Oozie
Sqoop
Ambari
HCatalog
Templeton

6. Demo

7. Provisionin
g

8. Provisionin
g

9. Leverage Azure Storage
Economic Flexibility
Scale
Geo-Redundancy

10. Secure
Isolated
Single REST Entrypoint

11. Hive, Pig, Mahout, Cascading, Scalding, Scoobi, Pegasus…
C#, F# Map/Reduce, LINQ to Hive, .NET management clients
JavaScript Map/Reduce, Browser hosted console, Node.js management clients
PowerShell, Cross Platform CLI tools

12. Price
Compute*
(~
+
Storage
(~

13. The Data Platform
for Modern Apps
 Any Data, Any Size, Anywhere
 Data Management and Insights at Scale

14. Resources
 Windows Azure
 Free trial
 Getting Started with HDInsight
 Pricing
 .NET SDK For Hadoop
 Halo 4 Case Study

15. start now.

17. Management
 UI Tooling
 Cluster usage
>_  Job authoring
 Result consumption in common tools
 PowerShell & Cross platform scripting
 API Surface
 RDFE – Azure provisioning
 Ambari – Cluster monitoring
 WebHCatalog – Metadata and job submission
 WebHDFS, Blob Storage – Storage

19. Existing Ecosystem
 Actively contributing to:
 Core
 Pig
 Hive
 HCatalog
 Branching to other projects
 Simple one-box developer install on
Windows

20. .NET
 Map/Reduce
 LINQ to Hive
 Client API’s
 WebHCat
 Ambari
 WebHDFS
 Azure
 Visual Studio Tooling
 Local debugging support

21. JavaScript
 MRjs – Map/Reduce in JavaScript
 Node.js client API’s
 WebHCat
 WebHDFS
 Ambari
 Azure

22. Management
 UI Tooling
 Cluster usage
>_  Job authoring
 Result consumption in common tools
 PowerShell & Cross platform scripting
 API Surface
 RDFE – Azure provisioning
 Ambari – Cluster monitoring
 WebHCatalog – Metadata and job submission
 WebHDFS, Blob Storage – Storage

23.  Sources
 http://hadoopsdk.codeplex.com
open  http://www.github.com/windowsazure
 NuGet packages
 Microsoft.Hadoop.MapReduce
 Microsoft.Hadoop.Hive
 Microsoft.Hadoop.WebHDFS => WebClient
 NPM packages
 Azure
 Azure-cli
 Hadoop REST clients pending…