O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

Building Tools for the Hadoop Developer

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio

Confira estes a seguir

1 de 8 Anúncio

Building Tools for the Hadoop Developer

In this session we’ll first discuss our experience extending Hadoop development to new platforms & languages and then discuss our experiments and experiences building supporting developer tools and plugins for those platforms. First, we’ll take a hands on approach to showing our experiments and successes extending Hadoop to languages such as JavaScript and .NET with LINQ. Second, we’ll walk through some of the developer & developer ops tools and plugins we’ve experimented with in an effort to simplify life for the Hadoop developer across both on premises and cloud-based projects.

In this session we’ll first discuss our experience extending Hadoop development to new platforms & languages and then discuss our experiments and experiences building supporting developer tools and plugins for those platforms. First, we’ll take a hands on approach to showing our experiments and successes extending Hadoop to languages such as JavaScript and .NET with LINQ. Second, we’ll walk through some of the developer & developer ops tools and plugins we’ve experimented with in an effort to simplify life for the Hadoop developer across both on premises and cloud-based projects.

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Building Tools for the Hadoop Developer (20)

Anúncio

Mais de DataWorks Summit (20)

Mais recentes (20)

Anúncio

Building Tools for the Hadoop Developer

  1. 1. BUILDING TOOLS FOR THE HADOOP DEVELOPER matt winkler @mwinkle Operated By: mike flasko @mflasko
  2. 2. C#, F# Map/Reduce, LINQ to Hive, .NET management clients Node.js management SDK Hive, Pig, Mahout, Cascading, Scalding, Scoobi, Pegasus… PowerShell, Cross Platform CLI tools
  3. 3. Existing Ecosystem  Actively contributing to:  Core  Pig  Hive  HCatalog  Branching to other projects  Streamlined, Simple Deploy  Simple one-box developer install on Windows  Simple scale up/out to the cloud
  4. 4. .NET  Map/Reduce  LINQ to Hive  Client API’s  WebHCat  Ambari  WebHDFS  Azure / Cloud  Visual Studio Tooling  Local debugging support
  5. 5. JavaScript  Node.js client API’s  WebHCat  WebHDFS  Ambari  Azure / Cloud
  6. 6. Management  UI Tooling  Cluster usage  Job authoring  Result consumption in common tools  PowerShell & Cross platform scripting  API Surface  RDFE – Azure provisioning  Ambari – Cluster monitoring  WebHCatalog – Metadata and job submission  WebHDFS, Blob Storage – Storage >_
  7. 7.  Sources  http://hadoopsdk.codeplex.com  http://www.github.com/windowsazure  NuGet packages  Microsoft.Hadoop.MapReduce  Microsoft.Hadoop.Hive  Microsoft.Hadoop.WebClient  NPM packages  Azure  Azure-cli open

Notas do Editor

  • View from Camp Muir looking to Mount Adams, Mount Rainier National Park, Washington 2011, © matt winkler
  • Innovate across the stack

×