Hands-on Hadoop Guide to Big Data and Hadoop Tools

•Transferir como PPTX, PDF•

1 gostou•481 visualizações

The document introduces Daniel Templeton and Inyoung Cho, who will be hosting a hands-on Hadoop lab. They define "big data" as any data that is difficult to store in a traditional database due to size, changing schemas, or being unstructured. The lab will take about 2 hours and cover Hadoop components like HDFS for distributed storage, MapReduce for parallel processing, and Hive, Impala, and Pig for querying data. Questions are welcome from attendees.

1
Hands on Hadoop
Daniel Templeton & Inyoung Cho
Cloudera, Inc.

2
Your Hosts
Daniel Templeton
• Certification Developer
• Crusty, old HPC guy
• Likes Perl
Inyoung Cho
• Certification Developer
• Recovering Java
Evangelist
• Invented JavaOne Hands-on
Labs
©2014 Cloudera, Inc. 2 All rights reserved.

3
What is “Big Data”?
• Super-cool marketing buzz word
• “Come see our new line of BIG DATA toasters…”
• “The Five V’s”
• Any data that is difficult to store in a traditional
RDBMS
• Too big, changes schemas too often, unstructured, …
©2014 Cloudera, Inc. 3 All rights reserved.

What is Hadoop?
©2014 Cloudera, Inc. 4 All rights reserved.

What is Hadoop?
©2014 Cloudera, Inc. 5 All rights reserved.

6
HDFS in a Nutshell
• Distributed “file system” service
• Highly scalable and fault resilient
• Chunks files into “blocks” that are replicated and
distributed across the cluster
©2014 Cloudera, Inc. 6 All rights reserved.

7
MapReduce in a Nutshell
• Embarrassingly parallel batch execution engine
• Two phases: map and reduce
• https://www.youtube.com/watch?v=bcjSe0xCHbE
• Tasks are scheduled to run where the data is
• Jobs are written to Java API
©2014 Cloudera, Inc. 7 All rights reserved.

8
Hive in a Nutshell
• SQL engine for Hadoop
• Translates HiveQL into MapReduce jobs
©2014 Cloudera, Inc. 8 All rights reserved.

9
Impala in a Nutshell
• Hive with the MapReduce
©2014 Cloudera, Inc. 9 All rights reserved.

10
Pig in a Nutshell
• Script-like language for data operations
• Translates into MapReduce jobs
©2014 Cloudera, Inc. 10 All rights reserved.

11
The Lab
• Self-paced
• Should take right about 2 hours
• “Additional Exercises” if you finish early
• Inyoung and I are here to answer questions
• Have fun!
©2014 Cloudera, Inc. 11 All rights reserved.

12 ©2014 Cloudera, Inc. All rights reserved.
Daniel Templeton &
Inyoung Cho

Mais conteúdo relacionado

Mais procurados

How Cloudify uses Chef as a Foundation for PaaSNati Shalom

NLUUG print conference May 26 2016Igmar Palsenberg

AWS for Start-ups - Case Study - PeoplePerHour Amazon Web Services

Wido den hollander cloud stack and cephShapeBlue

Hbasecon2013 Wrap UpMinwoo Kim

Amazon EMRDataKitchen

Python & Cassandra - Best FriendsJon Haddad

Open DatacentreDes Drury

Orchestrating VM & Container DeploymentsLars Wander

Apache Cassandra ManagementInstaclustr

Kubernetes trainingDes Drury

DevOps, Cloud, and the Death of Backup Tape Changerske4qqq

IMCSummit 2015 - Day 1 Developer Track - Open-Source In-Memory Platforms: Ben...In-Memory Computing Summit

Cassandra @ Sony: The good, the bad, and the ugly part 2DataStax Academy

2013-cloudconnect-OpenStack@BTuictamale

Way to cloudAndrew Yongjoon Kong

Openstack summit 2015Andrew Yongjoon Kong

Scalable On-Demand Hadoop Clusters with Docker and Mesosnelsonadpresent

Large Scale Data Analytics with Spark and Cassandra on the DSE PlatformDataStax Academy

Mesosphere and Contentteam: A New Way to Run CassandraDataStax Academy

Mais procurados (20)

How Cloudify uses Chef as a Foundation for PaaS

NLUUG print conference May 26 2016

AWS for Start-ups - Case Study - PeoplePerHour

Wido den hollander cloud stack and ceph

Hbasecon2013 Wrap Up

Amazon EMR

Python & Cassandra - Best Friends

Open Datacentre

Orchestrating VM & Container Deployments

Apache Cassandra Management

Kubernetes training

DevOps, Cloud, and the Death of Backup Tape Changers

IMCSummit 2015 - Day 1 Developer Track - Open-Source In-Memory Platforms: Ben...

Cassandra @ Sony: The good, the bad, and the ugly part 2

2013-cloudconnect-OpenStack@BT

Way to cloud

Openstack summit 2015

Scalable On-Demand Hadoop Clusters with Docker and Mesos

Large Scale Data Analytics with Spark and Cassandra on the DSE Platform

Mesosphere and Contentteam: A New Way to Run Cassandra

Destaque

Upper respiratory tract infectionNeha Bharti

Who is accountable?Aowfer Mustafa

Midtown Csi Presentation9 July27 2011sharonjoshea

1% for Education - National Education FundraisingNCEFS - National Consortium of Education Foundations

Futboldario940505

Profitability solution for bankarijitbhowmick

Five Emerging Education Trends - Are your schools readyNCEFS - National Consortium of Education Foundations

Project Report on Financial Statement Analysisarijitbhowmick

Destaque (8)

Upper respiratory tract infection

Who is accountable?

Midtown Csi Presentation9 July27 2011

1% for Education - National Education Fundraising

Futbol

Profitability solution for bank

Five Emerging Education Trends - Are your schools ready

Project Report on Financial Statement Analysis

Semelhante a Hands-on Hadoop Guide to Big Data and Hadoop Tools

Applications on Hadoopmarkgrover

Intro to Hadoop Presentation at Carnegie Mellon - Silicon Valleymarkgrover

Running Hadoop as Service in AltiScale PlatformInMobi Technology

YARNAlex Moundalexis

Webinar: Productionizing Hadoop: Lessons Learned - 20101208Cloudera, Inc.

OpenStack and Ceph case study at the University of AlabamaKamesh Pemmaraju

Case Study: University Alabama-Birmingham.Red_Hat_Storage

Application architectures with hadoop – big data techcon 2014Jonathan Seidman

Application architectures with Hadoop – Big Data TechCon 2014hadooparchbook

Impala use case @ edgeRam Kedem

Big data and mstr bridge the elephantKognitio

Big data - Online TrainingLearntek1

Hashicorp at holaluzRicard Clau

Houston Hadoop Meetup Presentation by Vikram Oberoi of ClouderaMark Kerzner

Hadoop Essentials -- The What, Why and How to Meet Agency ObjectivesCloudera, Inc.

Building a Hadoop Data Warehouse with ImpalaSwiss Big Data User Group

PyData: The Next Generation | Data Day Texas 2015Cloudera, Inc.

Data Science and CDSWJason Hubbard

Building a Hadoop Data Warehouse with Impalahuguk

50 Shades of SQLDataWorks Summit

Semelhante a Hands-on Hadoop Guide to Big Data and Hadoop Tools (20)

Applications on Hadoop

Intro to Hadoop Presentation at Carnegie Mellon - Silicon Valley

Running Hadoop as Service in AltiScale Platform

YARN

Webinar: Productionizing Hadoop: Lessons Learned - 20101208

OpenStack and Ceph case study at the University of Alabama

Case Study: University Alabama-Birmingham.

Application architectures with hadoop – big data techcon 2014

Application architectures with Hadoop – Big Data TechCon 2014

Impala use case @ edge

Big data and mstr bridge the elephant

Big data - Online Training

Hashicorp at holaluz

Houston Hadoop Meetup Presentation by Vikram Oberoi of Cloudera

Hadoop Essentials -- The What, Why and How to Meet Agency Objectives

Building a Hadoop Data Warehouse with Impala

PyData: The Next Generation | Data Day Texas 2015

Data Science and CDSW

Building a Hadoop Data Warehouse with Impala

50 Shades of SQL

Mais de templedf

Couchbase Servertempledf

Supermicro High Performance Enterprise Hadoop Infrastructuretempledf

Revolution Analyticstempledf

Talendtempledf

Datameer Analytics Solutiontempledf

Puppet Labs Puppet Enterprisetempledf

Couchbasetempledf

Pervasive DataRushtempledf

Composite Information Servertempledf

Mais de templedf (9)

Couchbase Server

Supermicro High Performance Enterprise Hadoop Infrastructure

Revolution Analytics

Talend

Datameer Analytics Solution

Puppet Labs Puppet Enterprise

Couchbase

Pervasive DataRush

Composite Information Server

Hands-on Hadoop Guide to Big Data and Hadoop Tools

1. 1 Hands on Hadoop Daniel Templeton & Inyoung Cho Cloudera, Inc.

2. 2 Your Hosts Daniel Templeton • Certification Developer • Crusty, old HPC guy • Likes Perl Inyoung Cho • Certification Developer • Recovering Java Evangelist • Invented JavaOne Hands-on Labs ©2014 Cloudera, Inc. 2 All rights reserved.

3. 3 What is “Big Data”? • Super-cool marketing buzz word • “Come see our new line of BIG DATA toasters…” • “The Five V’s” • Any data that is difficult to store in a traditional RDBMS • Too big, changes schemas too often, unstructured, … ©2014 Cloudera, Inc. 3 All rights reserved.

6. 6 HDFS in a Nutshell • Distributed “file system” service • Highly scalable and fault resilient • Chunks files into “blocks” that are replicated and distributed across the cluster ©2014 Cloudera, Inc. 6 All rights reserved.

7. 7 MapReduce in a Nutshell • Embarrassingly parallel batch execution engine • Two phases: map and reduce • https://www.youtube.com/watch?v=bcjSe0xCHbE • Tasks are scheduled to run where the data is • Jobs are written to Java API ©2014 Cloudera, Inc. 7 All rights reserved.

11. 11 The Lab • Self-paced • Should take right about 2 hours • “Additional Exercises” if you finish early • Inyoung and I are here to answer questions • Have fun! ©2014 Cloudera, Inc. 11 All rights reserved.

Hands-on Hadoop Guide to Big Data and Hadoop Tools

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (8)

Semelhante a Hands-on Hadoop Guide to Big Data and Hadoop Tools

Semelhante a Hands-on Hadoop Guide to Big Data and Hadoop Tools (20)

Mais de templedf

Mais de templedf (9)

Hands-on Hadoop Guide to Big Data and Hadoop Tools