Map Reduce v2 and YARN - CHUG - 20120604

•

27 gostaram•3,865 visualizações

Chicago Hadoop Users Group

View the accompanying video on vimeo: https://vimeo.com/43474797

An Introduction to
MapReduce 2 and
YARN

Tom White, Cloudera
@tom_e_white
June 4, 2012
Chicago HUG
Tuesday, June 5, 2012

Road Trip

Tuesday, June 5, 2012

About me
• Apache Hadoop Committer,
PMC Member, Apache
Member

• Engineer at Cloudera
working on core Hadoop

• Founder of Apache Whirr

• Author of “Hadoop: The
Deﬁnitive Guide”

• http://hadoopbook.com

Tuesday, June 5, 2012

First, whatʼs
MapReduce 1?

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Whatʼs wrong with
MR1?

Tuesday, June 5, 2012

Motivation 1

• Scaling >4000 nodes
• Fewer, larger clusters

Tuesday, June 5, 2012

Motivation 2

• HA of Job Tracker
• Large, complex state

Tuesday, June 5, 2012

Motivation 3

• Poor resource utilization
• Slots in MR1 are for either
map or reduce

Tuesday, June 5, 2012

Yet Another Resource Negotiator

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Node Manager
is a generalized Task Tracker
• Task Tracker
• ﬁxed number of map or reduce
slots
• Node Manager
• containers with variable resource
limits

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Tuesday, June 5, 2012

MR is user space
YARN is kernel

Tuesday, June 5, 2012

Bonus Apps

• Distributed shell
• MPI (MAPREDUCE-2911)

• Master-worker (MAPREDUCE-3315)

• Apache Giraph, Hama

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Old API ≠ MR1
New API ≠ MR2

Tuesday, June 5, 2012

Old API New API
o.a.h.mapred o.a.h.mapreduce

MR1 ✓ ✓

MR2 ✓ ✓

Tuesday, June 5, 2012

Tuesday, June 5, 2012

Try out MR2

• Apache Hadoop 2.0.0-alpha
• hadoop.apache.org
• CDH4 and Cloudera Manager
• cloudera.com
• Cloud - Apache Whirr
Tuesday, June 5, 2012

MR1
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>1.0.3</version>
</dependency>

MR2
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.0.0-alpha</version>
</dependency>

Tuesday, June 5, 2012

TODO

• Still alpha status
• Performance tuning
• Usability bug ﬁxes
• RM recovery
• Security in MR2 not complete

Tuesday, June 5, 2012

Questions?

Tuesday, June 5, 2012

Mais conteúdo relacionado

Destaque

An Introduction to MapReduce 2 and YARN

An Introduction to MapReduce 2 and YARN

An Introduction to MapReduce 2 and YARN

Introduction to MapReduce

Introduction to MapReduce

Introduction to MapReduce

Chicago Hadoop Users Group

Inefficient data workloads are all too common across enterprises - causing costly delays, breakages, hard-to-maintain complexity, and ultimately lost productivity. For a typical enterprise with multiple data warehouses, thousands of reports, and hundreds of thousands of ETL jobs being executed every day, this loss of productivity is a real problem. Add to all of this the complex handwritten SQL queries, and there can be nearly a million queries executed every month that desperately need to be optimized, especially to take advantage of the benefits of Apache Hadoop. How can enterprises dig through their workloads and inefficiencies to easily see which are the best fit for Hadoop and what’s the fastest path to get there? Cloudera Navigator Optimizer is the solution - analyzing existing SQL workloads to provide instant insights into your workloads and turns that into an intelligent optimization strategy so you can unlock peak performance and efficiency with Hadoop. As the newest addition to Cloudera’s enterprise Hadoop platform, and now available in limited beta, Navigator Optimizer has helped customers profile over 1.5 million queries and ultimately save millions by optimizing for Hadoop.

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Apache Spark Operations

Apache Spark Operations

Apache Spark Operations

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Valentin Zacharias

Spark on YARN: The Road Ahead

Spark on YARN: The Road Ahead

Spark on YARN: The Road Ahead

This is the presentation from the "Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS" webinar on May 28, 2014. Rohit Bahkshi, a senior product manager at Hortonworks, and Vinod Vavilapalli, PMC for Apache Hadoop, discuss an overview of YARN in HDFS and new features in HDP 2.1. Those new features include: HDFS extended ACLs, HTTPs wire encryption, HDFS DataNode caching, resource manager high availability, application timeline server, and capacity scheduler pre-emption.

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

The vortex of change continues all around us – inside the company, with our customers and partners. A new norm is upon us. Business models are being turned upside down – the hunters now the hunted, global equalization – size is no longer a guarantee of success. The innovative survive and thrive…the nervous and slow go under...what does all this change means for you? Find out how does Intel’s strengths help our customers in this world of change.

The Vortex of Change - Digital Transformation (Presented by Intel)

The Vortex of Change - Digital Transformation (Presented by Intel)

The Vortex of Change - Digital Transformation (Presented by Intel)

Using Big Data to Transform Your Customer’s Experience - Part 1

Using Big Data to Transform Your Customer’s Experience - Part 1

Using Big Data to Transform Your Customer’s Experience - Part 1 

At the StampedeCon 2015 Big Data Conference: YARN enables Hadoop to move beyond just pure batch processing. With that multiple workloads and tenants now must be able to share a single infrastructure for data processing. Features of the Capacity Scheduler enable resource sharing among multiple tenants in a fair manner with elastic queues to maximize utilization. This talk will focus on the features of the Capacity Scheduler that enable Multi-Tenancy and how resource sharing can be rebalanced using features like Preemption.

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

[225]yarn 기반의 deep learning application cluster 구축 김제민

[225]yarn 기반의 deep learning application cluster 구축 김제민

[225]yarn 기반의 deep learning application cluster 구축 김제민

Top 5 IoT Use Cases

Top 5 IoT Use Cases

Top 5 IoT Use Cases

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Install Apache Hadoop for Development/Production

Install Apache Hadoop for Development/Production

Install Apache Hadoop for Development/Production

Apache Spark & Hadoop : Train-the-trainer

Apache Spark & Hadoop : Train-the-trainer

Apache Spark & Hadoop : Train-the-trainer

Inefficient data workloads are all too common across enterprises - causing costly delays, breakages, hard-to-maintain complexity, and ultimately lost productivity. For a typical enterprise with multiple data warehouses, thousands of reports, and hundreds of thousands of ETL jobs being executed every day, this loss of productivity is a real problem. Add to all of this the complex handwritten SQL queries, and there can be nearly a million queries executed every month that desperately need to be optimized, especially to take advantage of the benefits of Apache Hadoop. How can enterprises dig through their workloads and inefficiencies to easily see which are the best fit for Hadoop and what’s the fastest path to get there? Cloudera Navigator Optimizer is the solution - analyzing existing SQL workloads to provide instant insights into your workloads and turns that into an intelligent optimization strategy so you can unlock peak performance and efficiency with Hadoop. As the newest addition to Cloudera’s enterprise Hadoop platform, and now available in limited beta, Navigator Optimizer has helped customers profile over 1.5 million queries and ultimately save millions by optimizing for Hadoop.

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

HDFS Internals

Hadoop Internals (2.3.0 or later)

Hadoop Internals (2.3.0 or later)

Hadoop Internals (2.3.0 or later)

Using Big Data to Drive Customer 360

Using Big Data to Drive Customer 360

Using Big Data to Drive Customer 360

Destaque (19)

An Introduction to MapReduce 2 and YARN

An Introduction to MapReduce 2 and YARN

An Introduction to MapReduce 2 and YARN

Introduction to MapReduce

Introduction to MapReduce

Introduction to MapReduce

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Hadoop for the Data Scientist: Spark in Cloudera 5.5

Apache Spark Operations

Apache Spark Operations

Apache Spark Operations

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Realtime BigData Step by Step mit Lambda, Kafka, Storm und Hadoop

Spark on YARN: The Road Ahead

Spark on YARN: The Road Ahead

Spark on YARN: The Road Ahead

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

Discover HDP 2.1: Apache Hadoop 2.4.0, YARN & HDFS

The Vortex of Change - Digital Transformation (Presented by Intel)

The Vortex of Change - Digital Transformation (Presented by Intel)

The Vortex of Change - Digital Transformation (Presented by Intel)

Using Big Data to Transform Your Customer’s Experience - Part 1

Using Big Data to Transform Your Customer’s Experience - Part 1

Using Big Data to Transform Your Customer’s Experience - Part 1 

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

Apache Hadoop YARN – Multi-Tenancy, Capacity Scheduler & Preemption - Stamped...

[225]yarn 기반의 deep learning application cluster 구축 김제민

[225]yarn 기반의 deep learning application cluster 구축 김제민

[225]yarn 기반의 deep learning application cluster 구축 김제민

Top 5 IoT Use Cases

Top 5 IoT Use Cases

Top 5 IoT Use Cases

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Part 2: Cloudera’s Operational Database: Unlocking New Benefits in the Cloud

Install Apache Hadoop for Development/Production

Install Apache Hadoop for Development/Production

Install Apache Hadoop for Development/Production

Apache Spark & Hadoop : Train-the-trainer

Apache Spark & Hadoop : Train-the-trainer

Apache Spark & Hadoop : Train-the-trainer

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5

HDFS Internals

Hadoop Internals (2.3.0 or later)

Hadoop Internals (2.3.0 or later)

Hadoop Internals (2.3.0 or later)

Using Big Data to Drive Customer 360

Using Big Data to Drive Customer 360

Using Big Data to Drive Customer 360

Semelhante a Map Reduce v2 and YARN - CHUG - 20120604

In this short talk, Ishow some great ways you can use Jenkins to supercharge both the development AND deployment of your Ruby projects. Finally, I will demonstrate how you can extend the Jenkins CI server (which is written in Java) with nothing Ruby code. If you aren't getting the most out of your CI tools, or you aren't using a CI tool at all, then this is for you! Implementing a proper CI is a game changer to the way you and your team develop software.

Ruby CI with Jenkins

Ruby CI with Jenkins

Ruby CI with Jenkins

"We should just call it testing - when it's not exploratory testing it's not real testing anyway" -Twitter, Summer 2011 Lately many professional testers have started to make a clear distinctions between thing that we call testing (like TDD and BDD) and what they consider testing - referring to TDD and BDD mostly as checking. And actually I – and I would think many of you as well – have seen projects with a test coverage of 80% and more that still fail to meet the clients' needs. Even though they meet the specifications perfectly. This points to some value that could be added to techniques like BDD and TDD by embracing the ideas from people like James Marcus Bach, Paul Carvalho and Michael Bolton. After giving an overview of current trends in the testing community like ET (exploratory testing) and ATDD (Acceptance Test Driven Development) this session will try to do exactly that: discuss the - often missing - intersection between BDD and exploratory testing and suggest ways to fill it.

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Michael Mahlberg

Introduction to Apache Pig

Introduction to Apache Pig

Introduction to Apache Pig

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

Chiradeep Vittal

High Availability in YARN

High Availability in YARN

High Availability in YARN

Real-Time Queries in Hadoop w/ Cloudera Impala

Real-Time Queries in Hadoop w/ Cloudera Impala

Real-Time Queries in Hadoop w/ Cloudera Impala

Data Science London

Hadoop-2 @ eBay

Hadoop-2 @ eBay

Hadoop-2 @ eBay

DataWorks Summit

Semelhante a Map Reduce v2 and YARN - CHUG - 20120604 (7)

Ruby CI with Jenkins

Ruby CI with Jenkins

Ruby CI with Jenkins

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Michael mahlberg exploratory-testing-the_missing_half_of_bdd

Introduction to Apache Pig

Introduction to Apache Pig

Introduction to Apache Pig

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

High Availability in YARN

High Availability in YARN

High Availability in YARN

Real-Time Queries in Hadoop w/ Cloudera Impala

Real-Time Queries in Hadoop w/ Cloudera Impala

Real-Time Queries in Hadoop w/ Cloudera Impala

Hadoop-2 @ eBay

Hadoop-2 @ eBay

Hadoop-2 @ eBay

Mais de Chicago Hadoop Users Group

Kinetica master chug_9.12

Kinetica master chug_9.12

Kinetica master chug_9.12

Chicago Hadoop Users Group

Chug dl presentation

Chug dl presentation

Chug dl presentation

Chicago Hadoop Users Group

Bobby Evans and Tom Graves, the engineering leads for Spark and Storm development at Yahoo will talk about how these technologies are used on Yahoo's grids and reasons why to use one or the other. Bobby Evans is the low latency data processing architect at Yahoo. He is a PMC member on many Apache projects including Storm, Hadoop, Spark, and Tez. His team is responsible for delivering Storm as a service to all of Yahoo and maintaining Spark on Yarn for Yahoo (Although Tom really does most of that work). Tom Graves a Senior Software Engineer on the Platform team at Yahoo. He is an Apache PMC member on Hadoop, Spark, and Tez. His team is responsible for delivering and maintaining Spark on Yarn for Yahoo.

Yahoo compares Storm and Spark

Yahoo compares Storm and Spark

Yahoo compares Storm and Spark

Chicago Hadoop Users Group

Jim Scott, CHUG co-founder and Director, Enterprise Strategy and Architecture for MapR presents "Using Apache Drill". This presentation was given on August 13th, 2014 at the Nokia office in Chicago, IL. Jim has held positions running Operations, Engineering, Architecture and QA teams. He has worked in the Consumer Packaged Goods, Digital Advertising, Digital Mapping, Chemical and Pharmaceutical industries. His work with high-throughput computing at Dow Chemical was a precursor to more standardized big data concepts like Hadoop. Apache Drill brings the power of standard ANSI:SQL 2003 to your desktop and your clusters. It is like AWK for Hadoop. Drill supports querying schemaless systems like HBase, Cassandra and MongoDB. Use standard JDBC and ODBC APIs to use Drill from your custom applications. Leveraging an efficient columnar storage format, an optimistic execution engine and a cache-conscious memory layout, Apache Drill is blazing fast. Coordination, query planning, optimization, scheduling, and execution are all distributed throughout nodes in a system to maximize parallelization. This presentation contains live demonstrations. The video can be found here: http://vimeo.com/chug/using-apache-drill

Using Apache Drill

Using Apache Drill

Using Apache Drill

Chicago Hadoop Users Group

John Leach Co-Founder and CTO of Splice Machine with 15+ years software development and machine learning experience will discuss how to use HBase co-processors to build an ANSI-99 SQL database with 1) parallelization of SQL execution plans, 2) ACID transactions with snapshot isolation and 3) consistent secondary indexing. Transactions are critical in traditional RDBMSs because they ensure reliable updates across multiple rows and tables. Most operational applications require transactions, but even analytics systems use transactions to reliably update secondary indexes after a record insert or update. In the Hadoop ecosystem, HBase is a key-value store with real-time updates, but it does not have multi-row, multi-table transactions, secondary indexes or a robust query language like SQL. Combining SQL with a full transactional model over HBase opens a whole new set of OLTP and OLAP use cases for Hadoop that was traditionally reserved for RDBMSs like MySQL or Oracle. However, a transactional HBase system has the advantage of scaling out with commodity servers, leading to a 5x-10x cost savings over traditional databases like MySQL or Oracle. HBase co-processors, introduced in release 0.92, provide a flexible and high-performance framework to extend HBase. In this talk, we show how we used HBase co-processors to support a full ANSI SQL RDBMS without modifying the core HBase source. We will discuss how endpoint transactions are used to serialize SQL execution plans over to regions so that computation is local to where the data is stored. Additionally, we will show how observer co-processors simultaneously support both transactions and secondary indexing. The talk will also discuss how Splice Machine extended the work of Google Percolator, Yahoo Labs’ OMID, and the University of Waterloo on distributed snapshot isolation for transactions. Lastly, performance benchmarks will be provided, including full TPC-C and TPC-H results that show how Hadoop/HBase can be a replacement of traditional RDBMS solutions. To view the accompanying slide deck: http://www.slideshare.net/ChicagoHUG/

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Chicago Hadoop Users Group

The Spark software stack includes a core data-processing engine, an interface for interactive querying, Spark-streaming for streaming data analysis, and growing libraries for machine-learning and graph analysis. Spark is quickly establishing itself as a leading environment for doing fast, iterative in-memory and streaming analysis. This talk will give an introduction to the Spark stack, explain how Spark has lightening fast results, and how it complements Apache Hadoop.

Meet Spark

Chicago Hadoop Users Group

Choosing the Right Big Data Architecture for your Business

Choosing the Right Big Data Architecture for your Business

Choosing the Right Big Data Architecture for your Business

Chicago Hadoop Users Group

Jeff Sposetti of Ambari discusses the Apache project Ambari used to help deploy and provision Hadoop Clusters - Ambari Overview and the Community - Ambari Architecture - Provisioning Clusters and Services -Standard Services: HDFS, YARN, MR2, Hive, new Services: Storm, Falcon - Management and Monitoring Capabilities -Nagios and Ganglia Integration - Key Innovation Features -Ambari Stacks providing dynamic service lifecycle -Ambari BluePrints powering Savannah OpenStack -Ambari Views enabling custom UI development

An Overview of Ambari

An Overview of Ambari

An Overview of Ambari

Chicago Hadoop Users Group

Hadoop and Big Data Security

Hadoop and Big Data Security

Hadoop and Big Data Security

Chicago Hadoop Users Group

Advanced Oozie

Chicago Hadoop Users Group

Scalding for Hadoop

Scalding for Hadoop

Scalding for Hadoop

Chicago Hadoop Users Group

Adam Gugliciello, a 15-year veteran in Software Engineering and Systems Architecture specializes in highly available, parallel systems. In this session, we will answer common questions and demonstrate use cases on how Hadoop and Datameer help with asset management and risk management, fraud detection and data security. This presentation was given on January 24th at the CME Group's offices in Chicago, IL.

Financial Data Analytics with Hadoop

Financial Data Analytics with Hadoop

Financial Data Analytics with Hadoop

Chicago Hadoop Users Group

Everything you wanted to know, but were afraid to ask about Oozie

Everything you wanted to know, but were afraid to ask about Oozie

Everything you wanted to know, but were afraid to ask about Oozie

Chicago Hadoop Users Group

An Introduction to Impala – Low Latency Queries for Apache Hadoop

An Introduction to Impala – Low Latency Queries for Apache Hadoop

An Introduction to Impala – Low Latency Queries for Apache Hadoop

Chicago Hadoop Users Group

HCatalog: Table Management for Hadoop - CHUG - 20120917

HCatalog: Table Management for Hadoop - CHUG - 20120917

HCatalog: Table Management for Hadoop - CHUG - 20120917

Chicago Hadoop Users Group

Hadoop in a Windows Shop - CHUG - 20120416

Hadoop in a Windows Shop - CHUG - 20120416

Hadoop in a Windows Shop - CHUG - 20120416

Chicago Hadoop Users Group

Running R on Hadoop - CHUG - 20120815

Running R on Hadoop - CHUG - 20120815

Running R on Hadoop - CHUG - 20120815

Chicago Hadoop Users Group

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Chicago Hadoop Users Group

Mais de Chicago Hadoop Users Group (18)

Kinetica master chug_9.12

Kinetica master chug_9.12

Kinetica master chug_9.12

Chug dl presentation

Chug dl presentation

Chug dl presentation

Yahoo compares Storm and Spark

Yahoo compares Storm and Spark

Yahoo compares Storm and Spark

Using Apache Drill

Using Apache Drill

Using Apache Drill

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Using HBase Co-Processors to Build a Distributed, Transactional RDBMS - Splic...

Meet Spark

Choosing the Right Big Data Architecture for your Business

Choosing the Right Big Data Architecture for your Business

Choosing the Right Big Data Architecture for your Business

An Overview of Ambari

An Overview of Ambari

An Overview of Ambari

Hadoop and Big Data Security

Hadoop and Big Data Security

Hadoop and Big Data Security

Advanced Oozie

Scalding for Hadoop

Scalding for Hadoop

Scalding for Hadoop

Financial Data Analytics with Hadoop

Financial Data Analytics with Hadoop

Financial Data Analytics with Hadoop

Everything you wanted to know, but were afraid to ask about Oozie

Everything you wanted to know, but were afraid to ask about Oozie

Everything you wanted to know, but were afraid to ask about Oozie

An Introduction to Impala – Low Latency Queries for Apache Hadoop

An Introduction to Impala – Low Latency Queries for Apache Hadoop

An Introduction to Impala – Low Latency Queries for Apache Hadoop

HCatalog: Table Management for Hadoop - CHUG - 20120917

HCatalog: Table Management for Hadoop - CHUG - 20120917

HCatalog: Table Management for Hadoop - CHUG - 20120917

Hadoop in a Windows Shop - CHUG - 20120416

Hadoop in a Windows Shop - CHUG - 20120416

Hadoop in a Windows Shop - CHUG - 20120416

Running R on Hadoop - CHUG - 20120815

Running R on Hadoop - CHUG - 20120815

Running R on Hadoop - CHUG - 20120815

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Avro - More Than Just a Serialization Framework - CHUG - 20120416

Último

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

A Principled Technologies deployment guide Conclusion Deploying VMware Cloud Foundation 5.1 on next gen Dell PowerEdge servers brings together critical virtualization capabilities and high-performing hardware infrastructure. Relying on our hands-on experience, this deployment guide offers a comprehensive roadmap that can guide your organization through the seamless integration of advanced VMware cloud solutions with the performance and reliability of Dell PowerEdge servers. In addition to the deployment efficiency, the Cloud Foundation 5.1 and PowerEdge solution delivered strong performance while running a MySQL database workload. By leveraging VMware Cloud Foundation 5.1 and PowerEdge servers, you could help your organization embrace cloud computing with confidence, potentially unlocking a new level of agility, scalability, and efficiency in your data center operations.

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Principled Technologies

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

HTML Injection Attacks: Impact and Mitigation Strategies

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

Último (20)

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Boost Fertility New Invention Ups Success Rates.pdf

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

Real Time Object Detection Using Open CV

HTML Injection Attacks: Impact and Mitigation Strategies

HTML Injection Attacks: Impact and Mitigation Strategies

HTML Injection Attacks: Impact and Mitigation Strategies

Map Reduce v2 and YARN - CHUG - 20120604

1. An Introduction to MapReduce 2 and YARN Tom White, Cloudera @tom_e_white June 4, 2012 Chicago HUG Tuesday, June 5, 2012

2. Road Trip Tuesday, June 5, 2012

3. About me • Apache Hadoop Committer, PMC Member, Apache Member • Engineer at Cloudera working on core Hadoop • Founder of Apache Whirr • Author of “Hadoop: The Deﬁnitive Guide” • http://hadoopbook.com Tuesday, June 5, 2012

4. First, whatʼs MapReduce 1? Tuesday, June 5, 2012

5. Tuesday, June 5, 2012

6. Whatʼs wrong with MR1? Tuesday, June 5, 2012

7. Motivation 1 • Scaling >4000 nodes • Fewer, larger clusters Tuesday, June 5, 2012

8. Motivation 2 • HA of Job Tracker • Large, complex state Tuesday, June 5, 2012

9. Motivation 3 • Poor resource utilization • Slots in MR1 are for either map or reduce Tuesday, June 5, 2012

10. Yet Another Resource Negotiator Tuesday, June 5, 2012

11. Tuesday, June 5, 2012

12. Tuesday, June 5, 2012

13. Node Manager is a generalized Task Tracker • Task Tracker • ﬁxed number of map or reduce slots • Node Manager • containers with variable resource limits Tuesday, June 5, 2012

14. Tuesday, June 5, 2012

15. Tuesday, June 5, 2012

16. MR is user space YARN is kernel Tuesday, June 5, 2012

17. Bonus Apps • Distributed shell • MPI (MAPREDUCE-2911) • Master-worker (MAPREDUCE-3315) • Apache Giraph, Hama Tuesday, June 5, 2012

18. Tuesday, June 5, 2012

19. Tuesday, June 5, 2012

20. Old API ≠ MR1 New API ≠ MR2 Tuesday, June 5, 2012

21. Old API New API o.a.h.mapred o.a.h.mapreduce MR1 ✓ ✓ MR2 ✓ ✓ Tuesday, June 5, 2012

22. Tuesday, June 5, 2012

23. Try out MR2 • Apache Hadoop 2.0.0-alpha • hadoop.apache.org • CDH4 and Cloudera Manager • cloudera.com • Cloud - Apache Whirr Tuesday, June 5, 2012

24. MR1 <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId> <version>1.0.3</version> </dependency> MR2 <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId> <version>2.0.0-alpha</version> </dependency> Tuesday, June 5, 2012

25. TODO • Still alpha status • Performance tuning • Usability bug ﬁxes • RM recovery • Security in MR2 not complete Tuesday, June 5, 2012

26. Questions? Tuesday, June 5, 2012