Enviar pesquisa
Carregar
Admission Control in Impala
•
15 gostaram
•
5,642 visualizações
Cloudera, Inc.
Seguir
Software
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 26
Recomendados
Dynamic filtering for presto join optimisation
Dynamic filtering for presto join optimisation
Ori Reshef
Strata London 2019 Scaling Impala
Strata London 2019 Scaling Impala
Manish Maheshwari
Hive Does ACID
Hive Does ACID
DataWorks Summit
Performance Optimizations in Apache Impala
Performance Optimizations in Apache Impala
Cloudera, Inc.
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Timothy Spann
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Spark Summit
Hybrid Apache Spark Architecture with YARN and Kubernetes
Hybrid Apache Spark Architecture with YARN and Kubernetes
Databricks
A Closer Look at Apache Kudu
A Closer Look at Apache Kudu
Andriy Zabavskyy
Recomendados
Dynamic filtering for presto join optimisation
Dynamic filtering for presto join optimisation
Ori Reshef
Strata London 2019 Scaling Impala
Strata London 2019 Scaling Impala
Manish Maheshwari
Hive Does ACID
Hive Does ACID
DataWorks Summit
Performance Optimizations in Apache Impala
Performance Optimizations in Apache Impala
Cloudera, Inc.
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
Timothy Spann
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Apache Spark-Bench: Simulate, Test, Compare, Exercise, and Yes, Benchmark wit...
Spark Summit
Hybrid Apache Spark Architecture with YARN and Kubernetes
Hybrid Apache Spark Architecture with YARN and Kubernetes
Databricks
A Closer Look at Apache Kudu
A Closer Look at Apache Kudu
Andriy Zabavskyy
The Impala Cookbook
The Impala Cookbook
Cloudera, Inc.
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Flink Forward
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
DataStax
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Databricks
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Flink Forward
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera, Inc.
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Yue Chen
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Databricks
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
Guido Schmutz
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Sandesh Rao
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Anya Bida
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Marcus Vinicius Miguel Pedro
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Databricks
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Flink Forward
Cloudera Impala Internals
Cloudera Impala Internals
David Groozman
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
Introduction to Storm
Introduction to Storm
Chandler Huang
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Inderaj (Raj) Bains
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
DataWorks Summit
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
huguk
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Cloudera, Inc.
Mais conteúdo relacionado
Mais procurados
The Impala Cookbook
The Impala Cookbook
Cloudera, Inc.
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Flink Forward
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
DataStax
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Databricks
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Flink Forward
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera, Inc.
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Yue Chen
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Databricks
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
Guido Schmutz
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Sandesh Rao
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Anya Bida
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Marcus Vinicius Miguel Pedro
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Databricks
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Flink Forward
Cloudera Impala Internals
Cloudera Impala Internals
David Groozman
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
Introduction to Storm
Introduction to Storm
Chandler Huang
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Inderaj (Raj) Bains
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
DataWorks Summit
Mais procurados
(20)
The Impala Cookbook
The Impala Cookbook
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Tame the small files problem and optimize data layout for streaming ingestion...
Tame the small files problem and optimize data layout for streaming ingestion...
Apache Kudu: Technical Deep Dive
Apache Kudu: Technical Deep Dive
Cloudera Impala Source Code Explanation and Analysis
Cloudera Impala Source Code Explanation and Analysis
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Running Apache Spark on Kubernetes: Best Practices and Pitfalls
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptx
Cloudera Impala Internals
Cloudera Impala Internals
Airflow Clustering and High Availability
Airflow Clustering and High Availability
Introduction to Storm
Introduction to Storm
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Using Apache Hive with High Performance
Using Apache Hive with High Performance
Presto query optimizer: pursuit of performance
Presto query optimizer: pursuit of performance
Destaque
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
huguk
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Cloudera, Inc.
Nested Types in Impala
Nested Types in Impala
Cloudera, Inc.
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Cloudera, Inc.
How Impala Works
How Impala Works
Yue Chen
Data Infused Product Design and Insights at LinkedIn
Data Infused Product Design and Insights at LinkedIn
Yael Garten
White paper hadoop performancetuning
White paper hadoop performancetuning
Anil Reddy
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
Yael Garten
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Yue Chen
Impala SQL Support
Impala SQL Support
Yue Chen
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
hadooparchbook
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
Yael Garten
SecPod: A Framework for Virtualization-based Security Systems
SecPod: A Framework for Virtualization-based Security Systems
Yue Chen
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Cloudera, Inc.
Impala use case @ Zoosk
Impala use case @ Zoosk
Cloudera, Inc.
Architecting next generation big data platform
Architecting next generation big data platform
hadooparchbook
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Cloudera, Inc.
Query Compilation in Impala
Query Compilation in Impala
Cloudera, Inc.
What no one tells you about writing a streaming app
What no one tells you about writing a streaming app
hadooparchbook
Hoodie: Incremental processing on hadoop
Hoodie: Incremental processing on hadoop
Prasanna Rajaperumal
Destaque
(20)
Cloudera Impala technical deep dive
Cloudera Impala technical deep dive
Apache Impala (incubating) 2.5 Performance Update
Apache Impala (incubating) 2.5 Performance Update
Nested Types in Impala
Nested Types in Impala
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
Hive, Impala, and Spark, Oh My: SQL-on-Hadoop in Cloudera 5.5
How Impala Works
How Impala Works
Data Infused Product Design and Insights at LinkedIn
Data Infused Product Design and Insights at LinkedIn
White paper hadoop performancetuning
White paper hadoop performancetuning
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
A Perspective from the intersection Data Science, Mobility, and Mobile Devices
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Remix: On-demand Live Randomization (Fine-grained live ASLR during runtime)
Impala SQL Support
Impala SQL Support
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
SecPod: A Framework for Virtualization-based Security Systems
SecPod: A Framework for Virtualization-based Security Systems
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Data Modeling for Data Science: Simplify Your Workload with Complex Types in ...
Impala use case @ Zoosk
Impala use case @ Zoosk
Architecting next generation big data platform
Architecting next generation big data platform
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Faster Batch Processing with Cloudera 5.7: Hive-on-Spark is ready for production
Query Compilation in Impala
Query Compilation in Impala
What no one tells you about writing a streaming app
What no one tells you about writing a streaming app
Hoodie: Incremental processing on hadoop
Hoodie: Incremental processing on hadoop
Semelhante a Admission Control in Impala
Impala Resource Management - OUTDATED
Impala Resource Management - OUTDATED
Matthew Jacobs
Strata London 2019 Scaling Impala.pptx
Strata London 2019 Scaling Impala.pptx
Manish Maheshwari
YARN
YARN
Alex Moundalexis
Troubleshooting Hadoop: Distributed Debugging
Troubleshooting Hadoop: Distributed Debugging
Great Wide Open
Rev Up Your HPC Engine
Rev Up Your HPC Engine
inside-BigData.com
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
jdcryans
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
UniFabric
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Jeremy Beard
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
JBUG London
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
Govind Kanshi
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise interaction
Govind Kanshi
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
MarkTaylorIBM
MySQL Enterprise Backup apr 2016
MySQL Enterprise Backup apr 2016
Ted Wennmark
Performance tuning Grails applications SpringOne 2GX 2014
Performance tuning Grails applications SpringOne 2GX 2014
Lari Hotari
Postgresql in Education
Postgresql in Education
dostatni
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)
MarkTaylorIBM
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
In-Memory Computing Summit
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Kellyn Pot'Vin-Gorman
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Ayon Sinha
Performance Tuning
Performance Tuning
Jannet Peetz
Semelhante a Admission Control in Impala
(20)
Impala Resource Management - OUTDATED
Impala Resource Management - OUTDATED
Strata London 2019 Scaling Impala.pptx
Strata London 2019 Scaling Impala.pptx
YARN
YARN
Troubleshooting Hadoop: Distributed Debugging
Troubleshooting Hadoop: Distributed Debugging
Rev Up Your HPC Engine
Rev Up Your HPC Engine
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
Kudu: Resolving Transactional and Analytic Trade-offs in Hadoop
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise interaction
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
MySQL Enterprise Backup apr 2016
MySQL Enterprise Backup apr 2016
Performance tuning Grails applications SpringOne 2GX 2014
Performance tuning Grails applications SpringOne 2GX 2014
Postgresql in Education
Postgresql in Education
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Updated Power of the AWR Warehouse, Dallas, HQ, etc.
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Eventual Consistency @WalmartLabs with Kafka, Avro, SolrCloud and Hadoop
Performance Tuning
Performance Tuning
Mais de Cloudera, Inc.
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera, Inc.
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
Cloudera, Inc.
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Cloudera, Inc.
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Cloudera, Inc.
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Cloudera, Inc.
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Cloudera, Inc.
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Cloudera, Inc.
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Cloudera, Inc.
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Cloudera, Inc.
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Cloudera, Inc.
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Cloudera, Inc.
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Cloudera, Inc.
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Cloudera, Inc.
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Cloudera, Inc.
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Cloudera, Inc.
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Cloudera, Inc.
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Cloudera, Inc.
Mais de Cloudera, Inc.
(20)
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Último
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
masabamasaba
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
panagenda
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
Jhone kinadey
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students
HimanshiGarg82
Exploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdf
proinshot.com
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
masabamasaba
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Steffen Staab
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
ayushiqss
%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban
masabamasaba
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
Papp Krisztián
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
Presentation.STUDIO
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
mohitmore19
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
AmarnathKambale
Software Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
Arshad QA
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
masabamasaba
Define the academic and professional writing..pdf
Define the academic and professional writing..pdf
PearlKirahMaeRagusta1
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
masabamasaba
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
VictorSzoltysek
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
10 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 2024
Mind IT Systems
Último
(20)
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students
Exploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdf
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
Software Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
Define the academic and professional writing..pdf
Define the academic and professional writing..pdf
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
10 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 2024
Admission Control in Impala
1.
1 Admission Control in
Impala Matthew Jacobs | @mattjacobs | mj@cloudera.com
2.
2 ©2014 Cloudera,
Inc. All rights reserved. • Too many concurrent queries -> oversubscription • All queries take more time • Application layer can throttle queries? • Not much you can do before Impala 1.3 What’s the Problem
3.
3 ©2014 Cloudera,
Inc. All rights reserved. • Add an admission control mechanism to Impala! • Throttle incoming requests • Queues requests when workload increases • Queued requests executed when resources available So what do we do?
4.
4 ©2014 Cloudera,
Inc. All rights reserved. • Yarn is a resource manager for Hadoop • Assumes jobs are composed of tasks, tasks can be restarted • Need to ask for all resources up front, resources “trickle in” • Non-trivial overhead: job creates “application master” (AM) • But cost is small compared to long batch jobs • Great for MR, things like MR • Not good for • Low-latency, high volume workloads • Gang scheduling, “parts of jobs” can’t be restarted What about Yarn?
5.
5 ©2014 Cloudera,
Inc. All rights reserved. • “Long Lived Application Master” • Long running AMs • Create fake requests to acquire necessary resources • Provides a “gang scheduling” abstraction, waits for all resources • Offers a resource expansion mechanism -> don’t need to ask for all up front • Offers a throttling mechanism • Caches Yarn containers -> lower latency • Looks like a square peg in a round hole… • To be fair, multi-level scheduling is a hard problem! Llama Bridges the Gap
6.
6 ©2014 Cloudera,
Inc. All rights reserved. • Good for Impala sharing resources with other frameworks • Good general purpose resource mgmt solution However: • Not everyone wants/needs to run Yarn and Llama • Still requires round-trips to a central server • Increases query latency • Unlikely to scale for highest latency/throughput requirements • Impala should have a fast, built in throttling mechanism Impala + Llama + Yarn?
7.
7 ©2014 Cloudera,
Inc. All rights reserved. • Throttle number of concurrent requests or memory • Fast • Decentralized • Works without Yarn/Llama • Works with CDH4/CDH5 Impala Admission Control
8.
8 ©2014 Cloudera,
Inc. All rights reserved. • Configure one or more resource “pools” • Max # concurrent queries, max memory, max queue size • Each Impalad capable of making admission decisions • No new single bottleneck/single point of failure • Incoming queries are executed, queued, or rejected • Queue if too many queries OR not enough memory • Reject if queue is full Design Overview
9.
9 ©2014 Cloudera,
Inc. All rights reserved. • Requests admitted or queued locally • Each Impalad keeps track of local state • # queries, pool memory, local queue size • Disseminates local stats via statestore -> global state • Uses cached global state in admission decisions • Decisions are fast; negligible impact on query latency • No single point of failure Localized Admission Decisions
10.
10 ©2014 Cloudera,
Inc. All rights reserved. • Using cached global state -> may “over-admit” • E.g. multiple impalads think 1 request can be admitted and admit before receiving updated state • Configured pool limits are “soft” limits • Fn(Submission rate, distribution across impalads) • Not a big problem in practice • May occasionally admit a few extra queries • Can increase statestore heartbeat frequency • Can add some buffer to configured pool limits Localized Admission Decisions (II)
11.
11 ©2014 Cloudera,
Inc. All rights reserved. • Max memory • Many workloads are limited by memory • Impalads kill queries when running out of memory, anyway • Max number of concurrent queries • Generic mechanism, not resource specific (e.g. memory) • Not as good if workload is heterogeneous • Queries may still be killed if impalads run out of memory Pool Limits
12.
12 Memory Limits • Impalads
track memory hierarchically • Per-process memory • Queries killed when limit is hit • Per-pool memory • For admission control • Per-query memory Process Pool1 Query1 Query2 Pool2 ©2014 Cloudera, Inc. All rights reserved.
13.
13 ©2014 Cloudera,
Inc. All rights reserved. • Admission decisions need more than memory usage • Incoming queries use no memory yet • Queries recently admitted haven’t ramped up yet • Use memory estimates from planning • Estimate pool memory usage with actual usage & estimates • Accounts for future memory usage of recently started queries Admit if: Pool mem estimate + query mem estimate < pool limit Memory Limits (II)
14.
14 ©2014 Cloudera,
Inc. All rights reserved. • Not perfect, query mem estimates are wrong • Hard problem; never have perfect estimates from planning • Usually overly conservative • Leads to underutilization • But at least queries won’t be killed • Less likely to hit process mem limit • Workarounds • Increase pool mem limit • Override with “MEM_LIMIT” query option • Future improvement: Update estimates as query executes • Query mem usage will approach the updated estimate Memory Limits (III)
15.
15 ©2014 Cloudera,
Inc. All rights reserved. • Modeled after Yarn resource queues • Same configuration as Yarn queues • Have a single configuration for Yarn & Impala • Usually want to have the same resource allocations mapped to an organization • E.g. HR gets 10%, Finance gets 30%, Eng gets 60% Request Pools
16.
16 ©2014 Cloudera,
Inc. All rights reserved. • Users are mapped to pools using the placement policy • Users are authorized using the specified ACLs • Pools are defined hierarchically • ACLs are inherited • Currently only enforces limits on leaf pools (IMPALA-905) Request Pools (II)
17.
17 ©2014 Cloudera,
Inc. All rights reserved. • Uses Yarn + Llama configs • Yarn fair scheduler allocation configuration (fair- scheduler.xml) • Llama configuration (llama-site.xml) • Only some of the configuration properties are used • See the documentation for sample config files • Cloudera Manager has a nice UI to configure • No need to touch the xml files Request Pool General Configuration
18.
18 ©2014 Cloudera,
Inc. All rights reserved.
19.
19 ©2014 Cloudera,
Inc. All rights reserved. Placement Rule Configuration Please change the default values
20.
20 ©2014 Cloudera,
Inc. All rights reserved. • If only 1 pool is needed, a separate (easy) configuration path exists • Uses a single “default” pool • No Yarn/Llama configs involved (not even accepted) • Configure the pool limits with impalad flags: • default_pool_max_queued • default_pool_max_requests • default_pool_mem_limit • Doesn’t work with CM5.0, fixed in CM5.0.1 Easy Config Path (Singleton Pool Only)
21.
21 ©2014 Cloudera,
Inc. All rights reserved. Submitting to a Pool
22.
22 ©2014 Cloudera,
Inc. All rights reserved. • Rejections and timeouts return error messages • Metrics • Exposed in impalad web UI: /metrics • Will be available in CM5.1 • Query profile has admission result • Impalad logs have lots of useful information “Debugging” Admission Control Decisions admission-controller.cc:259] Schedule for id=c541aae43af74ed1:afdec812127f8097 in pool_name=root.test/admin PoolConfig(max_requests=20 max_queued=50 mem_limit=-1.00 B) query cluster_mem_estimate=42.00 MB admission-controller.cc:265] Stats: pool=root.test/admin Total(num_running=20, num_queued=7, mem_usage=239.07 MB, mem_estimate=800.00 MB) Local(num_running=20, num_queued=7, mem_usage=239.07 MB, mem_estimate=800.00 MB) admission-controller.cc:303] Queuing, query id=c541aae43af74ed1:afdec812127f8097
23.
23 ©2014 Cloudera,
Inc. All rights reserved. Metrics
24.
24 ©2014 Cloudera,
Inc. All rights reserved. Query Profile Information
25.
25 ©2014 Cloudera,
Inc. All rights reserved. • Queue timeout • Defaults to 60sec, change with --queue_wait_timeout_ms • Running with Yarn/Llama • Same configs: “hard limits” enforced by Yarn+Llama • Disabled by default for CDH4 • Hue (<CDH4.6) doesn’t close queries • Enable with impalad flag (see --disable_admission_control) Some Notes
26.
26 ©2014 Cloudera,
Inc. All rights reserved. Matthew Jacobs @mattjacobs mj@cloudera.com