SlideShare uma empresa Scribd logo

Mesos

•
•
Anis Nasir
Anis Nasir

Mesos, Fine-grained resource sharing

Educação
1 de 117
Baixar para ler offline
Motivation Mesos Implementation Evaluation Conclusion Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Muhammad Anis uddin Nasir KTH Royal Institute of Technology November 20, 2012 Muhammad Anis uddin Nasir Mesos 1/32
Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 2/32
Motivation Mesos Implementation Evaluation Conclusion Motivation Diverse cluster computer framework Muhammad Anis uddin Nasir Mesos 3/32
Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Muhammad Anis uddin Nasir Mesos 4/32
Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Muhammad Anis uddin Nasir Mesos 4/32
Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Muhammad Anis uddin Nasir Mesos 4/32
Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Data sharing between clusters Muhammad Anis uddin Nasir Mesos 4/32
Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Data sharing between clusters Reduce Cost Muhammad Anis uddin Nasir Mesos 4/32
Motivation Mesos Implementation Evaluation Conclusion Existing Solutions Static Partitioning Muhammad Anis uddin Nasir Mesos 5/32
Motivation Mesos Implementation Evaluation Conclusion Existing Solutions Static Partitioning Virtual Machines Muhammad Anis uddin Nasir Mesos 5/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Data Locality Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Data Locality Utilization Muhammad Anis uddin Nasir Mesos 6/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 7/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Mesos Common resource sharing layer Fine grained sharing Across diverse cluster computing frameworks Muhammad Anis uddin Nasir Mesos 8/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Goals High Utilization Muhammad Anis uddin Nasir Mesos 9/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Goals High Utilization Data sharing among frameworks Muhammad Anis uddin Nasir Mesos 9/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Muhammad Anis uddin Nasir Mesos 10/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Muhammad Anis uddin Nasir Mesos 10/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Muhammad Anis uddin Nasir Mesos 10/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Fault tolerant Muhammad Anis uddin Nasir Mesos 10/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Fault tolerant Highly available Muhammad Anis uddin Nasir Mesos 10/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Muhammad Anis uddin Nasir Mesos 11/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Muhammad Anis uddin Nasir Mesos 11/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Run multiple versions of a framework Muhammad Anis uddin Nasir Mesos 11/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Run multiple versions of a framework Build special framework targeting particular problem domain Muhammad Anis uddin Nasir Mesos 11/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Simple and Scalable Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Simple and Scalable Application-controlled scheduling mechanism Muhammad Anis uddin Nasir Mesos 12/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Fine-Grained Sharing Muhammad Anis uddin Nasir Mesos 13/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Drawback Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Drawback decentralized decision might not be optimal Muhammad Anis uddin Nasir Mesos 14/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Architecture Muhammad Anis uddin Nasir Mesos 15/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Architecture Muhammad Anis uddin Nasir Mesos 16/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Linux Containers Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Report node failure to Solaris Project framework Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Report node failure to Solaris Project framework Multiple schedulers Muhammad Anis uddin Nasir Mesos 17/32
Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 18/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Torque and MPI Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Torque and MPI Spark Muhammad Anis uddin Nasir Mesos 19/32
Motivation Mesos Implementation Evaluation Conclusion Spark Data flow for logistic regression Muhammad Anis uddin Nasir Mesos 20/32
Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 21/32
Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing 96 Node Mesos Cluster Muhammad Anis uddin Nasir Mesos 22/32
Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing 96 Node Mesos Cluster 4 CPU Cores and 15GB Ram Muhammad Anis uddin Nasir Mesos 22/32
Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing Muhammad Anis uddin Nasir Mesos 23/32
Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes Muhammad Anis uddin Nasir Mesos 24/32
Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes 1.7x speed up with Mesos Muhammad Anis uddin Nasir Mesos 24/32
Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes 1.7x speed up with Mesos 97% data locality with 5sec data scheduling Muhammad Anis uddin Nasir Mesos 24/32
Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Muhammad Anis uddin Nasir Mesos 25/32
Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Scaled to 50,000 emulated slaves, 200 frameworks, 100K tasks Muhammad Anis uddin Nasir Mesos 25/32
Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Scaled to 50,000 emulated slaves, 200 frameworks, 100K tasks unable to scale beyond 50,000 slaves as Amazon EC2 cluster was the bottleneck Muhammad Anis uddin Nasir Mesos 25/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead LINPACK for MPI and Wordcount Hadoop Becnhmark Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead LINPACK for MPI and Wordcount Hadoop Becnhmark Overhead of Mesos was less than 4% Muhammad Anis uddin Nasir Mesos 26/32
Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 27/32
Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Muhammad Anis uddin Nasir Mesos 28/32
Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Muhammad Anis uddin Nasir Mesos 28/32
Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Resource Sharing, a scalable mechanism for application-controlled scheduling Muhammad Anis uddin Nasir Mesos 28/32
Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Resource Sharing, a scalable mechanism for application-controlled scheduling Enabales co-existence of current frameworks and development of new specialized frameworks Muhammad Anis uddin Nasir Mesos 28/32
Motivation Mesos Implementation Evaluation Conclusion References Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A. D., Katz, R., Shenker, S., et al. (n.d.). Mesos : A Platform for Fine-Grained Resource Sharing in the Data Center. http://incubator.apache.org/mesos/ http://datainthecloud.blogspot.se/2011/10/ mesos-platform-for-fine-grained.html https://www.usenix.org/conference/nsdi11/ mesos-platform-fine-grained-resource-sharing-data-cent http://static.usenix.org/event/nsdi11/tech/ slides/hindman.pdf Muhammad Anis uddin Nasir Mesos 29/32
Motivation Mesos Implementation Evaluation Conclusion Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Muhammad Anis uddin Nasir KTH Royal Institute of Technology November 20, 2012 Muhammad Anis uddin Nasir Mesos 30/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks elastic tasks Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks elastic tasks do not accept unknown resources Muhammad Anis uddin Nasir Mesos 31/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex framework has a choice Muhammad Anis uddin Nasir Mesos 32/32
Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex framework has a choice failures are easy to handle with resource offers Muhammad Anis uddin Nasir Mesos 32/32

Recomendados

Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache MesosJoe Stein
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache Mesostomasbart
 
Mesos study report 03v1.2
Mesos study report 03v1.2Mesos study report 03v1.2
Mesos study report 03v1.2Stefanie Zhao
 
Mesos - A Platform for Fine-Grained Resource Sharing in the Data Center
Mesos - A Platform for Fine-Grained Resource Sharing in the Data CenterMesos - A Platform for Fine-Grained Resource Sharing in the Data Center
Mesos - A Platform for Fine-Grained Resource Sharing in the Data CenterAnkur Chauhan
 
Introduction To Apache Mesos
Introduction To Apache MesosIntroduction To Apache Mesos
Introduction To Apache MesosTimothy St. Clair
 
Apache Mesos: a simple explanation of basics
Apache Mesos: a simple explanation of basicsApache Mesos: a simple explanation of basics
Apache Mesos: a simple explanation of basicsGladson Manuel
 
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...Anis Nasir
 
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...Anis Nasir
 

Recomendados

Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache MesosJoe Stein
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache Mesostomasbart
 
Mesos study report 03v1.2
Mesos study report 03v1.2Mesos study report 03v1.2
Mesos study report 03v1.2Stefanie Zhao
 
Mesos - A Platform for Fine-Grained Resource Sharing in the Data Center
Mesos - A Platform for Fine-Grained Resource Sharing in the Data CenterMesos - A Platform for Fine-Grained Resource Sharing in the Data Center
Mesos - A Platform for Fine-Grained Resource Sharing in the Data CenterAnkur Chauhan
 
Introduction To Apache Mesos
Introduction To Apache MesosIntroduction To Apache Mesos
Introduction To Apache MesosTimothy St. Clair
 
Apache Mesos: a simple explanation of basics
Apache Mesos: a simple explanation of basicsApache Mesos: a simple explanation of basics
Apache Mesos: a simple explanation of basicsGladson Manuel
 
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...
When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Pro...Anis Nasir
 
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...
The Power of Both Choices: Practical Load Balancing for Distributed Stream Pr...Anis Nasir
 
Gossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social NetworksGossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social NetworksAnis Nasir
 
Scaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip LearningScaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip LearningAnis Nasir
 
Pushing the cap
Pushing the capPushing the cap
Pushing the capAnis Nasir
 
NaaS
NaaSNaaS
NaaSAnis Nasir
 
NaaS
NaaSNaaS
NaaSAnis Nasir
 
Final presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmnFinal presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmnAnis Nasir
 
RESTvsSOAP
RESTvsSOAPRESTvsSOAP
RESTvsSOAPAnis Nasir
 
Concurrency Control in Database Management system
Concurrency Control in Database Management systemConcurrency Control in Database Management system
Concurrency Control in Database Management systemChristalin Nelson
 
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxQ4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxlancelewisportillo
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxMan or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxDhatriParmar
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataBabyAnnMotar
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxExcellence Foundation for South Sudan
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching ProfessionSri Sairam College Of Engineering Bengaluru
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
Scientific Writing :Research Discourse
Scientific Writing :Research DiscourseScientific Writing :Research Discourse
Scientific Writing :Research DiscourseAnita GoswamiGiri
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQuiz Club NITW
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptxmary850239
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvRicaMaeCastro1
 

Mais conteúdo relacionado

Mais de Anis Nasir

Gossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social NetworksGossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social NetworksAnis Nasir
 
Scaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip LearningScaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip LearningAnis Nasir
 
Pushing the cap
Pushing the capPushing the cap
Pushing the capAnis Nasir
 
Final presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmnFinal presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmnAnis Nasir
 

Mais de Anis Nasir (7)

Gossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social NetworksGossip based partitioning and replication for Online Social Networks
Gossip based partitioning and replication for Online Social Networks
 
Scaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip LearningScaling Online Social Networks: extended SPAR using Gossip Learning
Scaling Online Social Networks: extended SPAR using Gossip Learning
 
Pushing the cap
Pushing the capPushing the cap
Pushing the cap
 
NaaS
NaaSNaaS
NaaS
 
NaaS
NaaSNaaS
NaaS
 
Final presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmnFinal presentation survey p2p-videostreamingwmn
Final presentation survey p2p-videostreamingwmn
 
RESTvsSOAP
RESTvsSOAPRESTvsSOAP
RESTvsSOAP
 

Último

Concurrency Control in Database Management system
Concurrency Control in Database Management systemConcurrency Control in Database Management system
Concurrency Control in Database Management systemChristalin Nelson
 
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxQ4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxlancelewisportillo
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxMan or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxDhatriParmar
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataBabyAnnMotar
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
Scientific Writing :Research Discourse
Scientific Writing :Research DiscourseScientific Writing :Research Discourse
Scientific Writing :Research DiscourseAnita GoswamiGiri
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQuiz Club NITW
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptxmary850239
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvRicaMaeCastro1
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Seán Kennedy
 
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxBIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxSayali Powar
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...DhatriParmar
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWQuiz Club NITW
 

Último (20)

Concurrency Control in Database Management system
Concurrency Control in Database Management systemConcurrency Control in Database Management system
Concurrency Control in Database Management system
 
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxQ4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptxMan or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
Man or Manufactured_ Redefining Humanity Through Biopunk Narratives.pptx
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped data
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Profession
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1
 
Scientific Writing :Research Discourse
Scientific Writing :Research DiscourseScientific Writing :Research Discourse
Scientific Writing :Research Discourse
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...
 
Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"
 
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxBIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITW
 

Mesos

  • 1. Motivation Mesos Implementation Evaluation Conclusion Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Muhammad Anis uddin Nasir KTH Royal Institute of Technology November 20, 2012 Muhammad Anis uddin Nasir Mesos 1/32
  • 2. Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 2/32
  • 3. Motivation Mesos Implementation Evaluation Conclusion Motivation Diverse cluster computer framework Muhammad Anis uddin Nasir Mesos 3/32
  • 4. Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Muhammad Anis uddin Nasir Mesos 4/32
  • 5. Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Muhammad Anis uddin Nasir Mesos 4/32
  • 6. Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Muhammad Anis uddin Nasir Mesos 4/32
  • 7. Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Data sharing between clusters Muhammad Anis uddin Nasir Mesos 4/32
  • 8. Motivation Mesos Implementation Evaluation Conclusion Motivation No optimal framework Run multiple frameworks Higher Utilization Data sharing between clusters Reduce Cost Muhammad Anis uddin Nasir Mesos 4/32
  • 9. Motivation Mesos Implementation Evaluation Conclusion Existing Solutions Static Partitioning Muhammad Anis uddin Nasir Mesos 5/32
  • 10. Motivation Mesos Implementation Evaluation Conclusion Existing Solutions Static Partitioning Virtual Machines Muhammad Anis uddin Nasir Mesos 5/32
  • 11. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Muhammad Anis uddin Nasir Mesos 6/32
  • 12. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Muhammad Anis uddin Nasir Mesos 6/32
  • 13. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Muhammad Anis uddin Nasir Mesos 6/32
  • 14. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Muhammad Anis uddin Nasir Mesos 6/32
  • 15. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Data Locality Muhammad Anis uddin Nasir Mesos 6/32
  • 16. Motivation Mesos Implementation Evaluation Conclusion Fine-grained Hadoop and Dryad Slots Tasks Benefits Data Locality Utilization Muhammad Anis uddin Nasir Mesos 6/32
  • 17. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 7/32
  • 18. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Mesos Common resource sharing layer Fine grained sharing Across diverse cluster computing frameworks Muhammad Anis uddin Nasir Mesos 8/32
  • 19. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Goals High Utilization Muhammad Anis uddin Nasir Mesos 9/32
  • 20. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Goals High Utilization Data sharing among frameworks Muhammad Anis uddin Nasir Mesos 9/32
  • 21. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Muhammad Anis uddin Nasir Mesos 10/32
  • 22. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Muhammad Anis uddin Nasir Mesos 10/32
  • 23. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Muhammad Anis uddin Nasir Mesos 10/32
  • 24. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Fault tolerant Muhammad Anis uddin Nasir Mesos 10/32
  • 25. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Challenges Scalable Support diverse frameworks Efficient Fault tolerant Highly available Muhammad Anis uddin Nasir Mesos 10/32
  • 26. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Muhammad Anis uddin Nasir Mesos 11/32
  • 27. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Muhammad Anis uddin Nasir Mesos 11/32
  • 28. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Run multiple versions of a framework Muhammad Anis uddin Nasir Mesos 11/32
  • 29. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Other Benefits Run Multiple instances of same framework Isolate production and experimental jobs Run multiple versions of a framework Build special framework targeting particular problem domain Muhammad Anis uddin Nasir Mesos 11/32
  • 30. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Muhammad Anis uddin Nasir Mesos 12/32
  • 31. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Muhammad Anis uddin Nasir Mesos 12/32
  • 32. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Muhammad Anis uddin Nasir Mesos 12/32
  • 33. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Muhammad Anis uddin Nasir Mesos 12/32
  • 34. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Simple and Scalable Muhammad Anis uddin Nasir Mesos 12/32
  • 35. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Design Elements Fine-grained sharing Allocation at the level of tasks within a job Improve utilization, latency and data locality Resource offers Simple and Scalable Application-controlled scheduling mechanism Muhammad Anis uddin Nasir Mesos 12/32
  • 36. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Fine-Grained Sharing Muhammad Anis uddin Nasir Mesos 13/32
  • 37. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Muhammad Anis uddin Nasir Mesos 14/32
  • 38. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Muhammad Anis uddin Nasir Mesos 14/32
  • 39. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits Muhammad Anis uddin Nasir Mesos 14/32
  • 40. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation Muhammad Anis uddin Nasir Mesos 14/32
  • 41. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Muhammad Anis uddin Nasir Mesos 14/32
  • 42. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Drawback Muhammad Anis uddin Nasir Mesos 14/32
  • 43. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Resource Offers Offers resources to framework Framework choose resources according to needs Benefits gives freedom to framework for implementation keep Mesos simple and scalable Drawback decentralized decision might not be optimal Muhammad Anis uddin Nasir Mesos 14/32
  • 44. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Architecture Muhammad Anis uddin Nasir Mesos 15/32
  • 45. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion Architecture Muhammad Anis uddin Nasir Mesos 16/32
  • 46. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Muhammad Anis uddin Nasir Mesos 17/32
  • 47. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Muhammad Anis uddin Nasir Mesos 17/32
  • 48. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Muhammad Anis uddin Nasir Mesos 17/32
  • 49. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Muhammad Anis uddin Nasir Mesos 17/32
  • 50. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Muhammad Anis uddin Nasir Mesos 17/32
  • 51. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Linux Containers Muhammad Anis uddin Nasir Mesos 17/32
  • 52. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Resource Allocation Fair sharing Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 53. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 54. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 55. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 56. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 57. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 58. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Solaris Project Muhammad Anis uddin Nasir Mesos 17/32
  • 59. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Report node failure to Solaris Project framework Muhammad Anis uddin Nasir Mesos 17/32
  • 60. Motivation Mesos Overview Implementation Architecture Evaluation Conclusion More Features Scalability and Resource Allocation Robustness Fair sharing Filters Strict priorities Re-offering resources Delay sharing Incentives Fault Tolerance Isolation Soft state master Linux Containers Report node failure to Solaris Project framework Multiple schedulers Muhammad Anis uddin Nasir Mesos 17/32
  • 61. Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 18/32
  • 62. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Muhammad Anis uddin Nasir Mesos 19/32
  • 63. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X Muhammad Anis uddin Nasir Mesos 19/32
  • 64. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python Muhammad Anis uddin Nasir Mesos 19/32
  • 65. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Muhammad Anis uddin Nasir Mesos 19/32
  • 66. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Muhammad Anis uddin Nasir Mesos 19/32
  • 67. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Muhammad Anis uddin Nasir Mesos 19/32
  • 68. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Torque and MPI Muhammad Anis uddin Nasir Mesos 19/32
  • 69. Motivation Mesos Implementation Evaluation Conclusion Implementation 10,000 lines of code in C++ Linux, Solaris, OS X support framework written in Java, C++, Python ZooKeeper Frameworks Hadoop Torque and MPI Spark Muhammad Anis uddin Nasir Mesos 19/32
  • 70. Motivation Mesos Implementation Evaluation Conclusion Spark Data flow for logistic regression Muhammad Anis uddin Nasir Mesos 20/32
  • 71. Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 21/32
  • 72. Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing 96 Node Mesos Cluster Muhammad Anis uddin Nasir Mesos 22/32
  • 73. Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing 96 Node Mesos Cluster 4 CPU Cores and 15GB Ram Muhammad Anis uddin Nasir Mesos 22/32
  • 74. Motivation Mesos Implementation Evaluation Conclusion Dynamic Resource Sharing Muhammad Anis uddin Nasir Mesos 23/32
  • 75. Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes Muhammad Anis uddin Nasir Mesos 24/32
  • 76. Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes 1.7x speed up with Mesos Muhammad Anis uddin Nasir Mesos 24/32
  • 77. Motivation Mesos Implementation Evaluation Conclusion Data Locality with Resource Offers 16 instance of Hadoop using 93 EC2 nodes 1.7x speed up with Mesos 97% data locality with 5sec data scheduling Muhammad Anis uddin Nasir Mesos 24/32
  • 78. Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Muhammad Anis uddin Nasir Mesos 25/32
  • 79. Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Scaled to 50,000 emulated slaves, 200 frameworks, 100K tasks Muhammad Anis uddin Nasir Mesos 25/32
  • 80. Motivation Mesos Implementation Evaluation Conclusion Scalability 99 Amazon EC2 nodes Scaled to 50,000 emulated slaves, 200 frameworks, 100K tasks unable to scale beyond 50,000 slaves as Amazon EC2 cluster was the bottleneck Muhammad Anis uddin Nasir Mesos 25/32
  • 81. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Muhammad Anis uddin Nasir Mesos 26/32
  • 82. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum Muhammad Anis uddin Nasir Mesos 26/32
  • 83. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Muhammad Anis uddin Nasir Mesos 26/32
  • 84. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead Muhammad Anis uddin Nasir Mesos 26/32
  • 85. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead LINPACK for MPI and Wordcount Hadoop Becnhmark Muhammad Anis uddin Nasir Mesos 26/32
  • 86. Motivation Mesos Implementation Evaluation Conclusion Further Experiments Fault Tolerance Mesos masters connected to a 5-node ZooKeeper quorum fault detection and recovery in 10 sec Overhead LINPACK for MPI and Wordcount Hadoop Becnhmark Overhead of Mesos was less than 4% Muhammad Anis uddin Nasir Mesos 26/32
  • 87. Motivation Mesos Implementation Evaluation Conclusion 1 Motivation 2 Mesos Overview Architecture 3 Implementation 4 Evaluation 5 Conclusion Muhammad Anis uddin Nasir Mesos 27/32
  • 88. Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Muhammad Anis uddin Nasir Mesos 28/32
  • 89. Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Muhammad Anis uddin Nasir Mesos 28/32
  • 90. Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Resource Sharing, a scalable mechanism for application-controlled scheduling Muhammad Anis uddin Nasir Mesos 28/32
  • 91. Motivation Mesos Implementation Evaluation Conclusion Conclusion Mesos shares clusters efficiently among diverse frameworks Fine-grained sharing at the level of tasks Resource Sharing, a scalable mechanism for application-controlled scheduling Enabales co-existence of current frameworks and development of new specialized frameworks Muhammad Anis uddin Nasir Mesos 28/32
  • 92. Motivation Mesos Implementation Evaluation Conclusion References Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A. D., Katz, R., Shenker, S., et al. (n.d.). Mesos : A Platform for Fine-Grained Resource Sharing in the Data Center. http://incubator.apache.org/mesos/ http://datainthecloud.blogspot.se/2011/10/ mesos-platform-for-fine-grained.html https://www.usenix.org/conference/nsdi11/ mesos-platform-fine-grained-resource-sharing-data-cent http://static.usenix.org/event/nsdi11/tech/ slides/hindman.pdf Muhammad Anis uddin Nasir Mesos 29/32
  • 93. Motivation Mesos Implementation Evaluation Conclusion Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Muhammad Anis uddin Nasir KTH Royal Institute of Technology November 20, 2012 Muhammad Anis uddin Nasir Mesos 30/32
  • 94. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks Muhammad Anis uddin Nasir Mesos 31/32
  • 95. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration Muhammad Anis uddin Nasir Mesos 31/32
  • 96. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Muhammad Anis uddin Nasir Mesos 31/32
  • 97. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences Muhammad Anis uddin Nasir Mesos 31/32
  • 98. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy Muhammad Anis uddin Nasir Mesos 31/32
  • 99. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Muhammad Anis uddin Nasir Mesos 31/32
  • 100. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks Muhammad Anis uddin Nasir Mesos 31/32
  • 101. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment Muhammad Anis uddin Nasir Mesos 31/32
  • 102. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks Muhammad Anis uddin Nasir Mesos 31/32
  • 103. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Muhammad Anis uddin Nasir Mesos 31/32
  • 104. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives Muhammad Anis uddin Nasir Mesos 31/32
  • 105. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks Muhammad Anis uddin Nasir Mesos 31/32
  • 106. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks elastic tasks Muhammad Anis uddin Nasir Mesos 31/32
  • 107. Motivation Mesos Implementation Evaluation Conclusion Different workloads Homogeneous Tasks elastic frameworks with constant task duration rigid frameworks with exponential task duration Placement Preferences weighted fair allocation policy lottery scheduling Heterogeneous Tasks random task assignment reserve some resources on each node for small tasks maximum task duration Framework incentives short tasks elastic tasks do not accept unknown resources Muhammad Anis uddin Nasir Mesos 31/32
  • 108. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation Muhammad Anis uddin Nasir Mesos 32/32
  • 109. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing Muhammad Anis uddin Nasir Mesos 32/32
  • 110. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve Muhammad Anis uddin Nasir Mesos 32/32
  • 111. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Muhammad Anis uddin Nasir Mesos 32/32
  • 112. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints Muhammad Anis uddin Nasir Mesos 32/32
  • 113. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Muhammad Anis uddin Nasir Mesos 32/32
  • 114. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity Muhammad Anis uddin Nasir Mesos 32/32
  • 115. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex Muhammad Anis uddin Nasir Mesos 32/32
  • 116. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex framework has a choice Muhammad Anis uddin Nasir Mesos 32/32
  • 117. Motivation Mesos Implementation Evaluation Conclusion Limitations Fragmentation not optimal bin packing large jobs may starve minimum offer size on each slave Interdependent framework constraints scenarios where only one task can be accommodated Framework Complexity framework scheduling is complex framework has a choice failures are easy to handle with resource offers Muhammad Anis uddin Nasir Mesos 32/32