O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Kamal Hakimzadeh – Reproducible Distributed Experiments

6.384 visualizações

Publicada em

Flink Forward 2015

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Kamal Hakimzadeh – Reproducible Distributed Experiments

  1. 1. Reproducible Distributed Experiments Kamal Hakimzadeh PhD Student mahh@kth.se 1 Jim Dowling Associate Professor jdowling@kth.se www.karamel.io
  2. 2. Agenda 2 • Motivation • Reproducibility • Demo: Simple experiment 30-40 min • Karamel Rep. • Karamel Engine • Orchestration • Challenges www.karamel.io
  3. 3. Motivation 3 Analytical vs Empirical proof DS supports many scientific advancements Scheduling, fault tolerant, scalability … Extremely complex www.karamel.io
  4. 4. Reproducible vs. Replicable 4 1. Laboratory 2. Experimenter 3. Apparatus Reproducible Replicable Computational Reproducibility: Infrastructure, software, experiment and data www.karamel.io
  5. 5. Demo : Word Count 5 Text Generator Text Generator Text Generator Word Count www.karamel.io
  6. 6. Karamel: Rep. in different layers 6 Bare Metal Google Compute Engine Virtual Machine is and abstract entity Software is defined in Chef It is publicly available in Github www.karamel.io
  7. 7. Karamel Engine 7 DSL Service Cloud Clients Karamel Engine Physical Mapping Orchestrator www.karamel.io
  8. 8. Orchestration – queuing model 8www.karamel.io
  9. 9. Result 9www.karamel.io
  10. 10. Challenges and future work 10 Scalability Fault Recovery Model Elasticity – Handle Churn Instrumentation Recommendation System Language Support Load generators Scheduling Container base machines Result Management Debugging www.karamel.io
  11. 11. Team members 11 Kamal Hakimzadeh PhD Student at KTH mahh@kth.se Alberto Lorente Leal Software Developer at Comeon a.lorenteleal@gmail.com Jim Dowling Associate Professor at KTH jdowling@kth.se Hooman Peiro Sajjad PhD Student at KTH shps@kth.se Abhimanyu Babbar Backend Developer at Wrap abhimanyu.babbar88@gmail.comwww.karamel.io
  12. 12. 12www.karamel.io

×