O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Modern Distributed Optimization

131 visualizações

Publicada em

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2xsST7Y.

Matt Adereth talks about the Black-box optimization techniques, what’s actually going on inside of these black-boxes and discusses an idea of how they can be used to solve problems today. He deep dives into a few of the most popular ones, such as Distributed Nelder-Mead and Bayesian Optimization, and discusses their trade-offs. Filmed at qconnewyork.com.

Matt Adereth is a Managing Director at Two Sigma Investments, where he works on tools, infrastructure and methodologies for quantitative financial research. He previously worked at Microsoft on Office, focusing on data connectivity and visualization features. In his spare time, he designs open-source ergonomic keyboards using Clojure.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Modern Distributed Optimization

  1. 1. www.twosigma.com Modern Distributed Optimization October 6, 2017 Matt Adereth QCon NY 2017
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ black-box-optimization
  3. 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. Important Legal Information The views expressed herein are not necessarily the views of Two Sigma Investments, LP or any of its affiliates (collectively, “Two Sigma”). The information presented herein is only for informational and educational purposes and is not an offer to sell or the solicitation of an offer to buy any securities or other instruments. Additionally, the information is not intended to provide, and should not be relied upon for investment, accounting, legal or tax advice. Two Sigma makes no representations, express or implied, regarding the accuracy or completeness of this information, and you accept all risks in relying on the above information for any purpose whatsoever
  5. 5. • Unknown Function • Multiple Parameters • Expensive
  6. 6. Agenda • Real World Problems • Distributed Algorithm Deep Dive • How to Apply Now
  7. 7. Real World Problems
  8. 8. Cluster Configuration • Number of VMs • CPU count • CPU speed per core • RAM per core • Disk count • Disk speed • Network capacity of the VM
  9. 9. Cluster Configuration with CherryPick
  10. 10. JVM Tuning
  11. 11. JVM Tuning $ java -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version | wc -l java version "1.8.0_92" Java(TM) SE Runtime Environment (build 1.8.0_92-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode) 823
  12. 12. JVM Tuning at Twitter
  13. 13. A/B Testing at Yelp
  14. 14. Deep Learning
  15. 15. Deep Learning http://playground.tensorflow.org/
  16. 16. Algorithms
  17. 17. 💻(🎛) 👩(🎛) 🔮(🎛) • Guess and Check • Mental Model • Grid Search • Coordinate Descent
  18. 18. d = 2 n = 3 d = 1 n = 2 d = 3 n = 4
  19. 19. © User:Nicoguaro/ Wikimedia Commons / CC-BY-4.0
  20. 20. © User:Nicoguaro/ Wikimedia Commons / CC-BY-4.0
  21. 21. Multi-Start Nelder-Mead
  22. 22. Parallelized Nelder-Mead Options Speculative • Single large batch • All results influence next batch Multi-start • Multiple asynchronous tasks • Local results only influence local next steps
  23. 23. Bayesian Optimization 1. Estimate the underlying function 2. Evaluate the point that is most likely to be the optimum of underlying function 3. Update the estimate
  24. 24. Bayesian Optimization Demo
  25. 25. Parallel Bayesian Optimization Select the set of points that maximize the likelihood of any of them being the new optimum Unique benefit: Asynchronous with information sharing! Parallel Bayesian Global Optimization of Expensive Functions Jialei Wang, Scott C. Clark, Eric Liu, Peter I. Frazier arXiv:1602.05149 [stat.ML]
  26. 26. Advanced Parallel BO: Freeze-Thaw What if you can approximate the objective function early? Freeze-Thaw Bayesian Optimization Kevin Swersky, Jasper Snoek, Ryan Prescott Adams arXiv:1406.3896 [stat.ML]
  27. 27. How to Apply Now
  28. 28. How to apply now? • Packages • Spearmint • GPyOpt • BOAT • Generalized Services • MOE • SigOpt • Specialized Services • Skipjaq
  29. 29. Thanks! @adereth
  30. 30. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ black-box-optimization