O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Ewa Dominowska, Engineering Manager, Facebook at MLconf SEA - 5/01/15

1.247 visualizações

Publicada em

Managing Machine Learning Projects in Industry: As the use of machine learning techniques to analyze and find value in ‘big data’ is being adopted more broadly by industry, we see an increasing need to build teams that can execute on large and complex projects. It is not possible for a single machine learning expert to cover problems of the scope and magnitude that are encountered. The scale of these projects requires teams of researchers and engineers to coordinate and collaborate to deliver impact. In this talk I will touch on some learnings and considerations when building or expanding such a team. I will cover building a group, framing the problem, finding a solution, and evaluating the results. I will illustrate the points with examples drawn from my experience in large companies and startups. I hope to provoke consideration and discussion for the challenges in this area, as well as to illustrate some of the complexities.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Ewa Dominowska, Engineering Manager, Facebook at MLconf SEA - 5/01/15

  1. 1. Managing Machine Learning Projects in Industry Ewa Dominowska Facebook, Engineering Manager
  2. 2. Agenda • Building a Team • Selecting and Framing a Problem • Problem Solving Approach • Evaluating Solutions • Delivering Impact
  3. 3. Building a Team • Engineer + ML Expert + Statistician + IR Expert • Domain expertise • Academic vs. industry experience • Research + engineering + experimentation = applied research • Investing (domain) vs. outsourcing (science)
  4. 4. Selecting a Lead ML Expert EngineerManager 1 8 8 8
  5. 5. Organizational Structure • Centralized Research Team • Centralized Applied Research Team • Embedded Researchers • Team Members • Academia • Conferences, competitions, data releases, benchmarks MSR, Facebook AI Lab LiveLabs, FB Applied ML Source: Bonkers World
  6. 6. Motivating • Intellectual challenge • Creative work • Autonomy • Purpose • Mastery • Recognition • Publishing • Conferences Source: Motivationhacker
  7. 7. Source: Oreilly Selecting and Framing a Problem
  8. 8. Selecting and Framing a Problem Start with a business problem Break down the problem Understand the impact Find the right data Select an objective function Build Models Measure and Evaluate Experiment Productionalize / Scale
  9. 9. Problem Solving Approach • Establish a baseline • Check your assumptions • Select a modellearning technique • Select features • Measure and evaluation • Experiment • Stability, scalability and robustness Source: Sheldoncomics
  10. 10. Evaluating Solutions • Defining the right metrics • Offline evaluation • A|B testing • Meaningful vs. representative • Representativeness and stability of results • Offline vs. online metrics
  11. 11. • How to split traffic • user, request, budget effects • How long to run a test • statistical significance, power, seasonality, novelty • Calibration • Model interactions • Residue effects from previous experiments Experimentation – Practical Lessons
  12. 12. Delivering Impact • Plan for valuable failure • Measure long term/steady state effects • Engineering improvements • Re-use of components, tools, models and frameworks • Durability and robustness • Data, context changes • Measurement, monitoring, experimentation
  13. 13. Thank you! We are hiring at Facebook!