O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Próximos SlideShares
What to Upload to SlideShare
What to Upload to SlideShare
Carregando em…3
×
1 de 40

.NET Fest 2019. Оля Гавриш. Машинное обучение для .NET программистов

31 de Out de 2019
• 0 gostaram • 453 visualizações

0

Compartilhar

Educação

А Вы знали, что практически для каждого проекта можно применить машинное обучение? И теперь для этого не нужно изучать новый язык программирования (как Python или R) и осваивать численные методы. В этом докладе я расскажу об основах машинного обучения и о том, как легко начать использовать его в своих .NET проектах с помощью ML.NET и других решений от Microsoft.

Recomendadas

Livros relacionados

Gratuito durante 30 dias do Scribd

Ver tudo
Para que serve Deus: Em busca da verdadeira fé Philip Yancey
(4/5)
Gratuito
Cadeia De Hábitos: Descubra 97 Hábitos Simples Que Podem Mudar Sua Vida S.j. Scott
(4.5/5)
Gratuito
Qual é o seu legado? 101 ideias para começar a criar e construir um Anca Ioviţă
(0/5)
Gratuito
O Segredo das Mulheres Apaixonantes Eduardo Nunes
(4/5)
Gratuito
Saber Envelhecer Cícero
(0/5)
Gratuito
Limite zero: O sistema havaiano secreto para prosperidade, saúde, paz, e mais ainda Joe Vitale
(4.5/5)
Gratuito
O Estilo De Vida Minimalista - Simplifique, Organize E Descomplique A Sua Vida Joshua Michaels
(2.5/5)
Gratuito
O Estoicismo Hoje: Sabedoria Antiga para a Vida Moderna Patrick Ussher
(3.5/5)
Gratuito
O anseio furioso de Deus Brennan Manning
(4.5/5)
Gratuito
O Livro das Afirmações Mafalda Lempicka
(5/5)
Gratuito
A Arte De Bater Papo: Construindo Relacionamentos De Sucesso Jacob Andrews
(4.5/5)
Gratuito
Coaching Pessoal Miguel D'Addario
(4.5/5)
Gratuito
Prática Mental: Direcionar a Atenção APO HALMYRIS
(4.5/5)
Gratuito
Pense Como Um Gênio: Os Sete Passos Para Encontrar Soluções Brilhantes Para Problemas Comuns Raimon Samsó
(4/5)
Gratuito
TDAH em Adultos - Como Reconhecer e Lidar Com Adultos Que Sofrem de TDAH Em 30 Passos Simples The Blokehead
(4.5/5)
Gratuito
Chakras: Kundalini - 2 livros em 1 Jen Solis
(4/5)
Gratuito

Audiolivros relacionados

Gratuito durante 30 dias do Scribd

Ver tudo
21 dias para curar sua vida: Amando a si mesmo trabalhando com o espelho Louise Hay
(4.5/5)
Gratuito
52 atitudes de uma mulher poderosa Guta Gouveia
(4.5/5)
Gratuito
Focar: Supere a procrastinação e aumente a força de vontade e a atenção Dave Farrel
(4.5/5)
Gratuito
52 boas maneiras para dizer "Não" Clene Salles
(4.5/5)
Gratuito
52 maneiras de ganhar mais dinheiro Clene Salles
(4.5/5)
Gratuito
10 Maneiras de manter o foco James Fries
(4.5/5)
Gratuito
Vencendo gigantes Hernandes Dias Lopes
(5/5)
Gratuito
52 maneiras de preocupar-se menos Clene Salles
(4.5/5)
Gratuito
Manhãs de Propósito: Um ritual matinal simples pra você fazer em qualquer lugar do mundo, aumentando a produtividade, diminuindo a ansiedade e melhorando a saúde Arnaldo Neto
(4.5/5)
Gratuito
Desbloqueie o poder da sua mente: Programe o seu subconsciente para se libertar das dores e inseguranças e transforme a sua vida Michael Arruda
(4.5/5)
Gratuito
Lei da atração: Conecte-se ao universo através de afirmações positivas Jenny Hashkins
(4.5/5)
Gratuito
18 Maneiras de ser uma pessoa mais confiante Tom Hope
(4/5)
Gratuito
Plena Voz: A arte e a prática da presença vocal Barbara McAfee
(5/5)
Gratuito
Até que nada mais importe: Como viver longe de um mundo de performances religiosas e mais próximo do que Deus espera de você Luciano Subirá
(4.5/5)
Gratuito
10 Hábitos de uma pessoa de sucesso James Fries
(4.5/5)
Gratuito
10 Maneiras de não se importar com o que os outros pensam de você James Fries
(4.5/5)
Gratuito

.NET Fest 2019. Оля Гавриш. Машинное обучение для .NET программистов

  1. 1. Тема доклада Тема доклада Тема доклада KYIV 2019 Machine Learning for .NET developers .NET CONFERENCE #1 IN UKRAINE Olia Gavrysh
  2. 2. Тема доклада Тема доклада Тема доклада .NET LEVEL UP About me .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Olia Gavrysh Program Manager Microsoft, .NET team twitter: @oliagavrysh
  3. 3. Тема доклада Тема доклада Тема доклада Let me learn something about you…
  4. 4. Make Machine Learning
  5. 5. When you start Machine Learning without calculus
  6. 6. Тема доклада Тема доклада Тема доклада .NET LEVEL UP Agenda .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1. Machine Learning crash course 2. Building ML model with for your .NET app with ML.NET
  7. 7. Machile Learning crash course KYIV 2019 .NET CONFERENCE #1 IN UKRAINE
  8. 8. © Microsoft Corporation Examples Predicts prices for the next month Identify faces in images and videos Detect fraud transactions Classify if a customer is at retention risk Some problems are difficult to solve using traditional algorithms and procedural programming. These examples are good candidates for machine learning.
  9. 9. © Microsoft Corporation A repetitive decision or process Solution lacks an explicit definition A lot of historic data When can you use Machine Learning?
  10. 10. .NET LEVEL UP Machine Learning “Programming the UnProgrammable” .NET CONFERENCE #1 IN UKRAINE KYIV 2019 rooms, bedrooms, bathrooms location, view, near school footage year built garage, basement, patio … {f(x) {f(x)
  11. 11. Types of Machine Learning
  12. 12. Many ML Tasks Is this A or B? How much? How many? How is this organized? Regression ClusteringClassification And many more…
  13. 13. Types of Machine Learning
  14. 14. How ML works ŷ = f(x) Fcost = |y - ŷ| → 0 ŷ - our model y – actual values (known answers) Fcost - shows the difference between your prediction and the actual values
  15. 15. Overfitting and Underfitting
  16. 16. .NET LEVEL UP Creating ML Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Train Evaluate UseBuild
  17. 17. .NET LEVEL UP Building Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Build 1. Upload Data 2. Prepare Data 3. Choose Algorithm
  18. 18. .NET LEVEL UP Training Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Running the chosen algorithm on the data. Train
  19. 19. .NET LEVEL UP Evaluating Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Calculate metrics that show how good is the model using test data. If not good – go back to Build phase. Evaluate All metrics: https://docs.microsoft.com/dotnet/machine-learning/resources/metrics
  20. 20. .NET LEVEL UP Consuming Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Consume in your client applications. Use
  21. 21. Building a model with KYIV 2019 .NET CONFERENCE #1 IN UKRAINE
  22. 22. © Microsoft Corporation ML.NET Machine Learning framework for building custom ML Models Build for .NET developers Proven at scale Azure, Office, Windows Extensible TensorFlow, ONNX and Infer.NET Cross-platform and open-source Runs everywhere Custom ML made easy with tools CLI + UI-based tool for building models
  23. 23. In data science 80% of the time is spent on preparing the data and 20% of the time is spent on complaining about the need to prepare the data.
  24. 24. © Microsoft Corporation ML.NET Tooling to help you with making decisions AutoML Model Builder
  25. 25. © Microsoft Corporation How much is the taxi fare for 1 passenger going from Boryspil to Kreschatyk?
  26. 26. © Microsoft Corporation Criterion Loss Min Samples Split Min Samples Leaf XYZ Parameter 1 Parameter 2 Parameter 3 Parameter 4 … Distance Trip time Car type Passengers Time of day … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Distance Gradient Boosted 30% Model Car type Passengers Getting started with Machine Learning can be hard ML.NET takes care of data prep, feature selection & hyperparameter tuning Which algorithm? Which parameters?Which features?
  27. 27. © Microsoft Corporation N Neighbors Weights Metric P ZYX Which algorithm? Which parameters?Which features? Distance Trip time Car type Passengers Time of day … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Nearest Neighbors Criterion Loss Min Samples Split Min Samples Leaf XYZ 50% Model Iterate 30% Gradient BoostedDistance Car brand Year of make Car type Passengers Trip time Getting started w/machine learning can be hard ML.NET takes care of data prep, feature selection & hyperparameter tuning
  28. 28. © Microsoft Corporation Which algorithm? Which parameters?Which features? 50% 30% 70%30%45%50%65%95%35%10%75%20%70%30%15% Iterate Getting started w/machine learning can be hard A ML.NET takes care of data prep, feature selection & hyperparameter tuning
  29. 29. © Microsoft Corporation 25%40%70% 25% 95% 25% 25% 25% 25% 40% 40% 40% 40% 70% 70% 70%Enter data Define goals Apply constraints Input Intelligently test multiple models in parallel Optimized model 95% ML.NET accelerates model development
  30. 30. © Microsoft Corporation Demo ML.NET
  31. 31. http://dot.net/ml http://aka.ms/mlnetsamples http://aka.ms/mlnetdocs http://aka.ms/mlnet Resources
  32. 32. Тема доклада Тема доклада Тема доклада KYIV 2019 Thank you! twitter: @oliagavrysh
  33. 33. Тема доклада Тема доклада Тема доклада How long to train *Dataset Size Dataset Type Avg. Time to train* 0 - 10 Mb Numeric and Text 10 sec 10 - 100 Mb Numeric and Text 10 min 100 - 500 Mb Numeric and Text 30 min 500 - 1 Gb Numeric and Text 60 min 1 Gb+ Numeric and Text 3 hour+ The exact time to train is a function of a few parameters like: • The number of features or columns being used to predict • The type of columns i.e. text vs. numeric • The Type of machine learning task (e.g. regression vs. classification) We have tested Model Builder with even 1TB dataset but building a high-quality model for that size of dataset can take up to four days.
  34. 34. 1. Supervised and not supervised 2. Types of ML problems (https://docs.microsoft.com/en-us/dotnet/machine- learning/tutorials/index) Pictures for diff problem types: https://docs.microsoft.com/en-us/dotnet/machine- learning/automate-training-with-model-builder 1. Data prep https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to- guides/prepare-data-ml-net 2. Parameters, hyperparameter, labels 3. Training set evaluation set. Cross validation 4. Success metrics: accuracy, … https://github.com/dotnet/machinelearning- samples/blob/master/modelbuilder/readme.md#evaluate All metrics here: https://docs.microsoft.com/en-us/dotnet/machine- learning/resources/metrics 5. How to improve results: more time, more/better data (the last – that’s where data scientists are needed)
  35. 35. Price prediction step by step (one hot encoder, …): https://docs.microsoft.com/en- us/dotnet/machine-learning/tutorials/predict-prices
  36. 36. Difference between machine learning and AI: • If it’s written in Python, it’s probably machine learning • If it’s written in PowerPoint, it’s probably AI

Notas

  • In our projects, we are often asked to solve very hard problems.

    Many of these problems are relatively easy for a human to solve, but are difficult to program a computer to do.

    Consider these examples…

    How do you write code that determines whether a Netflix customer will renew their subscription or not?
    How do you code an application that can differentiate objects in real time from a video feed?
    How do you determine whether parts in a manufacturing line are defective?

    How do you solve these problems when the only tool available for us is procedural code and traditional algorithms.?


  • The 3 problems we just saw all have 3 common properties:

    One, they involve a repeated decision or evaluation process.
    Two, it is difficult or impossible to explicitly describe the solution.
    And three, you do have labeled data – or existing examples where you can describe the situation, and map it to the correct result.

    This is where ML can help!

    However you cannot use ML, if you do not have enough data – this is because you only get a model that is as good as the quality of your sampling data. Further, whenever you can use code to achieve your desired solution, there is no need to leverage ML as the computing resources needed can be very expensive.
  • .NET is a great tech stack for building a wide variety of applications. There is ASP.NET for web development, Xamarin for mobile development and with ML.NET we are trying to make .NET great for Machine Learning.
  • 25
  • ML.NET provides tooling that makes it easy to use. In particular, 2 really valuable tools are: AutoML and Model Builder

    What is AutoML? It is an API that accelerates model development for you. A lot of developers do not have the experience required to build or train Machine Learning models. With AutoML, the process of finding the best algorithm, is automated!

    Model Builder on the other hand provides an easy to understand visual interface to build, train, and deploy custom machine learning models. Prior machine learning expertise is not required. It also supports AutoML


    Rememeber depending on your data, giving you the error of each of the models and you can then decide which model to use. Most people just use the model with the least error.

    And we will see it in action soon.

  • The data scientist is definitely not happy with 30%. They try a second time with a different algorithm and this time they maybe score 50%.
  • This guesswork of what features with what algorithm, goes on and on, until the data scientist finds one that performs the best with a score close to 100% as much as possible.

    Often one gets tired and just goes with whichever model is good enough.
  • AutoML, however, replaces the data scientist’s repeated model selection effort.

    What this means, is that even without a data science background, you can now build a model, by just leveraging AutoML.

    All you need to do:
    is load you data
    define your goal (are you trying to classify objects into 2 categories or are you trying to predict a value based on past values?)
    and apply constraints for example (want performance of 70% and above)
  • In addition to ML.NET accelerating the model development phase, it also provides you with model explainability.

    As you can see above, Model A’s sensitivity to different features is different from that of Model B. Trip distance was very important is model A while Trip time was most important in Model B.

    Model explainability is very important because today there are so many people who build models and at the end are not sure which pieces of information were weighted heavily over the others by the resulting model.

    BUT AutoML provides with both autoML and model explainability.

    Now let us see AutoML in action!
  • In addition to ML.NET accelerating the model development phase, it also provides you with model explainability.

    As you can see above, Model A’s sensitivity to different features is different from that of Model B. Trip distance was very important is model A while Trip time was most important in Model B.

    Model explainability is very important because today there are so many people who build models and at the end are not sure which pieces of information were weighted heavily over the others by the resulting model.

    BUT AutoML provides with both autoML and model explainability.

    Now let us see AutoML in action!

    • ×