Recomendados
Mais conteúdo relacionado
Semelhante a C3.1 intro
Semelhante a C3.1 intro (20)
Mais de Daniel LIAO
Último
Último (20)
C3.1 intro
- 1. Machine Learning Specialization Classification: A machine learning perspective Emily Fox & Carlos Guestrin Machine Learning Specialization University of Washington ©2015-2016 Emily Fox & Carlos Guestrin
- 2. Machine Learning Specialization Part of a specialization ©2015-2016 Emily Fox & Carlos Guestrin
- 3. Machine Learning Specialization3 This course is a part of the Machine Learning Specialization ©2015-2016 Emily Fox & Carlos Guestrin 1. Foundations 2. Regression 3. Classification 4. Clustering & Retrieval 5. Recommender Systems 6. Capstone
- 4. Machine Learning Specialization What is the course about? ©2015-2016 Emily Fox & Carlos Guestrin
- 5. Machine Learning Specialization5 What is classification? From features to predictions ©2015-2016 Emily Fox & Carlos Guestrin Data ML Method IntelligenceClassifier Input x: features derived from data Predict y: categorical “output”, class or label Learn xày relationship
- 6. Machine Learning Specialization6 Sentiment classifier ©2015-2016 Emily Fox & Carlos Guestrin Easily best sushi in Seattle. Sentence Sentiment Classifier Input x: Output: y Sentiment
- 7. Machine Learning Specialization7 Classifier ©2015 Emily Fox & Carlos Guestrin Sentence from review Classifier MODEL Input: x Output: y Predicted class
- 8. Machine Learning Specialization8 Example multiclass classifier Output y has more than 2 categories ©2015 Emily Fox & Carlos Guestrin Education Finance Technology Input: x Webpage Output: y
- 9. Machine Learning Specialization9 Spam filtering Input: x Output: y Not spam Spam ©2015 Emily Fox & Carlos Guestrin Text of email, sender, IP,…
- 10. Machine Learning Specialization10 Image classification ©2015 Emily Fox & Carlos Guestrin Input: x Image pixels Output: y Predicted object
- 11. Machine Learning Specialization11 Personalized medical diagnosis ©2015 Emily Fox & Carlos Guestrin Disease Classifier MODEL Input: x Healthy Flu … Cold Pneumonia Output: y
- 12. Machine Learning Specialization12 Reading your mind ©2015-2016 Emily Fox & Carlos Guestrin Output y “Hammer” “House” Inputs x are brain region intensities
- 13. Machine Learning Specialization Impact of classification ©2015-2016 Emily Fox & Carlos Guestrin
- 14. Machine Learning Specialization14 Impact of classification ©2015-2016 Emily Fox & Carlos Guestrin
- 15. Machine Learning Specialization Course overview ©2015-2016 Emily Fox & Carlos Guestrin
- 16. Machine Learning Specialization16 Course philosophy: Always use case studies & … Core concept Practical Visual Implement Algorithm Advanced topics ©2015-2016 Emily Fox & Carlos Guestrin OPTIONAL
- 17. Machine Learning Specialization17 Overview of content Models Linear classifiers Logistic regression Decision trees Ensembles Algorithms Gradient Stochastic gradient Recursive greedy Boosting Core ML Alleviating overfitting Handling missing data Precision- recall Online learning ©2015-2016 Emily Fox & Carlos Guestrin
- 18. Machine Learning Specialization Course outline ©2015-2016 Emily Fox & Carlos Guestrin
- 19. Machine Learning Specialization19 Overview of modules Models Linear classifiers Module 1 Logistic regression Modules 1, 2, 3 Decision trees Modules 4 & 5 Ensembles Module 7 Algorithms Gradient Modules 2 & 3 Stochastic gradient Module 9 Recursive greedy Module 4 Boosting Module 8 Core ML Alleviating overfitting Modules 3 & 5 Handling missing data Module 6 Precision-recall Module 8 Online learning Module 9 ©2015-2016 Emily Fox & Carlos Guestrin
- 20. Machine Learning Specialization20 Module 1: Linear classifiers ©2015-2016 Emily Fox & Carlos Guestrin #awesome #awful 0 1 2 3 4 … 0 1 2 3 4 … Score(x) > 0 Score(x) < 0 Word Coefficient #awesome 1.0 #awful -1.5 Score(x) = 1.0 #awesome – 1.5 #awful
- 21. Machine Learning Specialization21 Module 1: Logistic regression represents probabilities ©2015-2016 Emily Fox & Carlos Guestrin P(y=+1|x,ŵ) = 1 . ⌃ 1 + e-ŵ h(x) T
- 22. Machine Learning Specialization23 Module 2: Learning “best” classifier ©2015-2016 Emily Fox & Carlos Guestrin #awesome #awful 0 1 2 3 4 … 0 1 2 3 4 … Maximize likelihood over all possible w0,w1,w2 ℓ(w0=1, w1=0.5, w2=-1.5) = 10-4 ℓ(w0=1, w1=1, w2=-1.5) = 10-5 ℓ(w0=0, w1=1, w2=-1.5) = 10-6 Best model with gradient ascent: Highest likelihood ℓ(w) ŵ = (w0=1, w1=0.5, w2=-1.5)
- 23. Machine Learning Specialization25 Module 3: Overfitting & regularization ©2015-2016 Emily Fox & Carlos Guestrin Model complexity Classification error True error Training error ℓ(w) - ||w||2(w) - ||w||2λ 2 Use regularization penalty to mitigate overfitting
- 24. Machine Learning Specialization26 Module 4: Decision trees ©2015-2016 Emily Fox & Carlos Guestrin Start Credit? Safe excellent Income? poor Term? Risky Safe fair 5 years 3 years Risky Low Term? Risky Safe high 5 years 3 years
- 25. Machine Learning Specialization27 Module 5: Overfitting in decision trees ©2015-2016 Emily Fox & Carlos Guestrin Logistic Regression Degree 2 featuresDegree 1 features Decision Tree Depth 3Depth 1 Depth 10 Degree 6 features
- 26. Machine Learning Specialization28 Module 5: Alleviate overfitting by learning simpler trees ©2015-2016 Emily Fox & Carlos Guestrin Complex Tree Simplify Simpler Tree Occam’s Razor: “Among competing hypotheses, the one with fewest assumptions should be selected”, William of Occam, 13th Century
- 27. Machine Learning Specialization30 Module 6: Handling missing data ©2015-2016 Emily Fox & Carlos Guestrin Credit Term Income y excellent 3 yrs high safe fair ? low risky fair 3 yrs high safe poor 5 yrs high risky excellent 3 yrs low risky fair 5 yrs high safe poor ? high risky poor 5 yrs low safe fair ? high safe Start Credit? Safe excellent Income? poor Term? Risky Safe fair 5 years 3 years Risky Low Term? Risky Safe high 5 years 3 years or unknown or unknown or unknown or unknown
- 28. Machine Learning Specialization32 Module 7: Boosting question “Can a set of weak learners be combined to create a stronger learner?” Kearns and Valiant (1988) Yes! Schapire (1990) Boosting Amazing impact: Ÿ simple approach Ÿ widely used in industry Ÿ wins most Kaggle competitions ©2015-2016 Emily Fox & Carlos Guestrin
- 29. Machine Learning Specialization33 ©2015-2016 Emily Fox & Carlos Guestrin Module 7: Boosting using AdaBoost f1(xi) = +1 Ensemble: Combine votes from many simple classifiers to learn complex classifiers Income>$100K? Safe Risky NoYes Income>$100K? Safe Risky NoYes f2(xi) = -1 Credit history? Risky Safe GoodBad f3(xi) = -1 Savings>$100K? Safe Risky NoYes f4(xi) = +1 Market conditions? Risky Safe GoodBad
- 30. Machine Learning Specialization34 Module 8: Precision-recall ©2015-2016 Emily Fox & Carlos Guestrin Need an automated, “authentic” marketing campaign Reviews SpokespeopleGreat quotes “Easily best sushi in Seattle.” Goal: increase # guests by 30% PRECISION Did I (mistakenly) show a negative sentence??? RECALL Did I not show a (great) positive sentence??? Accuracy not most important metric
- 31. Machine Learning Specialization35 Module 9: Scaling to huge datasets & online learning 500M Tweets/day4.8B webpages 5B views/day Avg.loglikelihood Better Gradient Stochastic gradient: tiny modification to gradient, a lot faster, but annoying in practice ©2015-2016 Emily Fox & Carlos Guestrin
- 32. Machine Learning Specialization Assumed background ©2015-2016 Emily Fox & Carlos Guestrin
- 33. Machine Learning Specialization37 Courses 1 & 2 in this ML Specialization • Course 1: Foundations - Overview of ML case studies - Black-box view of ML tasks - Programming & data manipulation skills • Course 2: Regression - Data representation (input, output, features) - Linear regression model - Basic ML concepts: • ML algorithm • Gradient descent • Overfitting • Validation set and cross-validation • Bias-variance tradeoff • Regularization ©2015-2016 Emily Fox & Carlos Guestrin
- 34. Machine Learning Specialization38 Math background • Basic calculus - Concept of derivatives • Basic vectors • Basic functions - Exponentiation ex - Logarithm ©2015-2016 Emily Fox & Carlos Guestrin
- 35. Machine Learning Specialization39 Programming experience • Basic Python used - Can pick up along the way if knowledge of other language ©2015-2016 Emily Fox & Carlos Guestrin
- 36. Machine Learning Specialization40 Reliance on GraphLab Create • SFrames will be used, though not required - open source project of Dato (creators of GraphLab Create) - can use pandas and numpy instead • Assignments will: 1. Use GraphLab Create to explore high-level concepts 2. Ask you to implement all algorithms without GraphLab Create • Net result: - learn how to code methods in Python ©2015-2016 Emily Fox & Carlos Guestrin
- 37. Machine Learning Specialization41 Computing needs ©2015-2016 Emily Fox & Carlos Guestrin • Basic 64-bit desktop or laptop • Access to internet • Ability to: - Install and run Python (and GraphLab Create) - Store a few GB of data
- 38. Machine Learning Specialization Let’s get started! ©2015-2016 Emily Fox & Carlos Guestrin