7. Machine Learning
"Field of study that gives computers the ability to learn
without being explicitly programmed” (Arthur Samuel, 1959)
7
8. What is Machine Learning?
8
Computer
Computer
Traditional Programming
Machine Learning
Data
Data
Program
Output
Program
Output
9. Sweet spot for Machine Learning
• It’s impossible to write down the rules in code:
• Too many rules
• Too many factors influencing the rules
• Too finely tuned
• We just don’t know the rules (image recognition)
• Lots of labeled data (examples) available (e.g. historical data)
9
10. Basic Machine Learning ‘workflow’
10
Feature
Vectors
Training
data
Labels
Machine
Learning
Algorithm
Feature
Vectors
New data Prediction
Training Phase
Operational Phase
Predictive
Model
11. Training Phase in more detail
11
Raw data
Data
preparation Feature
Vectors
Training
Data
Test
data
Model Building
(by ML
algorithm)
Model
Evaluation
Predictive
Model
Feedback loop
data cleansing
data transformation
normalization
feature extraction
aka
‘learning’
15. ML Algorithms: by Representation
Collection of candidate models/programs, aka hypothesis space
15
Decision trees
Instance-based
Neural networks
Model ensembles
16. ML Algorithms: by Evaluation
Evaluation: Quality measure for a model
16
Regression
Example metric: Root Mean Squared Error
RMSE =
Binary classification: confusion matrix
Accuracy: 8 + 971 -> 97,9%
Example: medical test
for a disease
Positive Negative
P
True
positives
TP
False
Negatives
FN
N
False
positives
FP
True
Negatives
TN
True
Class
Predicted class
Accuracy: Better evaluation metrics:
• Precision: 8 / (8 + 19)
• Recall: 8 / (8 + 2)
17. Optimization: how the algorithm ‘learns’, depends on representation and
evaluation
ML Algorithms: by Optimization
17
Greedy Search,
ex. of
combinatorial
optimization
Gradient Descent (or in general: Convex Optimization)
Linear Programming (or in general:
Constrained/Nonlinear Optimization)
19. Data Science for Business
• Focuses more on general principles
than specific algorithms
• Not math-heavy, does contain some
math
• O’Reilly link:
http://shop.oreilly.com/product/063692
0028918.do
• Book website: http://data-science-for-
biz.com/DSB/Home.html
19
20. Take-aways
• Goal of ML: generalize from training data (not optimization!!)
• Part of ‘Data Mining Process’, not a goal in and of itself
• No magic! Just some clever algorithms…
• Increasingly important non-technical aspects:
• Ethics
• Algorithmic transparency
20
Source for images: http://www.havlena.net/en/machine-learning/machine-learning-what-is-it-where-to-learn-about-it/
Go (DeepMind’s AlphaGo). How it works: https://www.tastehit.com/blog/google-deepmind-alphago-how-it-works/ Go is very different to Chess (DeepBlue 1996). Chess works with a game tree + sophisticated evaluation function. Go is too complex, and there are no good evaluation functions, because Go positions are harder to evaluate. Enter Monte Carlo Tree Search: simulation. Exploration/exploitation trade-off! No Go-knowledge required!
This diagram is attributed to Pedro Domingos who used it in his Coursera Machine Learning course in 2012.
