Machine learning workshop, session 3. - Data sets - Machine Learning Algorithms - Algorithms by Learning Style - Algorithms by Similarity - People to follow

1. Machine Learning
Lunch & Learn - Session 3
Luis Borbon
03/07/2017

2. Table of contents
1. Recap
2. Data sets
3. Machine Learning Algorithms
4. Algorithms by Learning Style
5. Algorithms by Similarity
6. People to follow

3. Recap

4. Recap
Linear Regression
It is used to estimate real values (cost of houses,
number of calls, total sales etc.) based on
continuous variable(s).
Here, we establish relationship between
independent and dependent variables by fitting
a best line.
This best fit line is known as regression line and
represented by a linear equation Y= a *X + b.

5. Recap

6. Data Sets

7. Training set
A set of examples used for learning: to fit the
parameters of the classifier.

8. Validation set
A set of examples used to tune the parameters
of a classifier.

9. Test set
A set of examples used only to assess the
performance of a fully-trained classifier.
If you have huge amount of data the exact
proportion of training to testing to validation data
is somewhere 50:25:25 and in case of less
amount of data 60:20:20.

10. Machine Learning Algorithms

11. No Free Lunch
In a nutshell, it states that no one algorithm
works best for every problem.
As a result, you should try many different
algorithms for your problem, while using a hold-
out “test set” of data to evaluate performance
and select the winner.

13. Algorithms by Learning Style

14. Supervised Learning
Input data is called training data and has a
known label or result such as spam/not-spam or
a stock price at a time.
This algorithm consist of a target / outcome
variable (or dependent variable) which is to be
predicted from a given set of predictors
(independent variables). Using these set of
variables, we generate a function that map
inputs to desired outputs. The training process
continues until the model achieves a desired
level of accuracy on the training data.

15. Supervised Learning
Example problems are classification and
regression.
Example algorithms include:
● Regression
● Logistic Regression
● Decision Tree
● Back Propagation Neural Network.

16. Unsupervised Learning
Input data is not labeled and does not have a
known result.
In this algorithm, we do not have any target or
outcome variable to predict / estimate. It is used
for clustering population in different groups,
which is widely used for segmenting customers
in different groups for specific intervention.

17. Unsupervised Learning
Example problems are clustering, dimensionality
reduction and association rule learning.
Example algorithms include:
● Apriori algorithm
● k-Means

18. Semi-Supervised Learning
Input data is a mixture of labeled and unlabelled
examples.
There is a desired prediction problem but the
model must learn the structures to organize the
data as well as make predictions.

19. Semi-Supervised Learning
Example problems are classification and
regression.
Example algorithms are extensions to other
flexible methods that make assumptions about
how to model the unlabeled data.

20. Algorithms by Similarity

21. Regression Algorithms
Regression is concerned with modeling the
relationship between variables that is iteratively
refined using a measure of error in the
predictions made by the model.
Regression methods are a workhorse of statistics
and have been co-opted into statistical machine
learning. This may be confusing because we can
use regression to refer to the class of problem
and the class of algorithm. Really, regression is a
process.

22. Regression Algorithms
The most popular regression algorithms are:
● Ordinary Least Squares Regression (OLSR)
● Linear Regression
● Logistic Regression
● Stepwise Regression
● Multivariate Adaptive Regression Splines
(MARS)
● Locally Estimated Scatterplot Smoothing
(LOESS)

23. Instance-based Algorithms
Instance-based learning model is a decision
problem with instances or examples of training
data that are deemed important or required to
the model.
Such methods typically build up a database of
example data and compare new data to the
database using a similarity measure in order to
find the best match and make a prediction. For
this reason, instance-based methods are also
called winner-take-all methods and memory-
based learning. Focus is put on the
representation of the stored instances and
similarity measures used between instances.

24. Instance-based Algorithms
The most popular instance-based algorithms are:
● k-Nearest Neighbor (kNN)
● Learning Vector Quantization (LVQ)
● Self-Organizing Map (SOM)
● Locally Weighted Learning (LWL)

25. Regularization Algorithms
An extension made to another method (typically
regression methods) that penalizes models
based on their complexity, favoring simpler
models that are also better at generalizing.
I have listed regularization algorithms separately
here because they are popular, powerful and
generally simple modifications made to other
methods.

26. Regularization Algorithms
The most popular regularization algorithms are:
● Ridge Regression
● Least Absolute Shrinkage and Selection
Operator (LASSO)
● Elastic Net
● Least-Angle Regression (LARS)

27. Decision Tree Algorithms
Decision tree methods construct a model of
decisions made based on actual values of
attributes in the data.
Decisions fork in tree structures until a prediction
decision is made for a given record. Decision
trees are trained on data for classification and
regression problems. Decision trees are often
fast and accurate and a big favorite in machine
learning.

28. Decision Tree Algorithms
The most popular decision tree algorithms are:
● Classification and Regression Tree (CART)
● Iterative Dichotomiser 3 (ID3)
● C4.5 and C5.0 (different versions of a
powerful approach)
● Chi-squared Automatic Interaction
Detection (CHAID)
● Decision Stump
● M5
● Conditional Decision Trees

29. People to Follow

30. Andrew Ng
American computer scientist
Andrew Yan-Tak Ng is a Chinese American computer
scientist. He is the former chief scientist at Baidu, where he
led the company's Artificial Intelligence Group. He is an
adjunct professor at Stanford University.
● Born: 20 July 1972 (age 44), London, United Kingdom
● Nationality: American
● Spouse: Carol Reiley (m. 2014)
● Doctoral advisor: Michael I. Jordan
● Education: University of California, Berkeley (1993),
Carnegie Mellon University
● Awards: TR35, Sloan Research Fellowship
● http://www.andrewng.org/
● @AndrewYNg

31. Yann LeCun
Yann LeCun is a computer scientist with contributions in
machine learning, computer vision, mobile robotics and
computational neuroscience.
He is well known for his work on optical character recognition
and computer vision using convolutional neural networks
(CNN), and is a founding father of convolutional nets.
● Born: 8 July 1960 (age 56), Paris, France
● Alma mater: Pierre-and-Marie-Curie University
● Thesis: Modeles connexionnistes de l'apprentissage
(connectionist learning models) (1987)
● Known for: Deep learning
● Notable student: Wojciech Zaremba
● Academic advisor: Geoffrey Hinton
● http://yann.lecun.com/
● @ylecun

32. Machine Learning Conference
MLconf gathers communities to discuss the recent research
and application of Algorithms, Tools, and Platforms to solve
the hard problems that exist within organizing and analyzing
massive and noisy data sets.
● http://mlconf.com/
● @MLconf