overview of Machine Learning

1. MACHINE LEARNING
Priyadharshini .R
Mtech-(IT)
Reg no:2016246017 1

2. Evolution of Machine Learning
2

3. What is machine learning?
A branch of artificial intelligence,
concerned with the design and development
of algorithms that allow computers to evolve
behaviors based on empirical data.
As intelligence requires knowledge, it is
necessary for the computers to acquire
knowledge.
Definition of Machine learning:
Machine learning focuses on the
development of computer programs that can
access data and use it learn for themselves.
3

4. History of Machine Learning
• 1950s
– Samuel’s checker player
– Selfridge’s Pandemonium
• 1960s:
– Neural networks: Perceptron
– Pattern recognition
– Learning in the limit theory
– Minsky and Papert prove limitations of Perceptron
• 1970s:
– Symbolic concept induction
– Winston’s arch learner
– Expert systems and the knowledge acquisition
bottleneck
– Quinlan’s ID3
4

5. History of Machine Learning (cont.)
• 1980s:
– Advanced decision tree and rule learning
– Explanation-based Learning (EBL)
– Learning and planning and problem solving
– Utility problem
– Analogy
– Cognitive architectures
– Resurgence of neural networks (connectionism,
backpropagation)
– Valiant’s Learning Theory
– Focus on experimental methodology
5

6. 1990s:
-Data mining
-Adaptive software agents and web applications
-Text learning
-Reinforcement learning (RL)
-Inductive Logic Programming (ILP)
-Ensembles: Bagging, Boosting, and Stacking
-Bayes Net learning
History of Machine Learning (cont.)
6

7. History of Machine Learning (cont.)
• 2000s
– Support vector machines
– Kernel methods
– Graphical models
– Statistical relational learning
– Transfer learning
– Sequence labeling
– Collective classification and structured outputs
– Computer Systems Applications
• Compilers
• Debugging
• Graphics
• Security (intrusion, virus, and worm detection)
7

8. Traditional Programming
Machine Learning
Computer
Data
Program
Output
Computer
Data
Output
Program
8

9. Types of Learning
• Supervised (inductive) learning
– Training data includes desired outputs
• Unsupervised learning
– Training data does not include desired outputs
• Semi-supervised learning
– Training data includes a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
9

10. Many online software packages & datasets
• onlineData sets
• UC Respirtory
• http://www.kdnuggets.com/datasets/index.html
• Software (much related to data mining)
• JMIR Open Source
• Weka
• Shogun
• RapidMiner
• ODM
• Orange
• CMU
• Several researchers put their software online
10

11. Training and testing
Training set
(observed)
Universal
set
(unobserved)
Testing set
(unobserved)
Data acquisition Practical usage
11

12. 12
Learning (training): Learn a model using the training data
Testing: Test the model using unseen test data to assess the
model accuracy
Training and testing

13. Optimization
Combinatorial optimization
– E.g.: Greedy search
Convex optimization
– E.g.: Gradient descent
Constrained optimization
– E.g.: Linear programming
13

14.  The success of machine learning system also depends on the
algorithms.
 The algorithms control the search to find and build the
knowledge structures.
 The learning algorithms should extract useful information
from training examples.
Algorithms
14

15.  Supervised learning
– Prediction
– Classification (discrete labels), Regression (real values)
 Unsupervised learning
– Clustering
– Probability
– estimation
– Finding association (in features)
– Dimension reduction
 Semi-supervised learning
 Reinforcement learning
– Decision making (robot, chess machine)
Algorithms
15

16. Algorithms
Supervised learning Unsupervised learning
Semi-supervised learning 16

17.  Supervised learning categories and techniques
 Linear classifier (numerical functions)
 Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden
Markov models (HMM), Probabilistic graphical models
 Non-parametric (Instance-based functions)
• K-nearest neighbors, Kernel regression, Kernel density estimation,
Local regression
 Non-metric (Symbolic functions)
• Classification and regression tree (CART), decision tree
 Aggregation
• Bagging (bootstrap + aggregation), Adaboost, Random forest
Learning techniques
17

18.  Unsupervised learning categories and techniques
Clustering
• K-means clustering
• Spectral clustering
Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis
Learning techniques
18

19. Issues in Machine Learning
• Problem representation / feature extraction
• Intention/independent learning
• Integrating learning with systems
• What are the theoretical limits of learnability
• Transfer learning
• Continuous learning
19

20. Measuring Performance
• Generalization accuracy
• Solution correctness
• Solution quality (length, efficiency)
• Speed of performance
20

21. Working Applications of ML
• Electrical power control
• Chemical process control
• Character recognition
• Face recognition
• DNA classification
• Credit card fraud detection
• Cancer cell detection
21

22. Current Machine Learning Research
• Representation
– data sequences
– spatial/temporal data
– probabilistic relational models
– …
• Approaches
– ensemble methods
– cost-sensitive learning
– active learning
– semi-supervised learning
– collective classification
– …
22

23. We have a simple overview of some techniques and
algorithms in machine learning. Furthermore, there are more and
more techniques apply machine learning as a solution. In the
future, machine learning will play an important role in our daily
life.
Conclusion
23

24. [1] W. L. Chao, J. J. Ding, “Integrated Machine
Learning Algorithms for Human Age
Estimation”, NTU, 2011.
Reference
24