A presentation for course 91.543 Artificial Intelligence I took in Fall 2008

1. By Beibei Yang and Zheng Fang
January 7, 2009 1

2. Introduction
Machine Learning: A computer program is said to learn from
experience E with respect to some class of tasks T and performance
measure P, if its performance at tasks in T, as measured by P, improves
with experience E
ith i E.
Overfitting: Given a hypothesis space H, a hypothesis h H is said to
overfit the training data if there exists some alternative hypothesis h’
H,
H such th t h h smaller error th h’ over th t i i examples, b t
h that has ll than the training l but
h’ has smaller error than h over the entire distribution of instances.
Decision tree: (aka classification trees or regression trees) In these
tree structures, leaves represent classifications and b
t t t l t l ifi ti d branches represent
h t
conjunctions of features that lead to those classifications.
Our implementation used the ID3 algorithm, which uses the entropy and
gain of each node t create a classification t
i f h d to t l ifi ti tree.
January 7, 2009 2

3. Cause of Overfitting (1) Lack of training data
Standard
vs.
Insufficiency
January 7, 2009 3

4. Cause of Overfitting (2) Biased data
Standard
vs.
Bias
January 7, 2009 4