1. Regression Methods in
Machine Learning
Categorical Variable Conversion
Portland Data Science Group
Andrew Ferlitsch
Community Outreach Officer
July, 2017

2. Linear Regression
• All the features (independent variables) need to be a
real number.
• CANNOT be a categorical value, ie., a named or
enumerated value.
• Example:
Male vs. Female
Red, Blue, Green
Apple, Banana, Pear, Orange

3. Categorical Variables
Age Gender Income
25 Male 25000
26 Female 22000
30 Male 45000
24 Female 26000
Independent Variables (Features)
Dependent Variables (Label)
Real Values Value to Predict
Categorical Values

4. Dummy Variable Conversion
Known in Python as OneHotEncoder
For each categorical feature:
1. Scan the dataset and determine all the unique instances.
2. Create a new feature (i.e., dummy variable) in dataset, one
per unique instance.
3. Remove the categorical feature from the dataset.
4. For each sample (row), set a 1 in the feature (dummy
variable) that corresponds to that categorical value instance,
and:
5. Set a 0 in the remaining features (dummy variables) for that
categorical field.
6. Remove one dummy variable field.

5. Dummy Variable Trap
Gender
Male
Female
Male
Female
Need to Drop one Dummy Variable!
Male Female
1 0
0 1
1 0
0 1
x1 x2 x3
Multicollinearity occurs when one variable predicts another.
i.e., x2 = ( 1 – x3)
As a result, a regression analysis cannot distinguish between the
contribution of x2 and x3.

6. Drop one of Dummy Variables
Age Male Income
25 1 25000
26 0 22000
30 1 45000
24 0 26000
Drop one of the Dummy Variables
Age Gender Income
25 Male 25000
26 Female 22000
30 Male 45000
24 Female 26000
Gender is Replaced with Male
Age Race Income
20 White Apple
26 Hispanic 22000
30 Asian 45000
24 Asian 26000
Age White Asian Income
20 1 0 Apple
26 0 0 22000
30 0 1 45000
24 0 1 26000
Dropped Hispanic (i.e., Hispanic = White: 0, Asian: 0)