A lot of people talk about Data Mining, Machine Learning and Big Data. It clearly must be important, right? A lot of people are also trying to sell you snake oil - sometimes half-arsed and overpriced products or solutions promising a world of insight into your customers or users if you handover your data to them. Instead, trying to understanding your own data and what you could do with it, should be the first thing you’d be looking at. In this talk, we’ll introduce some basic terminology about Data and Text Mining as well as Machine Learning and will have a look at what you can on your own to understand more about your data and discover patterns in your data.

1. Introduction to
Data Mining
Kai Koenig
@AgentK

2. Web/Mobile Developer since the late 1990s
Interested in: Java & JVM, CFML, Functional
Programming, Go, Android, Data Science
And this is my view of the world…
Me

4. 1.What is Data Mining?
2. Concepts and Terminology
3.Weka
4.Algorithms
5. Dealing with Text
6. Java integration
Agenda

6. We are overwhelmed
with data.

8. 1.What is Data Mining?

9. Fundamentals
Why do we nowadays have SO MUCH data?
Reasons include:
- Cheap storage and better processing power
- Legal & Business requirements
- Digital hoarding

10. Fundamentals
Data Mining is all about going from data to useful
and meaningful information.
- Recommendation in online shops
- Finding an “optimal” partner
- Weather prediction
- Judgement decisions (credit applications)

11. Fundamentals

12. A better definition
“Data Mining is defined as the process of
discovering patterns in data.The process must be
automatic or (more usually) semiautomatic.The
patterns discovered must be meaningful in that
they lead to some advantage, often an economic
one.”
(Prof. Dr. Ian Witten)

13. How can you express patterns?

14. Finding and applying rules
Tear Production
Rate == reduced
none

15. Finding and applying rules
Age == young &&
Astigmatism == no
soft
Age == young &&
Astigmatism == no
soft

16. A Result: Decision lists
If outlook = sunny and humidity = high then play = no
If outlook = rainy and windy = true then play = no
If outlook = overcast then play = yes
If humidity = normal then play = yes
If none of the above then play = yes

17. Not all rules are equal
Classification rules: predict an outcome
Association rules: rules that strongly associate
different attribute values
If temperature = cool then humidity = normal
If humidity = normal and windy = false then play = yes
If outlook = sunny and play = no then humidity = high

18. 2. Concepts and
Terminology

19. Learning
What is Learning? And what is Machine Learning?
A good approach is:
“Things learn when they change their
behaviour in a way that makes them perform
better in the future”

20. Learning types
Classification learning
Association learning
Clustering
Numerical Prediction

21. Some basic terminology
The thing to be learned is the concept.
The output of a learning scheme is the
concept description.
Classification learning is sometimes called
supervised learning. The outcome is the
class.
Examples are called instances.

24. Some more basic terminology
Discrete attribute values are usually called
nominal values, continuous attribute values are
called just numeric values.
Algorithms used to process data and find
patterns are often called classifiers.There are
lots of them and all of them can be heavily
configured.

26. 3.Weka

27. What is Weka?
Waikato Environment for Knowledge Analysis
Developed by a group in the Dept. of Computer
Science at the University of Waikato in New
Zealand.
Also,Weka is a New Zealand-only bird.

28. What is Weka?
Download for Mac OS X, Linux and Windows:
http://www.cs.waikato.ac.nz/~ml/weka/
index.html
Weka is written in Java, comes either as native
applications or executable .jar file and is licensed
under GPL v3.

29. Getting data into Weka
Easiest and common for experimenting: .arff
Also supported: CSV, JSON, XML, JDBC
connections etc.
Filters in Weka can then be used to preprocess
data.

30. Features
50+ Preprocessing tools
75+ Classification/Regression algorithms
~10 clustering algorithms
… and a packet manager to load and install
more if you want.

31. 4.Algorithms

32. Classifiers
There are literally hundreds with lots of tuning
options.
Main Categories:
- Rule-based (ZeroR, OneR, PART etc.)
- Tree-based (J48, J48graft, CART etc.)
- Bayes-based (NaiveBayes etc.)
- Functions-based (LR, Logistic etc.)
- Lazy (IB1, IBk etc.)

33. OneR
Very simplistic classifier and based on a single
attribute.
For each attribute,
For each value of that attribute, make a rule as follows:
count how often each class appears
find the most frequent class
make the rule assign that class to this attribute value.
Calculate the error rate of the rules.
Choose the rules with the smallest error rate.

34. C4.5 (J48)
Produces a decision tree, derived from divide-
and-conquer tree building techniques.
Decision trees are often verbose and need to be
pruned - J48 uses post-pruning, pruning can in
some instances be costly.
J48 usually provides a good balance re quality vs.
cost (execution times etc.)

35. NaiveBayes
Very good and popular for document (text)
classification.
Based on statistical modelling (Bayes formula of
conditional probability)
In document classification we treat the existence
or absence of a word as a Boolean attribute.

37. Training and Testing
We implicitly trained and tested our classifiers in
the previous examples using Cross-Validation.

38. Training and Testing
Test data and Training data NEED to be different.
If you have only one dataset, split it up.
n-fold Cross-Validation:
- Divides your dataset into n parts, holds out
each part in turn
- Trains with n-1 parts, tests with the held out
part
- Stratified CV is even better

40. 5. Dealing with Text

41. Bag of Words
Generally for document classification we treat a
document as a bag of words and the existence
or absence of a word is a Boolean attribute.
This results in problems with very many
attributes having 2 values each.
This is quite a bit different from the usual
classification problem.

43. Filtered Classifiers
First step: use Filtered classifier with J48 and
StringToWordVector filter.
Example: Reuters Corn datasets (train/test)
We get 97% accuracy, but there’s still an issue
here -> investigate the confusion matrix
Is accuracy the best way to evaluate quality?

44. Better approaches to evaluation
Accuracy: (a+d)/(a+b+c+d)
Recall: R = d/(c+d)
Precision: P = d/(b+d)
F-Measure: 2PR/(P+R)
False positive rate FP: b/(a+b)
True negative rate TN: a/(a+b)
False negative rate FN: c/(c+d)
predicted
– +
true
– a b
+ c d

45. ROC (threshold) curves
Area under the threshold curve determines the
overall quality of a classifier.

47. NaiveBayesMultinomial
Often the best classifier for document
classification. In particular:
- good ROC
- good results on minority class (often what we
want)

48. NaiveBayesMultinomial
J48: 96% accuracy, 38/57 on grain docs, 544/547
on non-grain docs, ROC 0.91
NaiveBayes: 80% accuracy, 46/57 on grain docs,
439/547 on non-grain docs, ROC 0.885
NaiveBayesMultinomial: 91% accuracy, 52/57 on
grain docs, 496/547 on non-grain docs, ROC
0.973

50. NaiveBayesMultinomial
NaiveBayesMultinomial with stoplist, lowerCase
and outputWords: 94% accuracy, 56/57 on grain
docs, 504/547 on non-grain docs, ROC 0.978
Why? NBM is designed for text:
- based solely on word appearance
- can deal with multiple repetitions of a word
- faster than NB

51. 6. Java integration

52. Weka is written in Java
The UI is essentially making use of a vast
underlying data mining and machine learning
API.
Obviously this fact
invites us to use the
API directly :)

53. Setting up a project (IntelliJ IDEA)
Create new Java project in IntelliJ
Import weka.jar
Import weka-src.jar
Off you go!

54. The main classes/packages you need…
import weka.classifiers.Evaluation;
import weka.classifiers.trees.J48;
import weka.core.Instances;

55. Getting stuff done
Instances train = new Instances(bReader);
train.setClassIndex(train.numAttributes()-1);
J48 j48 = new J48();
j48.buildClassifier(train);
Evaluation eval = new Evaluation(train);
eval.crossValidateModel(
j48,
train,
10,
new Random(1));

56. You can also grab Java code off Weka UI

57. Photo Credits
https://www.flickr.com/photos/johnnystiletto/3339808858/
https://www.flickr.com/photos/theequinest/5056055144/
https://www.flickr.com/photos/flyingkiwigirl/17385243168
https://www.flickr.com/photos/x6e38/3440973490/
https://www.flickr.com/photos/42931449@N07/5418402840/
https://www.flickr.com/photos/gerardstolk/12194108005/
https://www.flickr.com/photos/zzpza/3269784239/in/
https://www.flickr.com/photos/internationaltransportforum/
14258907973/

58. Get in touch
Kai Koenig
Email: kai@ventego-creative.co.nz
www.ventego-creative.co.nz
Blog: www.bloginblack.de
Twitter: @AgentK