👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
Voting Based Learning Classifier System for Multi-Label Classification
1. Voting-Based Learning
Classifier System
for multi-label classification
Kaveh Ahmadi-Abhari (Presenter)
Ali Hamzeh
Sattar Hashemi
IWLCS 2011 – Dublin, Ireland, 13th July 2011
2. Multi-label Classification
Single Label Classification
Exclusive classes: each
example belongs to one
class
Multi-label Classification
Each instance can belong
to more than one class
Kaveh Ahmadi-Abhari 2 Shiraz University, Soft Computing Group
3. Multi-label Classification
Sky
People
Single Label Classification
Exclusive classes: each
example belongs to one
class
Multi-label Classification
Each instance can belong
to more than one class
Sand
Kaveh Ahmadi-Abhari 3 Shiraz University, Soft Computing Group
4. Current Methods
Problem • Transfer problem to a single-
Transformation label classification problem
Algorithm • Adapt single-label classifiers
Adaptation to Solve the problem
[Tsoumakas & Katakis, 2007]
Kaveh Ahmadi-Abhari 4 Shiraz University, Soft Computing Group
7. Motivation
A lot has been done in terms of classifications
using LCSs
Most of these studies have been conducted for
single-label classification problems
Multi-label classification is in its inception [Vallim
et al., IWLCS 08]
Kaveh Ahmadi-Abhari 7 Shiraz University, Soft Computing Group
8. Voting Based Learning Classifier System
How can we guide the discovery mechanism
(e.g. evolutionary operators) in LCSs?
Kaveh Ahmadi-Abhari 8 Shiraz University, Soft Computing Group
9. Voting Based Learning Classifier System
How can we guide the discovery mechanism
(e.g. evolutionary operators) in LCSs?
Using the prior knowledge gained from
past experiences
Kaveh Ahmadi-Abhari 9 Shiraz University, Soft Computing Group
10. Voting Based Learning Classifier System
How can we guide the discovery mechanism
(e.g. evolutionary operators) in LCSs?
Using the prior knowledge gained from
past experiences
Training instances vote their matched rules
according to how correct the rule is
Kaveh Ahmadi-Abhari 10 Shiraz University, Soft Computing Group
11. Voting Based Learning Classifier System
How can we guide the discovery mechanism
(e.g. evolutionary operators) in LCSs?
Using the prior knowledge gained from
past experiences
Training instances vote their matched rules
according to how correct the rule is
Fitness measure
Kaveh Ahmadi-Abhari 11 Shiraz University, Soft Computing Group
12. Voting Defining Rule Types
How can the given votes describe the
quality of the rules accurately?
Define different types for the rules such that each of these types
describes the quality status the rule might have.
Kaveh Ahmadi-Abhari 12 Shiraz University, Soft Computing Group
13. Rule Types
Example:
in a single-label classification problem, rule types
might be correct or wrong.
Each rule might receive a “correct” or “wrong” vote from each
matched training instance.
A rule receives a combination of “correct” and “wrong” votes from its
matched training instances
Kaveh Ahmadi-Abhari 13 Shiraz University, Soft Computing Group
14. Votes as Fitness Measure
• Given votes
• Describe the quality of the rules
• Use as a fitness measure for
guiding the discovery mechanism.
• For example, a rule with more “wrong”
votes, should be discovered with a high
probability to achieve a meaningful rule
Kaveh Ahmadi-Abhari 14 Shiraz University, Soft Computing Group
15. Rules Definition
Antecedent / Consequent
###1 / 110
0011 / 001
Antecedent part matches with the feature vector.
Consequent part are the classes predicted by the rule.
One bit for each class in the consequent part.
Value 1 in the bit indicates existence of the respective class.
Kaveh Ahmadi-Abhari 15 Shiraz University, Soft Computing Group
16. VLCS Vote Types for Multi-label Problem
Correct
Wrong Subset
Multi-label
Vote Types for
VLCS
Partial Superset
Kaveh Ahmadi-Abhari 16 Shiraz University, Soft Computing Group
18. VLCS Voting Options for Multi-label Problem
Correct Rules (C) 111
000
001
1, 4
110
1, 3
00# /1001 2, 4 010
1, 2
101
• Is correct when it matches with: 011
• 000 or 100
• 001
Kaveh Ahmadi-Abhari 18 Shiraz University, Soft Computing Group
19. VLCS Voting Options for Multi-label Problem
Wrong Rules (W) 111
000
001
1, 4
110
1, 3
0#0/0010 2, 4 010
1, 2
101
• Is wrong when it matches with: 011
• 000 or 100
• 010
Kaveh Ahmadi-Abhari 19 Shiraz University, Soft Computing Group
20. VLCS Voting Options for Multi-label Problem
Subset Rules 111
000
001
1, 4
110
1, 3
#01/1000 2, 4 010
1, 2
101
• Is subset when it matches with: 011
• 001 or 100
• 101
Kaveh Ahmadi-Abhari 20 Shiraz University, Soft Computing Group
21. VLCS Voting Options for Multi-label Problem
Subset Rules 111
000
001
1, 4
110
1, 3
#01/1000 2, 4 010
1, 2
101
• Is subset when it matches with: 011
• 001 or 100
• 101
Excepted Classes:
1, 4
Kaveh Ahmadi-Abhari 21 Shiraz University, Soft Computing Group
22. VLCS Voting Options for Multi-label Problem
Superset Rules 111
000
001
1, 4
110
1, 3
#00/1101 2, 4 010
1, 2
101
• Is superset when it matches with: 011
• 001 or 100
• 101
Kaveh Ahmadi-Abhari 22 Shiraz University, Soft Computing Group
23. VLCS Voting Options for Multi-label Problem
Superset Rules 111
000
001
1, 4
110
1, 3
#00/1101 2, 4 010
1, 2
101
• Is superset when it matches with: 011
• 001 or 100
• 101
Excepted Classes:
1, 4
Kaveh Ahmadi-Abhari 23 Shiraz University, Soft Computing Group
24. VLCS Voting Options for Multi-label Problem
Partial-set Rules 111
000
001
1, 4
110
1, 3
#1# / 0110 2, 4 010
1, 2
101
• Is superset when it matches with: 011
• 010 or 100
• 111
Kaveh Ahmadi-Abhari 24 Shiraz University, Soft Computing Group
25. VLCS Voting Options for Multi-label Problem
Partial-set Rules 111
000
001
1, 4
110
1, 3
#1# / 0110 2, 4 010
1, 2
101
• Is superset when it matches with: 011
• 010 or 100
• 111
Excepted Classes:
2, 4
Kaveh Ahmadi-Abhari 25 Shiraz University, Soft Computing Group
26. VLCS Voting Options for Multi-label Problem
000
Rules might receive different votes 111
001
during the time 1, 4
110
1, 3
2, 4 010
1, 2
#0# / 1001 101
011
100
Kaveh Ahmadi-Abhari 26 Shiraz University, Soft Computing Group
27. VLCS Voting Options for Multi-label Problem
000
Rules might receive different votes 111
001
during the time 1, 4
110
1, 3
2, 4 010
1, 2
#0# / 1001 101
011
100
Is correct for
instance 000
Kaveh Ahmadi-Abhari 27 Shiraz University, Soft Computing Group
28. VLCS Voting Options for Multi-label Problem
000
Rules might receive different votes 111
001
during the time 1, 4
110
1, 3
2, 4 010
1, 2
#0# / 1001 101
011
100
Is correct for Is partial-set
instance 000 for instance
101
Kaveh Ahmadi-Abhari 28 Shiraz University, Soft Computing Group
29. Using Stored Prior Knowledge
Consider a rule that all received votes
are superset } Information
}
The rule is covering an appropriate area
of the problem
Inference
The rule is predicting greater number
of classes for the matched input
instance
The number of the classes the rule
predicts should be subtracted
Kaveh Ahmadi-Abhari 29 Shiraz University, Soft Computing Group
30. Discovery Operators
In the discovery mechanism an evolutionary algorithm with
four mutation operators is defined:
Kaveh Ahmadi-Abhari 30 Shiraz University, Soft Computing Group
31. Discovery Operators
Mutation operators on rule’s antecedent part
Generalize the rule by flipping the 0
MA-G or 1 bits to #
Specializes the rule by flipping #
MA-S bits to 1 or 0
Kaveh Ahmadi-Abhari 31 Shiraz University, Soft Computing Group
32. Discovery Operators
Mutation operators on rule’s consequent part
Subtract the number of predicted
MC-S classes by flipping 1 bits to 0
Adds more classes to predicted
MC-A classes by flipping 0 bits to 1
Kaveh Ahmadi-Abhari 32 Shiraz University, Soft Computing Group
33. Which Discovery Operator?
The votes each rule has received guide which mutation
operator should act.
Kaveh Ahmadi-Abhari 33 Shiraz University, Soft Computing Group
34. Which Discovery Operator?
The votes each rule has received guide which mutation
operator should act.
Wrongly Subtract the
assigned some number of
Superset Rule
non-expected predicted
classes classes (MC-S)
Kaveh Ahmadi-Abhari 34 Shiraz University, Soft Computing Group
36. Mutation Rate
• Mutation operator performs bit flipping
using a probability, which is the mutation
rate.
• The strength of a rule is the amount of
reward we predict the system to receive if
the rule acts.
• The more the strength, the less the mutation
rate.
Kaveh Ahmadi-Abhari 36 Shiraz University, Soft Computing Group
37. Strength of a Rule
The mean of the rewards the rule gets over time.
Reward Function:
C rule ∆C expected
R = 1−
C rule C expected
Alteration of [Vallim et al., GECCO’ 08]
Kaveh Ahmadi-Abhari 37 Shiraz University, Soft Computing Group
38. Strength of a Rule
The mean of the rewards the rule gets over time.
Reward Function:
C rule ∆C expected
R = 1−
C rule C expected
A ∆B
= {x : ( x ∈ A ) ⊕ ( x ∈ B )}
Alteration of [Vallim et al., GECCO’ 08]
Kaveh Ahmadi-Abhari 38 Shiraz University, Soft Computing Group
40. Experimental Results
Data Sets:
Two binary datasets in the bioinformatics domain
[Chan and Freitas, GECCO’ 06 ]
Extracted from [Alves et al., 2009]
Kaveh Ahmadi-Abhari 40 Shiraz University, Soft Computing Group
41. Experimental Results
Quality Metrics:
Accuracy
• Proportion of predicted classes among all predicted or
true classes
Precision
• Proportion of true classes among all predicted classes
Recall
• Proportion of predicted classes among all true classes
[Tsoumakas & Katakis, 2007]
Kaveh Ahmadi-Abhari 41 Shiraz University, Soft Computing Group
42. Experimental Results
For the VLCS, we use a 5-fold cross validation in which the
training part is used to evaluate the rules using the voting
mechanism described above.
Fixed size population
initially are the most general possible rules.
In each generation, each rule is voted by its matched
instances
reward is assigned
Defined mutation operators to discover new rules
The combination of the best rules among the parents and the
off-springs make the next generation.
We stop the training phase if the mean strength of the rules
decreases in a number of consecutive generations.
Kaveh Ahmadi-Abhari 42 Shiraz University, Soft Computing Group
43. Experimental Results
[Chan and Freitas, GECCO’ 06 ]
135 instances
152 attributes
Two classes
• Each instance could have one or both of the available class labels.
Method Accuracy Precision Recall
BR 0.89 0.89 0.87
ML-KNN 0.91 0.93 0.91
VLCS 0.89 0.89 0.89
Kaveh Ahmadi-Abhari 43 Shiraz University, Soft Computing Group
44. Experimental Results
Extracted from [Alves et al., 2009]
7877 proteins
40 attributes
Six classes
• Each instance could have some of the available class labels.
Method Accuracy Precision Recall
BR 0.78 0.77 0.78
ML-KNN 0.80 0.81 0.80
VLCS 0.81 0.83 0.82
Kaveh Ahmadi-Abhari 44 Shiraz University, Soft Computing Group
45. Conclusion
Guiding the discovery mechanism
with a prior knowledge, such that is
used in VLCS, can help us solve
applicable problems
Kaveh Ahmadi-Abhari 45 Shiraz University, Soft Computing Group
46. Future Work
A representation for dealing with numeric and nominal
datasets.
Future studies on scalability and stability of the system is
necessary.
Additional studies on system performance in dealing with
imbalanced data and noise is also required.
Improving evolutionary operators, guiding mechanism and
rule refinement.
Kaveh Ahmadi-Abhari 46 Shiraz University, Soft Computing Group
47. Any Question?
The most exciting phrase to hear in
science, the one that heralds new
discoveries is not “Eureka”! (I found
it!) but “That's funny...”
- Isaac Asimov
Kaveh Ahmadi-Abhari 47 Shiraz University, Soft Computing Group