1. Improving Evolutionary Testing by Means of
Efficiency Enhancement Techniques
Matteo Miraz, Pier Luca Lanzi, and Luciano Baresi
(presented by Daniele Loiacono)
Politecnico di Milano, Italy

2. 2
Table of Contents
1. TestFul: an Evolutionary Test Approach for Java
2. Improving the Approach
1. Seeding
2. Hybridization
3. Fitness Inheritance
3. Conclusions and Future Work
DEI

3. Testful 3
• Leverages Search-Based Techniques to generate Tests
• We are not alone:
• Search Based Software Engineering is a vibrant community!
• Targets stateful systems (e.g., Java classes)
• A sequence of operations
is required to exercise
the features of a class
enables
DEI

4. Test Representation 4
• A test is a sequence of operations
• Assign a primitive value to a variable
• char_0 = „c‟
• Create an object
• cut_1 = new CUT(char_0)
• Invoke a method on an object
• char_1 = cut_1.fooBar(int_1)
• Working on variables
• Includes all the parameters‟ types
• Enables polymorphism
• Calculate compatibilities among types
DEI

5. Our Evolutionary Testing Approach 5
Goal: Find tests that put objects in interesting states
• We use a NSGA-II Evolutionary Algorithm, guided by:
 Number of statements and branches covered
• Tests with a high branch coverage stress more deeply the class
under test and put objects in more interesting states
 Compactness of tests
• Unnecessary long tests waste computational resources during their
evaluation
• Recombination:
• variable-length one-point crossover, replacement mutation
crossover mutation
DEI

6. Need for Efficiency Enhancement Techniques 6
• Testful achieves good results
• generates good tests for complex classes (GECCO ‟09 poster)
• It requires huge computational requirements, because:
• It starts the evolution from a population with a poor quality.
• Most of the initial tests do not exercise any feature
• The most difficult elements in the class are not exercised
• It does not provide any guidance to exercise uncovered elements
• The fitness function only counts the number of branch exercised
• The fitness function is expensive
• It requires to execute a test to measure its branch coverage
DEI

7. Seeding 7
• Problem:
• Testful starts the evolution from a population with a poor quality.
• Solution:
• Run an inexpensive technique (random search) for a short time,
and use its result as initial population
Branch Coverage with a Random Search
DEI

8. Seeding 8
Simple Problem
(Fraction)
Complex Problem
(Red-Black Tree)
DEI

9. Hybridization 9
• Problem:
• Testful does not provide any guidance to exercise uncovered
branches
• Solution:
• Hybridize the evolutionary search
with a local search
• It focuses on the easiest
branch not yet executed
(works at method level)
• It adopts a simpler search
strategy (hill climbing)
• Which element to improve?
• One of the best elements
• 5% of the population
• 10% of the population
DEI

10. Hybridization 10
Simple Problem
(Fraction)
Complex Problem
(Disjoint Set Fast)
DEI

11. Fitness Inheritance 11
• Problem:
• The fitness function is expensive
• Solution:
• Use Fitness Inheritance
• Evaluate the “real” fitness only on a part of the population
• Other individuals inherit the fitness of their parents
• Which policy to select individuals to evaluate?
• Uniform selection
• Fronteer selection: evaluate more often individuals on the frontier
• How to evaluate the improvements?
• Number of generation evaluated w.r.t. elapsed time
• Number of branch covered
• (to evaluate the impact of a noisier fitness function)
DEI

12. Fitness Inheritance: # of generations 12
Complex Problem
(Stack Array)
More complex Problem
(Red-Black Tree)
DEI

13. Fitness Inheritance: branch executed 13
Simple Problem
(Fraction)
Computationally-intensive Problem
(Disjoint Set Fast)
DEI

14. Conclusions and Future Work 14
• Efficiency enhancement techniques allow Testful to reach better
results using less computational resources
• The improvement is even higher if the three efficiency
enhancement techniques are combined
• We plan to leverage even more on the Seeding technique
• To test newer versions of a class
• To test similar classes of a class
DEI

15. Improving Evolutionary Testing by Means of
Efficiency Enhancement Techniques
M. Miraz, P. L. Lanzi, L. Baresi
{miraz,lanzi,baresi}@elet.polimi.it
http://code.google.com/p/testful/