Software testing, test case generation, constraints, search-based testing

1. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Constraint-based Fitness Function
for Search-based Software Testing
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Abdelilah Sakti
Yann-Ga¨l Gu´h´neuc
e
e e
Gilles Pesant
Department of Computer and Software Engineering
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e
May 21, 2013
CPAIOR, Yorktown Heights
Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab

2. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
2 / 14
What is the cost of a programming error ?

3. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
2 / 14
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.

4. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
2 / 14
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.

5. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.
Only a high quality software can reduce errors cost.
Testing is an important technique for validating and
checking the correctness of software.
Despite that the testing may cost more than 50% of the
budget of critical software, it remains an inevitable
phase in the life cycle of software.
2 / 14

6. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
3 / 14
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.

7. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.
3 / 14

8. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.
One main challenge is automating inputs data generation.
3 / 14

9. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
4 / 14
Search Based Software Testing (SBST)

10. Constraint-based
Fitness Function
for Search-based
Software Testing
Search Based Software Testing (SBST)
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Fitness Evaluation
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
The Fitness Evaluation is the core of SBST.
4 / 14

11. Constraint-based
Fitness Function
for Search-based
Software Testing
Search Based Software Testing (SBST)
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)
Fitness Evaluation
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
The Fitness Evaluation is the core of SBST.
An evolution in the fitness evaluation may bring a significant
enhancement to SBST
4 / 14

12. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
5 / 14
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

13. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
tinyy == z
tinyy > 0
tinyx == 10
Evaluation
Conclusions
5 / 14
tinyTarget
tinyEND

14. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
tinyy > 0
tinyx == 10
Evaluation
Conclusions
5 / 14
tinyTarget
tinyEND

15. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
Conclusions
5 / 14
tinyTarget
tinyEND

16. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
3
fSE

17. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
3+ 90
91
fSE

18. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
3+ 90
91
fSE
90
91

19. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
3+ 90
91
fSE
90
91
+ 31
32

20. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
3+ 90
91
90
91
+ 31
32
fSE
+0

21. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
i2
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
90
3+ 91 = 3.9890
2 + 21 =2.9545
22
fSE
31
+ 32 +0= 1.9577
+ 21 + 20 =1.9068
22
21
90
91
0

22. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Motivating Example
tinyx, y, z
1
2
Introduction
3
Motivating
Example
5
Fitness Function
Based on Branch
Hardness
4
6
int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
i1 : (10,−30,60)
i2 : (30, −20, −20)
tinyy == z
i1
tinyy > 0
i2
Critical Branch of i1
tinyx == 10
Evaluation
tinyTarget
Conclusions
tinyEND
Test candidate
i1
i2
5 / 14
fAL
90
3+ 91 = 3.9890
2 + 21 =2.9545
22
fSE
31
+ 32 +0= 1.9577
+ 21 + 20 =1.9068
22
21
90
91
0
Both fitness functions favour i2 over i1 because either they
do not consider non-executed branches or they ignore some
branches’ future.

23. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
6 / 14
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.

24. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
6 / 14
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).

25. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
6 / 14
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.

26. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
6 / 14
Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.
2. A projection tightness close to 0 will indicate high
constrainedness and hardness to satisfy a constraint.

27. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
7 / 14
Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B 2 ·
1
arityc
+ B · (1 − tightness) + 1 .

28. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B 2 ·
1
arityc
+ B · (1 − tightness) + 1 .
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
Difficulty Level (DL)
DL is based on DC ranking (r) ;
DL is a representation of a relative hardness level of a
constraint in a set of constraints ;
DL(c, C) =
7 / 14
|C|,
2r−1 · (|C| + 1),
if r = 0
.
if r > 0

29. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
8 / 14
Branch-Hardness Fitness Functions
DC Fitness Function (fDC )
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC (i, C) =
c∈C
DC(c) · η(i, c) .

30. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Branch-Hardness Fitness Functions
DC Fitness Function (fDC )
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC (i, C) =
c∈C
DC(c) · η(i, c) .
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
DL Fitness Function (fDL )
DL is used as a constant penalty for breaking a
constraint in a set of constraints to satisfy ;
fDL (i, C) =
where (i, c) =
8 / 14
c∈C
(i, c) + η(i, c),
0,
if η(i, c) = 0
.
DL(c, C), if η(i, c) = 0

31. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
9 / 14
Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].
DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.

32. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].
DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example
Evaluation
Conclusions
(fDC ) for each test candidate
90
fDC (i1 , C) = 60.95· 91 +106· 31 +110.95· 0 = 162.9677 ;
32
1
fDC (i2 , C) = 60.95· 0 +106· 21 +110.95· 20 = 206.8485.
1
22
21
Contrary to fAL and fSE this fitness function makes the
adequate choice by choosing the test candidate i1 instead of
i2 .
9 / 14

33. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
10 / 14
Evaluation
Subjects and configuration
analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL .

34. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Evaluation
Subjects and configuration
analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL .
Subjects
440 synthetic test targets that were randomly generated.
20 executions for every combination of fitness function
and meta-heuristic algorithm.
If test data was not found after 25000 (respectively
100000) fitness evaluations for EA (respectively SA),
the search was terminated.
10 / 14

35. Constraint-based
Fitness Function
for Search-based
Software Testing
Evaluation
EA Results
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
EA: Comparing all fitnesses on 440 test targets
Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
80
60
Evaluation
Subjects and configuration
EA Results
40
SA Results
Conclusions
20
0
0
5000
10000
15000
Evaluations
11 / 14
20000
25000

36. Constraint-based
Fitness Function
for Search-based
Software Testing
Evaluation
SA Results
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
SA: Comparing all fitnesses on 440 test targets
Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level
Introduction
80
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results
Conclusions
Branch Coverage %
Motivating
Example
60
40
20
0
0e+00
2e+04
4e+04
6e+04
Evaluations
12 / 14
8e+04
1e+05

37. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
13 / 14
Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.

38. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.
Future work
Performing more experiments on real world programs ;
Extending our approach by defining new ways to
compute projection tightness for constraints involving
other data types than integer.
13 / 14

39. Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions
14 / 14
Thank you
Questions ?