On Partitioned Fitness Distributions of Genetic Operators for Predicting GA Performance
1. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
1 of 25
On Partitioned Fitness-Distributions
of Genetic Operators for Predicting
GA Performance
WSC6
6th Online World Conference on Soft Computing in
Industrial Applications
Departamento de Lenguajes y
Ciencias de la Computación
Universidad de Málaga
Rafael Nogueras, Carlos Cotta
September 10-24, 2001
2. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
2 of 25
Outline of the
Presentation
1. Introduction
2. Virtual Genetic Algorithms
3. Enhancing the Model
4. Experimental Results
5. Conclusions
Outline of the Presentation
3. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
3 of 25
• The computational cost of running a GA can be
overwhelming.
• Maximizing GA effectiveness minimizing the
number of fitness calculations needed to achieve
a certain solution.
• A careful instantiation of the basic template is
required (popsize, pX, pm, …).
• Tools for predicting GA performance are
necessary:
▪ Information-based
▪ Fitness-based
Introduction
Introduction
4. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
4 of 25
Virtual Genetic Algorithms
• Virtual Genetic Algorithms (VGAs) constitute a
statistical approach aimed at GA-performance
prediction.
• VGAs vs GAs: individuals do not carry
genetic information; they are used as place-
holders for fitness information.
• Fitness is transmitted from parents to
offspring, using a statistical model to simulate
the effects of genetic operators.
• This statistical model is utilized to predict the
fitness of a new individual in terms of the
fitnesses of its parents.
Virtual Genetic
Algorithms
5. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
5 of 25
Sketch of a VGA
Initialize population
Termination
condition?
Select parents
Apply virtual
operators
Insert in population
NO
YES
Statistical model
fo = fo(fp,P)
Parameters of the model
Inferred off-line
Virtual Genetic
Algorithms
Sketch of a VGA
6. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
6 of 25
• Based on a linear regression approach for
each genetic operator.
• Input: a dataset comprising N pairs (fp, fo).
– fp = (mean) fitness of random (pair of) solution(s).
– fo = fitness of the solution obtained by applying an
genetic operator to the solution(s) above.
• Output: a tuple (a, b, sa, sb).
– a, b = Least-squares fit of fo = a+ bfp to the dataset.
– sa, sb = Least-squares fit of so = sa + sb fp, where
so and fp are the standard deviation of fo and
the average of fp when grouped in bins of K
pairs.
The Basic VGA
Virtual Genetic
Algorithms
The Basic VGA
7. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
7 of 25
The Basic VGA
• The VGA predicts the descendant fitness using
a Gaussian distribution: fo=N(a+bfp, sa+ sbfp) .
• This approach just provides discrete results.
OneMax Brachystochrone
• Two major reasons: acquisition of the dataset
and asymmetry of the distribution.
Virtual Genetic
Algorithms
The Basic VGA
8. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
8 of 25
Problems of the Basic VGA
Acquisition of the dataset:
Random sampling provides significant
information in simple functions, but not in
general.
OneMax Brachystochrone
Virtual Genetic
Algorithms
Problems of the Basic
VGA
9. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
9 of 25
Problems of the Basic VGA
Asymmetry of the distribution:
The Gaussian distribution is symmetric, but
the fitness distribution can be asymmetric.
Brachystochrone Rastrigin Function
Virtual Genetic
Algorithms
Problems of the Basic
VGA
10. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
10 of 25
The L-VGA1
• An enhanced version of the VGA was obtained.
• Acquisition of the dataset: use a dataset
obtained by means of a pilot run of a GA.
– Computational cost similar to random sampling.
– The dataset covers a larger region of the fitness space.
• Asymmetry: find the parameters of the linear fit
fo=a+bfp, and split the dataset in two datasets:
– U = { (fp,fo) | fo > a + bfp }
– L = { (fp,fo) | fo a + bfp }
Virtual Genetic
Algorithms
Enhancing the Model
11. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
11 of 25
The L-VGA1
• Input: a dataset comprising N pairs (fp, fo).
• Output: a tuple
• COMPUTE-MODEL:
– a, b = Least-squares fit of fo = a+ bfp to the dataset.
– = Least-squares fit of so = sa + sb fp, where so
and fp are the standard deviation of fo and the average of fp
when grouped in bins of K pairs For U or L.
– = Least-squares fit of mo = la + lb fp, where mo
and fp are the minimum of fo and the average of fp when
grouped in bins of K pairs for U or L.
– p = relative weight of U vs. L.
)
,
,
,
,
,
,
,
, p
b
a L
b
L
a
U
b
U
a
L
b
L
a
U
b
U
a l
l
l
l
s
s
s
s
(
)
, |
| L
U
b
L
U
a s
(s
)
, |
| L
U
b
L
U
a l
(l
Virtual Genetic
Algorithms
Enhancing the Model
12. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
12 of 25
Some Results
Brachystochrone
Some Results
Virtual Genetic
Algorithms
13. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
13 of 25
Some Results
Rastrigin Function
Some Results
Virtual Genetic
Algorithms
14. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
14 of 25
Some Results
Flowshop Scheduling
Some Results
Virtual Genetic
Algorithms
15. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
15 of 25
Partitioning the Dataset
Problem of the L-VGA1:
• The underlying linear model is somewhat
inflexible.
• Different behavior of the GA in different
regions of fitness space.
• The chances for obtaining a descendant better
than the parent are higher when the latter has a
lower quality, for example, during some initial
stages of the GA run.
Partitioning the
Dataset
16. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
16 of 25
Partitioning the Dataset
Solutions:
• Use a higher-degree polynomial model, but it
does not offer a conclusive improvement for a
low-degree polynomial.
• Partition of the dataset into several prediction
intervals, and extract a model for each of them.
Flowshop Scheduling
Brachystochrone
Partitioning the
Dataset
17. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
17 of 25
The L-VGAM
Input: a dataset comprising N pairs (fp, fo).
Output: M tuples
• Sort the dataset according to fp, and partition it into M
subsets Si.
• Si = [Li, Ri), Li = Ri-1, L1 = min(fp), RM = max(fp)+1, i.e.,
Si = { ( fp,fo ) | Li fp < Ri }
• Mi = COMPUTE MODEL for Si, i=1..M, where:
Mi =
• OUTPUT the complete model M = {M1,…, MM}
)
,
,
,
,
,
,
,
, p
b
a L
b
L
a
U
b
U
a
L
b
L
a
U
b
U
a l
l
l
l
s
s
s
s
(
)
,
,
,
,
,
,
,
, p
b
a L
b
L
a
U
b
U
a
L
b
L
a
U
b
U
a l
l
l
l
s
s
s
s
(
The Enhanced VGA
Partitioning the
Dataset
18. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
18 of 25
Experimental Results
• Experiments have been done with an elitist
generational GA/VGA (popsize = 100, pX = .9,
pm=.01, maxevals=10000) and PGA/PVGA
(popsize = 300/400, pX = .9, pm = .01, maxevals =
= 20000) using ranking selection.
• All results correspond to series of 50 runs.
• Three problems have been considered: the Rastrigin
function, the Fletcher-Powell function, and flowshop
scheduling.
• Issues tackled:
– Goodness of quantitative estimation
– Goodness of qualitative estimation
– Computational speed-up
Experimental
Results
19. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
19 of 25
Quantitative Estimations
Fletcher-Powell Function
Experimental
Results
Quantitative Estimations
20. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
20 of 25
Quantitative Estimations
Flowshop Scheduling
Experimental
Results
Quantitative Estimations
21. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
21 of 25
Qualitative Estimations
Three different GA configurations are compared
on the Rastrigin Function:
A = {popsize=100, pX=.9, pm=.01}
B = {popsize=50, pX=.5, pm=.10}
C = {popsize=80, pX=.7, pm=.05}
Real GA Virtual GA
Relative properties are preserved.
Experimental
Results
Qualitative Estimations
22. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
22 of 25
Qualitative Estimations
Four different Parallel GA (PGA) configurations are
compared on a flowshop scheduling problem:
A = {GAs=1}, C = {GAs=4} B = {GAs=2}, D = {GAs=8}
Real PGA Virtual PGA
Relative properties are preserved.
Experimental
Results
Qualitative Estimations
(ring topology)
23. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
23 of 25
Qualitative Estimations
Three different PGA configurations are compared
on a flowshop scheduling problem:
A = {GAs = 2, migfreq = 1000, migrate-best}
B = {GAs = 2, migfreq = 300, migrate-random}
C = {GAs = 4, migfreq = 500, migrate-random}
Real PGA Virtual PGA
Relative properties are preserved.
Experimental
Results
Qualitative Estimations
24. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
24 of 25
Speed-Up
The larger the problem instance, the higher the
speed-up of the VGA vs. GA.
Experimental
Results
Speed-Up
25. On Partitioned Fitness-
Distributions of Genetic
Operators for Predicting
GA Performance
Outline of the
Presentation
Introduction
Virtual Genetic
Algorithms
Experimental
Results
Conclusions
Sketch of a VGA
The Basic VGA
Problems of the Basic
VGA
Enhancing the Model
Quantitative Estimations
Qualitative Estimations
Speed-Up
Some Results
Partitioning the
Dataset
The Enhanced VGA
9/26/2021
25 of 25
Conclusions
• VGAs can be a useful tool for predicting GA
performance.
• An enhanced VGA model has been proposed
to grasp some properties of the fitness
distribution such as asymmetry, statistical
anisotropy, or the different behaviour of GA.
• The obtained results show:
✓ a much more accurate performance estimation
✓ high computational speed-ups
• Future work: study other schemes for
partitioning the dataset and other non-linear
statistical models
Conclusions