Antonio Mora, Antonio Fernández-Ares, Juan-J Merelo-Guervós, Pablo García-Sánchez This work describes an evolutionary algorithm (EA) for evolving the constants, weights and probabilities of a rule-based decision engine of a bot designed to play the Planet Wars game. The evaluation of the individuals is based on the result of some non-deterministic combats, whose outcome depends on random draws as well as the enemy action, and is thus noisy. This noisy fitness is addressed in the EA and then, its effects are deeply analysed in the experimental section. The conclusions shows that reducing randomness via repeated combats and re-evaluations reduces the effect of the noisy fitness, making then the EA an effective approach for solving the problem.

1. Dealing with Noisy Fitness in the Design of a RTS Game Bot
A.M. Mora , A. Fernández-Ares, J.J. Merelo,
P. García-Sánchez, y C.M. Fernandes

2. Index
Problem description
Baseline: GoogleBot
First Bot: AresBot
GeneBot
Experiments
Noise fitness study
Conclusions

3. Introduction
Google AI Challenge 2010.
Design a bot that can play to:
Planet Wars

5. Simple view of Planet Wars Game

6. Problem Description
Actual List of
Bot
State actions
Restrictions
RTS->Real time strategy with pseudo-turns
1 turn = 1 second
Forbidden use of memory between turns.
Actual state: planets and fleets.
List of Actions: un simple file text witch the movement of
fleets from a planets belongs to player and another planet.

7. Baseline: GoogleBot
It was include in the Initial Kit of the contest
How it works?
Choose the BASE planet (the one with most of its starship)
Choose the TARGET planet (calculating the ratio between the grow
rate and the number of ships)
It wastes the rest of time until the attack has finished

8. First Bot: AresBot
For each turn:
Choose the BASE planet (the one with most starships). The
rest of planet are called COLONIES.
Choose the TARGET (who are not already being attacked):
EXPANSION: if TARGET is neutral.
CONQUEST: if TARGET belongs to enemy.
Extra actions for each COLONY:
TITHE?
SUPPORT ATTACK?

9. Diagram of states

10. Parameters of AresBot
titheperc: percentage of fleets (of the stored) that
COLONIE sent to the BASE.
titheprob: probability of the TITHE for each COLONIE.
ωNS−DIS: weight of the number of starship hosted at the
planet and the distance form the BASE to the TARGET
(it’s used to the score function of target planet).
ωGR: weight of the planet growth rate (it’s used to the score
function of target planet).

11. Parameters of AresBot II
poolperc: proportion of extra starships that the bot sends
from the base planet to the target planet.
supportperc: percentage of extra starships that the bot
sends from each COLONIE to TARGET.
supportprob: probability of sending extra fleets from one
COLONIE to TARGET (if COLONIE is closer to the TARGET than
BASE are).

12. Operation “GeneBot”
Using intergalactic techniques (GA) improves AresBot
to create the powerful Genebot.
Gen: array of parameters (standardized values)
2-Tournament
BLX-alpha crossover
Re-evaluation

13. Fitness
For each “bot” fight against the GoogleBot in 5
characteristic maps.
One bot is better than other if:
Wins in more maps.
Needs less turns, in case of tie.

14. Noise Fitness Study
Unpromising Bot
Fitness assigned on first execution: 2057
3005
2505
2005
fitness
1505
1005
505
5
0 10 20 30 40 50 60 70 80 90 100
Evaluation #
Wins in 5 maps Loses on any of the 5 maps

15. Noise Fitness Study II
Promising Bot
Fitness assigned on firts execution: 578
3005
2505
2005
fitness
1505
1005
505
5
0 10 20 30 40 50 60 70 80 90 100
Evaluation #
Wins in 5 maps Loses on any of the 5 maps

16. Experiments
Num. Generations: 100
Num. Individuals: 200
Crossover prob: 0.6
α : 0.5
Mutation prob: 0.02
Replacement Policy: 2-elitims

17. Experiments II
Each single “evaluation” takes 40 seconds.
40 seconds * 200 * 100 ≈ 9.25 days of execution
• SOLUTION → Parallel algorithm
• N evaluation at same time. (N = 4 in experiments)
• Make 15 executions, getting 15 “best bots”.

18. Experiments III
GA
Evaluation
Bot Bot Bot Bot
Fitness Fitness Fitness Fitness
Simulator Simulator Simulator Simulator
GoogleBot GeneBot GoogleBot GeneBot GoogleBot GeneBot GoogleBot GeneBot

19. Results
Relationship of fitness of best individual of each execution
900
857
800
700
687
Fitness (less is better)
600
590
559
500 527 531 528 528 540 525 528 525
458 470
400 425
300
200
100
0
E01 E02 E03 E04 E05 E06 E07 E08 E09 E10 E11 E12 E13 E14 E15
Executions

20. Results II
Relationship of the best bot wins each execution
100%
98% 98% 99% 98% 98%
90% 94%
90% 91% 90%
89%
87%
85% 84%
80% 82% 83%
Winning percentage in 100 maps
70%
60%
50%
40%
30%
20%
10%
0%
E01 E02 E03 E04 E05 E06 E07 E08 E09 E10 E11 E12 E13 E14 E15

21. Results III
titheperc titheprob ωNS−DIS ωGR poolperc supportperc supportprob
AresBot 0,1 0,5 1 1 0,25 0,5 0,9
GeneBot 0,0179 0,00823 0,50954 0,23273 0,73321 0,58946 0,97405
Average 0,17386 0,09702 0,47252 0,36409 0,65732 0,59987 0,59987

22. Result IV
Fitness Evolution
700
Average
650 Best execution
The best of all
executions
Aggregated number of turns
600
550
500
450
400
350
G0 G5 G10 G15 G20 G25 G30 G35 G40 G45 G50 G55 G60 G65 G70 G75 G80 G85 G90 G95 G100

23. Conclusions and future work
The First Genebot finished in the 1454th position in the contest (36%), but
it had executed just once (with less individuals and generations).
Genebot wins 99% of battle versus AresBot
Our fitness is a good measure of the bot.
To DO:
Distributed algorithm.
Multi-objective.
Co-evolution.
Recognition of environment and specialist bots
Macro and micro evolution…

24. Thanks for your time…
Any question?