2010 Simulated Car Racing Championship @ WCCI-2010

The 2010 Simulated Car Racing
Championship @ WCCI-2010
Daniele Loiacono, Luigi Cardamone,
Martin V. Butz, and Pier Luca Lanzi

2010 Simulated Car Racing Championship @ WCCI-2010

2

2010 Simulated Car Racing Championship
9 races during 3 conferences
ACM GECCO-2010, Portland, OR (USA), July 7-
IEEE WCCI-2010, Barcelona (Spain), July 18-23
IEEE CIG-2010, Copenhagen (Denmark), August 18–21

Develop a driver for TORCS
(hand-coded, learned, evolved, …)

Drivers will be awarded based on their score
in each conference competition

At the end, the team with highest overall score
wins the championship


3

What is the structure of a race?

Three stages: warm up, qualifiers, actual race

During warm-up, each driver can explore the
track and learn something useful

During qualifiers, each driver races alone against
the clock (the best 8 drivers move to the race)

During the race all the drivers race together


Motivations

 Proposing a relevant game-based competition
more representative of commercial games AI
more similar to a real-world problem
 Proposing a funny and exciting competition
you can see and play with the entries of this competition
human players can interact with AI
a lot of programmed AI available for comparison
 Proposing a challenging competition
not designed with Machine Learning in mind
computationally expensive
real-time
dealing with a lot of practical issues


What‘s new?

If everything seems under control, you're not going fast enough
— Mario Andretti
 Warm-up stage
Before qualifying stage, competitors have 100000 game
ticks to race on the track
Allows track learning and optimization of parameters
 Noisy sensors
Track sensors and opponent sensors are affected by a
Gaussian noise (standard deviation equal to 10% of the
readings)
 Extended sensor model
Focus sensors
Z position and speed
Direction of track sensors fully customizable
Added clutch control and focus command


The Open Racing Car Simulator

 TORCS is a state of the art open source simulator written in C++

 Main features
Sophisticated dynamics
Provided with several
cars, tracks, and
controllers
Active community of
users and developers
Easy to develop your
own controller

 OS Support
Linux: binaries and building from sources
Windows: binaries and ―limited‖ bulding from sources support
OSX: legacy binaries and no building from sources support 


The Open Racing Car Simulator
& the Competition Software

TORCS TORCS
PATCH

BOT BOT BOT SBOT SBOT SBOT

 The competition server
UDP UDP UDP
 Separates the bots from TORCS
 Build a well-defined sensor model
 Works in real-time
BOT BOT BOT


Sensors and actuators

 Rangefinders for edges on the track and opponents
 Speed, RPM, fuel, damage, angle with track, distance race, position
on track, etc.

 Six effectors: steering wheel [-1,+1], gas pedal [0, +1], brake
pedal [0,+1], gearbox {-1,0,1,2,3,4,5,6}, clutch [0,+1], focus
direction

The competitors

 Five entries in the second leg
AUTOPIA, Madrid and Granada
J. Muñoz, Carlos III University of Madrid
S.Pohl, J. Quadflieg and T. Delbrügger, TU Dortmund
Joseph Alton, University of Birmingham
Timothy Alford (Xiaodong Li), RMIT University, Melbourne

 Two more entries from the 2009 championship
COBOSTAR (T. Lönneker & M.V. Butz, University of
Würzburg)
POLIMI (Cardamone, Politecnico di Milano)


AUTOPIA

Industrial Computer Science Department.
Centro de Automática y Robótica
Consejo Superior de Investigaciones Científicas
Madrid, Spain
Contact:E. Onieva (enrique.onieva@car.upm-csic.es)

Architecture Schema

 Three basic modules for gear, steering and speed control

 Steady state genetic algorithm to compute the best weights
to combine parameters for steering and target speed control

 Opponents module
Acts on steering and brake signal to overtake opponents
and avoid collisions

 Learning Module in Warm-up Stage
Factors over the target speed in certain track segments


Learning Module (Warm-Up)

 Running normally in warm-up stage.
Maintain a vector with as many real values as tracklength
in meters.
Vector initialized to 1.0
If the vehicle goes out of the track or suffers damage
then multiply vector positions from 250 meters before the
current position by 0.95.

 Vector is multiplied by F to make the driver more cautious in
function of the damage:
F=1-0.02*round(damage/1000)


Mr Racer

Susanna Pohl, Jan Quadflieg and Tim Delbrügger
TU Dortmund


Mr. Racer 2009-2010

 Mr Racer 2009
Good classifier which identifies six situations
Acceleration/brake learned offline using an EA
Model of the track learned online
Simple heuristic to use the model: override the learned
behaviour on straights and in full speed corners to drive
flat out
 Mr Racer 2010
Save the model after warmup, use it during
qualifying and the race
Use the model to derive a plan consisting of
target speeds and a racing line
Optimize the parameter set of the planning module,
left to be done for the next round


Mr. Racer - Classification

Angle based measure mapped to six situations (straight, fast
corner, slow corner, etc)


Mr. Racer 2010 – The catch

 Noise completely breaks our classifier

 Without a descent classifier we can‗t learn the track

 Without a trackmodel we can‗t drive 

 Workaround for GECCO-2010:
Classify the whole track as being straight

 New classifier for the next leg at WCCI-2010


Jorge Muñoz

Department of Computer Science
Carlos III University of Madrid


Jorge Muñoz

 Build a model of the track during the warm-up stage.

 Two neural networks to predict the trajectory using the track
model. Two neural networks to predict the target speed
given the model of the track and the current car position

 The four neural networks are trained with backpropagation
using data retrieved from a human player.

 The controller tries to imitate a human player.

 A scripted policy is used to follow the trajectory (steering
value), set the speed (accelerate and brake values), set the
clutch and the gear


Jorge Muñoz

 Other optimizations performed during
the warm-up and used in the race:
The car remember where it goes out of the car or drives
far form the trajectory and in the next laps goes slower in
those points
The car remember where it follows the trajectory perfectly
and tries to go faster in the next laps.

 Overtaking is made by means of modifying the predicted
trajectory, the modification is bigger in straighs than in turns

 To avoid being overtaken the car also modifies the trajectoy,
trying to stay in front of the opponent car.


Joseph Alton

Joseph Alton


Steering

 4 sensors at -30° and 30°
 Each sensors is mapped to the steering as:
Steering += left feeler * 0.005
Steering -= right feeler * 0.005
 Noise filtering is done through having multiple sensors at the
same position
 The effect of repeating these calculations for each sensor
means the steering grows exponentially when the feeler
proportion changes, so the driver can turn sharply enough

Full left
No left steer steer

Full right steer
No right steer


Warm-up

 During warm-up we go around the track at 60 km/h (which
is fast enough to complete all possible road tracks and obtain
good readings)
 During this time for each segment (meter) of the track we
record the biggest turn and map this to a speed.
 The speeds are as follows:

Type Value Speed ( km/h)
Sharp turn > Absolute 0.1 60
Turn > Absolute 0.05 100
Straight 200


Competitive phase look-ahead

 During the competitive phase the circular array of target
speeds for each segment is loaded.
 For each game tick we look ahead 40 segments (metres) and
pick the lowest speed as our ‗Target Speed‘.
 The effect is that the driver is always prepared for the worst
case scenario and drives safely.
 Adjusting to target speeds are handled by the ‗Speed
Control‘ module. The module aims to keep the car within a
range 5 km/h of a given ‗target speed‘ (which constantly
changes).
 If the car falls below this range acceleration is gradually
applied until this range is met. Likewise if we are above this
range the brake is gradually applied until we fall within this
ideal speed range.


Timothy Alford (supervised by Xiaodong Li)
RMIT University, Melbourne, Victoria


Overview

 All components of the car are controlled by Fuzzy Logic
(excluding gears and recovery )
 Recovery, Gears are controlled with simple rules
 GA is exploited in the Warm-up

Fuzzy system
Action object
Y
Gears

Sensors on track
?

N Action object
Recovery


Controlling the car

 Input is collected from the sensors and fuzzified this is
achieved by using membership functions.

Input from function (calculate membersip
Membership sensors (speed, angle,
[how true fuzzy to track are] for ...) set)
distance variables edge each
1

0

Fuzzy values (slow=0.2
normal=0.8 fast=0)


Controlling the car, continued

 Using the membership values calculated previously, 'fired'
rules are determined and output(s) can be inferred.

Membership values

Collection of rules
IF distance=close THEN speed=slow
IF RPM=high THEN gear=up
IF angle=negative AND speed=fast THEN turn=hard_right

Inferencing
Fired rules
process

Real world
output (crisp)

Warming up

 (1+1) ES (one parent, one child)
Parent uses already 'good' values
Generates children based on these values
Child becomes new parent if better, else rejected
 Optimisations stored in XML file
 Controller will optimise when warm-up stage is selected


COBOSTAR

Thies Lönneker and Martin V. Butz
University of Würzburg

http://www.coboslab.psychologie.uni-wuerzburg.de

CIG-2008 Champ

Luigi Cardamone
Politecnico di Milano


References

 Loiacono, D.; Lanzi, P. L.; Togelius, J.; Onieva, E.; Pelta, D.
A.; Butz, M. V.; Lonneker, T. D.; Cardamone, L.; Perez, D.;
Saez, Y.; Preuss, M.; Quadflieg, J.; , The 2009 Simulated Car
Racing Championship, Computational Intelligence and AI in
Games, IEEE Transactions on , vol.2, no.2, pp.131-147, June
2010
 Enrique Onieva, David A. Pelta, Javier Alonso, Vicente
Milanés and Joshué Pérez. A Modular Parametric Architecture
for the TORCS Racing Engine. In Proc. of IEEE Symposium on
Computational Intelligence and Games (CIG'09), pag. 256-
262, 2009
 Butz, M.V., & Lönneker, T. Optimized sensory-motor
couplings plus strategy extensions for the TORCS car racing
challenge. IEEE Symposium on Computational Intelligence in
Games, IEEE CIG 2009, 317-324.
 CIG-2009 (papers of session on racing games)


Scoring process: Warm-up Qualifying

 Scoring process involves three road tracks:
Wild-Speed
Petit
Brondehach

 All the tracks are not provided with the standard TORCS
distribution:
Petit and Brondehach are designed by TORCS users
Wild-Speed has been provided by the organizers

 Each controller raced for 100000 game ticks in the warm-up
stage and then its performance is computed in the qualifying
stage as the distance covered within 10000 game ticks


Qualifying: Wild-Speed

Timothy
8824.99
Alford

MR. Racer 7897.91

Joseph
2693.95
Alton
Jorge
10032.9
Muñoz

COBOSTAR 10483.9

Cardamone 8906.23

Autopia 10417

0 2000 4000 6000 8000 10000 12000


Qualifying: Petit

Timothy
8572.59
Alford

MR. Racer 5439.3

Joseph
3986.65
Alton
Jorge
8164.82
Muñoz

COBOSTAR 9541.39

Cardamone 7668.49

Autopia 7733.64

0 2000 4000 6000 8000 10000 12000


Qualifying: Brondehach

Timothy
787.932
Alford

MR. Racer 1645.72

COBOSTAR 9134

Cardamone 2623.54

Joseph
3597.55
Alton
Jorge
798.197
Muñoz

Autopia 7010.28

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000


Qualifyng summary

Competitor Wild-Speed Petit Brondehach Total

COBOSTAR 10 10 10 30

Autopia 8 5 8 21

Jorge Muñoz 6 6 3 15

Cardamone 5 4 5 14

Timothy Alford 4 8 2 14

Mr. Racer 3 3 4 10

Joseph Alton 2 2 6 10


How much does noise affect the
performance?


Petit with and without noise

Timothy 8736.34
Alford 8572.59

9451.8
MR. Racer
5439.3

Joseph 5467.67
Alton 3986.65

Jorge 9896.16
Muñoz 8164.82

9197.55
COBOSTAR
9541.39

7705.83
Cardamone
7668.49

7734.76
Autopia
7733.64

0 2000 4000 6000 8000 10000 12000

Without Noise With Noise


What about qualifying?

COBOSTAR is the fastest driver

On complex track noise seems to affect significantly the
performance

However, some controllers are able to reach almost the same
performance even with noise

44

The Three GPs

For each track we run 7 races
with random starting grids

Each race is scored using the F1 point system
(10 to first, 8 to second, 6 to third, …)

Two points to the controller with lesser damage

Two points for the fastest lap of the race


Race: Wild-Speed

Competitor Wild-Speed
Jorge Muñoz 10
COBOSTAR 8
Autopia 10
Cardamone 5
Joseph 2
Tim Alford 4
Mr. Racer 4


Race: Petit

Competitor Wild-Speed Petit
Jorge Muñoz 10 10
COBOSTAR 8 10
Autopia 10 8
Cardamone 5 5
Joseph 2 4
Tim Alford 4 3
Mr. Racer 4 3


Race: Brondehach

Competitor Wild-Speed Petit Brondehach
Jorge Muñoz 10 10 8
COBOSTAR 8 10 10
Autopia 10 8 6
Cardamone 5 5 6
Joseph 2 4 5
Tim Alford 4 3 3
Mr. Racer 4 3 2


Race: Overall

Jorge Muñoz 10 10 8 28
COBOSTAR 8 10 10 28
Autopia 10 8 6 24
Cardamone 5 5 6 16
Joseph 2 4 5 11
Tim Alford 4 3 3 10
Mr. Racer 4 3 2 9


How much does noise affect the
performance in races?


Race without noise

Jorge Muñoz 8 (-2) 8 (-2) 5 (-3) 21 (-7)
COBOSTAR 10 (+2) 10 (=) 12 (+2) 32 (+4)
Autopia 12 (+2) 6 (-2) 8 (+2) 26 (+2)
Cardamone 2 (-3) 5 (=) 6 (=) 13 (-3)
Joseph 3 (+1) 2 (-2) 5 (=) 10 (-1)
Tim Alford 4 (=) 3 (=) 3 (=) 10 (=)
Mr. Racer 5 (+1) 5 (+2) 4 (+2) 14 (+5)


What about the race results?
Race and Qualifying have similar outcomes

COBOSTAR still very competitive

Track learning not so effective…

Opponent manegement on complex track is
less important and very difficult with noise

Championship Standings

Competitor GECCO WCCI Total
Autopia 34 24 58
Jorge Muñoz 22.5 28 50.5
COBOSTAR 14 28 42
Cardamone 16 16 32
Joseph 15.5 11 26.5
Mr. Racer 16 9 25
Tim Alford - 10 10


2010 Simulated Car Racing Championship @ WCCI-2010

Recommended

Recommended

More Related Content

Similar to 2010 Simulated Car Racing Championship @ WCCI-2010

Similar to 2010 Simulated Car Racing Championship @ WCCI-2010 (20)

More from Daniele Loiacono

More from Daniele Loiacono (20)

Recently uploaded

Recently uploaded (20)

2010 Simulated Car Racing Championship @ WCCI-2010