Lecture given at the INdAM symposium in Rome, 2017. The lecture shows how you can use differential games to model traffic flows, focussing on pedestrian simulation.

1. Modelling (Active) Traffic
An Introduction to Applications of Optimal Control and Differential Game Theory
Prof. dr. Serge Hoogendoorn
1

2. 2
Pedestrian modeling
questions in practise…
• Redesign of the Al Mataf Mosque
(testing for safety and throughput)
• Station design (e.g. Amsterdam Zuid)
to show if LOS remains acceptable
• Investigating building evacuations
(e.g. impact of counterflow WTC 9/11)
• Testing crowd management plans
during SAIL tallship event in
Amsterdam
For each of these (and many others)
applications, the key question is: “Can
we provide predictively valid
models that can predict LOS,
throughput, and safety pinch points in
case of regular or irregular conditions”

3. “A model is as good as the predictions it provides”
• Questionable if from this engineering
perspective, ped models are predictively valid
(in contrast to models for car traffic)
• Why? In our field, DATA is key in the
development of theory and models
• Pedestrian theory (and cyclists) has suffered
from lack of data and has been “assumption
rich and data poor”
• Some examples of different data collection
exercises that we have performed from say
2000 onward
3
Understanding transport
begins and ends with data

4. Let’s start with the pedestrians…

5. Speed density relation pedestrian flow
• Fundamental diagram for pedestrian
flow stemming from 2002 experiments
• Non-increasing relation between
speed and density showing reduction
in speed as density increases (or
increase in density as speed
reduces?)
• Strong impact of flow composition
(e.g. purpose, age, gender, etc.)
• Can you interpret the relation? That
is, can you explain the shape?
5

6. 6
Example shared-space region
Amsterdam Central Station
Empirical facts of self-organisation (2002 experiment)
• Bi-directional flow experiment revealed self-organisation of lanes
• Process has chaotic features (e.g. equilibrium states formed depend critically on initial and
boundary conditions, ill-predictable)
• Collected microscopic data allowed for unique quantification self-organised features

7. 7
Example shared-space region
Amsterdam Central Station
Empirical facts of self-organisation (2002 experiment)
• Bi-directional flow experiment revealed self-organisation of lanes
• Process has chaotic features (e.g. equilibrium states formed depend critically on initial and
boundary conditions, ill-predictable)
• Collected microscopic data allowed for unique quantification self-organised features
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Density = 0.25 to 0.5 Ped/m2
2
0.2
0.4
0.6
Relativefrequency
Density = 0.75 to 1 Ped/m2
0 0.2 0.4 0.6 0.8
0.9
1
1.1
1.2
1.3
density (Ped/m2
)
speed(m/s)
1
2
3
4
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Density = 0 to 0.25 Ped/m2
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Density = 0.25 to 0.5 Ped/m2
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Density = 0.5 to 0.75 Ped/m2
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Density = 0.75 to 1 Ped/m2
• Examples show how likelihood of certain number of lanes formed depends on density
• Right picture shows how speed-density relation is (to a limited extent) dependent on
the number of lanes that is formed

8. Further experimentation supports and extends our findings
• Example shows results by Prof. Seyfried (downloadable via the Juelich Centre website)
http://www.fz-juelich.de/ias/jsc/EN/Research/ModellingSimulation/
CivilSecurityTraffic/PedestrianDynamics/Activities/database/

9. 9
Example shared-space region
Amsterdam Central Station
Traffic Flow
Phenomena
• Other self-
organised patterns
found for other
experiments
• Example shows
formation of
diagonal stripes
• Other examples
include zipper
effect, viscous
fingering, and faster
is slower effect
• Self-organisation
also occurs in
practise…

10. 10
Example shared-space region
Amsterdam Central StationSelf-organisation during LowLands…

11. Modelling challenge…
• To come up with a model (and underlying theory) that can predict observed
relations (e.g. speed-density) and phenomena for different situations
• In 2003, we opted for differential game theory to model the behaviour of
pedestrians that are competing for the use of (scarce) space
• Some motivation?
- We know that under specific conditions, differential game theory predicts
occurrence of (meta-) stable equilibrium state, which (hopefully) are our self-
organised patterns…
- Moreover, research from the late Seventies and Eighties provides us with evidence
that could be used as a basis for a game-theoretical model…
• Let us have a brief look at the different behavioural assumptions (the theory) that
underly the game theoretical model…
11

12. Microscopic pedestrian modelling
• Main assumption “pedestrian economicus”
based on principle of least effort:
For all available options (accelerating, changing
direction, do nothing) he chooses option yielding
smallest predicted disutility (predicted effort)
• When predicting walking effort, he values and
combines predicted attributes characterising
available options (risk to collide, walking too
slow, straying from intended walking path, etc.)
12

13. Six additional behavioural assumptions…
• Pedestrians are feedback-oriented,
reconsidering their decisions
based on current situation
• They anticipate behaviour of others
by predicting their walking behaviour
according to non-co-operative or
co-operative strategies
• Their predicting abilities are limited,
reflected by discounting effort of
their actions over time and space
• Pedestrians are anisotropic in that
react mainly to stimuli in front of
them
• They minimise predicted discounted
costs resulting from: (a) straying from
planned path; (b) vicinity of other
peds (+ obstacles); (c) applying
control
• Pedestrians are more evasive
encountering a group than a single
pedestrian

14. The Math…
• State equation describes ‘mental model’ peds to predict from
where the state describes the positions and velocities of ped p
and his opponents q (e.g. ), and were the control is the acceleration
that a pedestrian can apply
• Prediction model describes kinematics of pedestrians, e.g. and
• Ped p chooses control (accel.) minimising effort for
˙x(t) = f(t, x, u) x(tk) = xksubject to
r(t) v(t)x(t)
rp(t) u(t)
˙r = v ˙v = u
[tk, tk + T)
Jp =
Z tk+T
tk
e ⌘s
Lp(s, x(s), u(s))ds + e ⌘(tk+T )
p(tk + T, x(tk + T))
t = tk
u[tk,tk+T )

15. Using the assumptions to specify model
• Most behavioural assumptions are specified via running cost
where:
and with the proximity cost equal to:
Lp(t, x, u)
Lp = Lstray
p + Lprox
p + Laccel
p
Lstray
p =
1
2
(v0
p vp)2
Laccel
p =
1
2
u2
p
Lprox
p =
X
q2Q
e dpq/RP
✓
p + (1 p)
1 + cos ✓pq
2
◆
and

16. Using the assumptions to specify model
• Most behavioural assumptions are specified via running cost
where:
and with the proximity cost equal to:
Lp(t, x, u)
Lp = Lstray
p + Lprox
p + Laccel
p
Lprox
p =
X
q2Q
e dpq/RP
✓
p + (1 p)
1 + cos ✓pq
2
◆
andLstray
p =
1
2
(v0
p vp)2
Straying cost describe the impact of
not walking in the desired direction
and at the desired speed
Laccel
p =
1
2
u2
p
Acceleration cost describe the cost of
applying the control acceleration in
long. and lat. direction

17. Lstray
p =
1
2
(v0
p vp)2
Laccel
p =
1
2
u2
p
Using the assumptions to specify model
• Most behavioural assumptions are specified via running cost
where:
and with the proximity cost equal to:
Lp(t, x, u)
Lp = Lstray
p + Lprox
p + Laccel
p
and
Proximity cost shows spatial discounting of
cost impact using distance
Lprox
p =
X
q2Q
e dpq/RP
✓
p + (1 p)
1 + cos ✓pq
2
◆
Impact of ‘groups’ by
adding proximity
costs over opponents
Anisotropy is reflected
by making cost
dependent on angle ✓pq
p
q
✓pq
rq rp
vp
dpq = ||rq rp||

18. Solving the problem?
• Problem can be solved using the Minimum Principle of Pontryagin
• Without going into details…
• Define Hamiltonian function:
and use it for necessary conditions for optimality of control signal
• Next to the state equation + initial conditions, we can derive an equation for
the co-states (a.k.a. marginal costs) + terminal condition
and optimality conditions to determine optimal acceleration
18
Hp = e ⌘t
Lp + 0
p · f
u⇤
[tk,tk+T )
˙ p = @Hp/@xp p(tk + T) = @ p/@xand
u⇤
p = arg min H(t, x, u, p)

19. Finding solutions?
• Assume non-cooperative behaviour:
pedestrian p optimises own cost
function while begin aware that the
opponents will do the same
• Novel iterative approach for mixed
initial-terminal boundary condition
problem can be applied to small test
cases due to computational burden
• Example shows ‘self-organisation’
crossing flow case: intersection
without signals?

20. Can we come up with a simpler model?
• The trick we used: simplify prediction model of pedestrian p
• Pedestrian p assumes that opponents do not change their speed or direction
(i.e. ) during the prediction period with
• Time-invariant infinite horizon discounted cost problem has closed-form
solution which is similar to the social-forces model of Helbing:
[tk, tk + T)uq = 0 T ! 1
up =
v0
p vp
⌧p
Ap
X
q6=p
e dpq/Rp
✓
p + (1 p)
1 + cos ✓pq
2
◆
npq
p
q
✓pq
vp
npq
Acceleration towards desired velocity
Push away from ped q
+ …

21. Speed-density relation?
• Assume pedestrians walking in straight line
• Equilibrium: no acceleration, equal
distances R between pedestrians
• We can easily determine equilibrium speed
for pedestrian q (q > p means q is in front)
• Speed-density diagram looks reasonable
for positive values of anisotropy factor
21
0 2 4 6
density (P/m)
0
0.5
1
1.5
speed(m/s)
1
speed(m/s)
= 0
= 1
= 0.6
ve
p = v0
p ⌧pAp
X
q>p
e (q p)d/Rp
X
q<p
e (p q)d/R
!

22. Characteristics of the simplified model
• Simple model captures macroscopic characteristics of flows well
• Also self-organised phenomena are captured, including dynamic lane formation, formation of diagonal stripes, viscous fingering, etc.
• Does model capture ‘faster is slower effect’?
• If it does not, what would be needed to include it?
Application of differential game theory:
• Pedestrians minimise predicted walking cost, due
to straying from intended path, being too close to
others / obstacles and effort, yielding:
• Simplified model is similar to Social Forces model of Helbing
Face validity?
• Model results in reasonable macroscopic flow characteristics
• What about self-organisation? 22
Characteristics of the
simplified model
• Simple model captures some key
relations (e.g. speed-density
curve) reasonable well!
• Also self-organised phenomena
are captured, including dynamic
lane formation, formation of
diagonal stripes, viscous fingering.
• Self-organisation depends
critically on paramaters and
variance: freezing by heating
• Do you know other features we
have not discussed? Would the
model be able to reproduce these?
• Self-organisation of bi-directional lanes
fails if demand becomes too high
• Parameters determine threshold

23. 23
• Faster = slower effect states that the faster people try to get out, the longer an evacuation will take
• Different experiments showed substantial reduction in outflow when evacuees rush
• Capacity reduction is caused by friction / arc-formation in front of door due to increased pressure

24. Introducing the Faster is Slower effect
• What could you do to reproduce the capacity drop
phenomenon / faster = slower effect?

25. Introducing the Faster is Slower effect
• What could you do to reproduce the capacity drop
phenomenon / faster = slower effect?
• Analogy with squash balls:
• Pedestrians are ‘compressible circles’
• Normal force pushing away the other pedestrian
• Tangential friction force increasing with reduction
of distance of pedestrian centers
• Capacity decreases / evacuation time increases with
increasing pressure / haste
friction
normal force

26. Calibration and validation• Capacity could be reproduced with 4% error
26
empirical
simulation
1 2 3 4
0
0.2
0.4
0.6
Number of lanes formed
Relativefrequency
Some lessons…
• Importance of sensitivity
analysis to see which
parameters are most
relevant
• Different average
parameter values are
found when calibrating
per situation (bi-dir flow,
cross flow, bottleneck)
• Reasonable average
parameter set was found
for all situations
• Improvement when
including explicit delay
• Large differences in
microscopic valuesCalibration and validation

27. Calibration and validation
• Novel estimation technique (based on ML) allowed using the microscopic data
from experiments to estimate parameter distribution and correlations
• Table shows results of estimation procedure for illustration purposes: note that
for some parameters, variance is substantial as is the correlation!

28. 28
Example shared-space region
Amsterdam Central Station
Fascination with active-mode traffic modelling
Amsterdam cycling during rush hour

29. Versatility of the game-theory approach?
• Modelling cyclist behaviour
• Approach is very similar to pedestrian
modelling, however the kinematics of
cyclists is different:
• First results of application to cycle flow
• Example shows application to mixed
pedestrian / cyclist flow (shared space)
29
˙xp = vp cos ↵p ˙yp = vp sin ↵p
˙vp = ap ˙↵p = !p

30. Successful shared-space implementation
30
Example shared-space region
Amsterdam Central Station
Shared-space:
interaction
between bikes
and peds…
• Area behind
Amsterdam Central
Station
• Mix between
pedestrians going
to / coming from
station to ferries and
crossing pedestrians
• Self-organisation
yields reasonable
flow operations
• Modelling using
game-theory

31. Successful shared-space implementation
31
Example simulation results for shared-space situation
Shared-space:
interaction
between bikes
and peds…
• Area behind
Amsterdam Central
Station
• Mix between
pedestrians going to /
coming from station
to ferries and crossing
pedestrians
• Modelling using
game-theory
• Depending on
parameter choices,
self-organisation
occufs…

32. Use of game theory for C-ACC systems
• Use optimal control approach to
minimise joint cost of vehicles in
platoon by controlling few vehicles
• Cost function can reflect efficiency,
comfort, emissions:
Follower 2 -
Human-driven
vehicle
Follower 1-
Cooperative vehicle
Leader –
Human-driven
vehicle
s1, Δv1s2, Δv2
J = J(u[t,T ) ) = c1Jsafety +c2Jefficiency +c3Jfuel +...

33. Use of game theory for C-ACC systems
• Differences no control, ACC and C-ACC

34. Autonomous cars? What about bikes?

35. Applications of game theory to modelling traffic flows
Contributions of talk
• Importance of data & examples of typical features of traffic flow
• Motivate use of differential game theory
• Apply game theory to pedestrian modelling
• Show model features and discuss possible further improvements
• Calibration and validation issues
• Show other applications (cooperative vehicles, bicycle flow and shared space)
Work is based on research contributions from 2002 onward with Winnie
Daamen, Mario Campanella, Dorine Duives, Meng Wang, and others!

36. 9th Workshop on the
Mathematical Foundations of
Traffic @ TU Delft
Co-organised by the
TU Delft Transport Institute
&
TU Delft Institute for Computational Science and Engineering
Dates soon to follow!
36