Iscram summerschool12 decisions

Decision Making and Scenario Planning
2012 ISCRAM Summer School on Humanitarian Information Management
Tina Comes

Research Group: Risk Management
Institute for Industrial Production (IIP)

KIT – University of the State of Baden-Wuerttemberg and
National Research Center of the Helmholtz Association www.kit.edu

Risk Management?

Aim: support decision-makers in complex and
uncertain situations
 bridge the gap between formal models and transparent,
ready-to-use evaluations

 collaborative and distributed decision support tools based on modern
ICT systems

Tina Comes Decision Making and Scenario Planning
2
Institute for Industrial Production (IIP) ISCRAM Summer School 2012

Making decisions…

What is the current situation?

How will the future unfold?

Yes

No

3

How to improve the crystal ball?

Each action has consequences
Which of them are relevant?
How do they evolve?
How to compare different consequences?

200 60
people, %, beca
because use …
…

4

Making decisions

1. Identify objectives System disaster
what would you ideally achieve? • environment
2. Describe the system • actors and
their decisions
what are the constitutent elements?
how are they related?
3. Derive relevant consequences from the higher-
level objective Actions Consequences
how to compare consequences? • supply water • number of
and food casualties
4. Find actions to improve • number of
• evacuate
the consequences people evacuated
• ...
what can be done?
5. Compare and analyze
what to do?
 improve actions and iterate
 make decision
5

... but this is difficult in emergencies!
Multiple stakeholders and decision makers
Heterogeneous information on various aspects of the situation
Uncertainty: unforeseen events and reactions
Limited time to make a decision and pressure

Actors possibly geographically dispersed

Bounded availability of experts

Risk of information overload and lack of information

6

Strategic decisions
60 %
1. Multiple goals, diverse actors 200
 how to make trade-offs people
explicit?
 how to build 100
consensus? people

2. Uncertainty and complexity
 what could the consequences of a decision be? 50 %
 what can go wrong?
 why?

3. How to integrate uncertainty into the decision-making?
 what is the best option given limited knowledge?

7

An approach for scenario-based decisions

Collecting information:
a distributed system with heterogeneous experts
Human and artificial  different skills, backgrounds and knowledge

Scenario-Based Multi-Criteria Decision Analysis
Orchestrate distributed scenario generation
Generate relevant, consistent, plausible and coherent scenarios
Use the decision-makers‟ and experts‟ information needs as rationale
for information filtering and sharing
Provide understandable decision analyses and evaluations

8

Challenges

1. Improving the crystal ball: objectives and information needs

2. How to get relevant information?

3. How to combine and process information?

4. How to manage the combinatorics?

5. Supporting decision makers:
how to analyse, interpret and communicate the results?

9

More concretely...

http://www.bbc.co.uk/news/world-asia-pacific-12149921
http://www.theaustralian.com.au/in-depth/queensland-floods

10

Example Situation
Flood currently controlled by levee
Risk: quick flooding if water rises higher

Threat

current uncertain
situation
developments

Time

1. Do nothing?
What to do? 2. Protect buildings,
provide supplies?
3. Evacuation? The Kia Ora Levee
http://www.crikey.com.au/2011/02/28/levees-
and-the-lack-of-regulation-that-could-cost-
millions/

11

What is best decision ?

5 Groups
1. Residents
2. Local industry and infrastructure providers
3. EM staff (fire fighters, health care, police, ...)
4. Political authorities (responsible to make the decision)
5. Moderators

Your aim: Establish a consensus about what to do!
1. Preparation and analysis of options
2. Discussion and consensus building
 one member per team

12

CHALLENGE #1

Improving the crystal ball:
objectives and information needs

13

Determining possible futures

Relevant
consequences
Situation
information

What goes
here?

Ranking of
Alternatives alternatives
for action

14

http://www.theaustralian.com.au/news/nation/queenslands-flood-disaster-a-
long-way-from-over-warns-anna-bligh/story-e6frg6nf-1225979264551

15

What are the relevant consequences?

Discuss in your team:
1. From your perspective, what the relevant consequences?
health and safety, avoid economic losses, efficiency of operations, ...

2. Which of them are the most relevant for you?

3. How can the consequences be measured?
Use indicators that quantify the consequences, such as “duration of
business interruption” for economic losses!

16

How are the consequences related?

Aim: structured evaluation of a decisions consequences
taking into account the decision makers preferences
modelling the problem by an attribute tree

# people evacuated
per day
health
1. do nothing
# people exposed
to flood
2. protection and
supplies
total
performance
firefighters [man-h]
3. evacuation
effort
police [man-h]

17

Back to the example

In your team, structure the problem by an attribute tree

1. do nothing

2. protection and
supplies
total
performance
3. evacuation

18

Determining the consequences?

Decision tables specify the consequences for all alternatives with
respect to each attribute

# people # people firefighters police
evacuated exposed [man-h] [man-h]
per day to flood
1. do
nothing
2. protect
3. evacuate

How to fill in the blanks?
1. collect information
2. manage uncertainty
19

An example from chemical emergency
management

# pp unshelt &

police [manh]
# pp shelt &

firefighters
losses [k€]
alternative

economic

[manh]
exp

exp
E&S1 15 0 0 247,50 123,75
S1 7 0 0 165,00 82,50
DN 0 0 0 0,00 0,00

20

An example from chemical emergency
management – determining the basic
information
What information is required to determine the attributes?

variables indicators variables ATTRIBUTES

affected* (GVP/d,

affected* (GVP/d,
population registry

# pp unshelt & exp

firefighters [manh]
economic losses

# pp shelt & exp
firms indirectly
critical objects

infrastructure*
transportation
infrastructure

police [manh]
firms directly
source term*

population
alternative

presence*
leak size*

chemical
weather*

building
registry
plume

[k€]
k€)

k€)
E&S
NW none Cl_2 none none 750 0 5 0 0,33 5 0,67 15 0 0 247,5 123,8
1

S1 NW none Cl_2 none none 500 0 5 0 0,33 5 0,67 7 0 0 165 82,50

0
DN NW none Cl_2 none none 0 0 5 0 0,33 5 0,67 0 0 0 0

21

CHALLENGE #2

Collecting Information:
Getting Experts to Cooperate

22

How to determine a decision’s consequences?

Monolithic System
Seems like a good idea
Built exactly to system specification
Quick simulation of results
Artificial intelligence techniques are mature
…

However
Vendor lock-in
Specification changes over time as problem changes
Artificial Intelligence techniques are expensive
…

23

An alternative approach

In your team discuss:
1. Which information do you need to determine the best
alternative from your perspective?

2. Who can provide it?

3. How to combine it?

24

Using a Hybrid Heterogeneous Distributed System

Network of experts
Hybrid: both human and artificial experts

Diverse backgrounds, skills and expertise

 breaking down complex problems into manageable sub-problems

Experts cooperate…
… to determine a set of possible futures: scenarios

… via a standardized communication „engine‟

25

Cooperating experts?

What goes here?

26

A distributed problem solving approach

Cooperation structure
Distributed information processing workflow
Workflow setup: combined top-down bottom-up approach
Based on information need („backwards‟): request for information
Based on event („forwards‟): information available  further processing
Matching the experts‟ processing capabilities
Based on profiles per expert
Match based on
information types
(input & output)
expertise
(e.g., location, capabilities)

27

Orchestrated information processing

28

Experts in workflow for the chemical
emergency example

29

Another distributed system

Summer of extreme weather - sbs.com.au/news
http://maps.google.com.au/maps/ms?ie=UTF8&hq=&hnear=Bundarra+New+South+Wales&gl=au&t=h&so
urce=embed&oe=UTF8&msa=0&msid=216305641036137584677.000498fa830661a4cbafb
.
30

Summer of extreme weather - sbs.com.au/news
http://maps.google.com.au/maps/ms?ie=UTF8&hq=&hnear=Bundarra+New+South+Wales&gl=au&t=h&so
urce=embed&oe=UTF8&msa=0&msid=216305641036137584677.000498fa830661a4cbafb
.
31

Local information

http://www.rockhamptonregion.qld.gov.au/Council_Services/New
s_and_Announcements/Latest_News/Evacuation_Centre_open_
8am_Friday_31_December

32

33

Trying it out

Establish a rationale for the negotiations referring to the goals and
objectives you identified!
- where would you enforce evacuation?
- recommend evacuation?
- recommend sheltering?
- other?

Some sources you may find useful
http://www.qldreconstruction.org.au/maps/aerial-imaging-and-mapping-pdfs
http://highload.131940.qld.gov.au/#11
http://maps.google.com.au/maps/ms?ie=UTF8&hq=&hnear=Bundarra+New+South+Wales&
gl=au&t=h&source=embed&oe=UTF8&msa=0&msid=216305641036137584677.000498fa83
0661a4cbafb

34

CHALLENGE #3

Keeping track of the future

35

Why information is not perfect

Uncertainty Ambiguity

Incomplete and uncertain
information in consequences
and evaluation

Constraints in
Time Constraints
resources

36

Robust Decision-Making

Aim: Find the alternative that performs satisfactory in many (all) scenarios.

Score
Score

Satisfactory
threshold

Time Time

Considering one scenario per Considering multiple scenarios per
alternative results in one scoring. alternative results in spread of scoring.

37

Considering several futures…

A £
A’
$

B
B’
E
1.2
C
2.5 C’

25
512 E’

D
D’

38

The flood?

39

Media Coverage
At the scene: Nick Bryant BBC News,
Rockhampton
Almost completely encircled by muddy floodwaters,
Rockhampton risked being entirely cut off if those rose much
further, but they peaked slightly lower than the authorities
had feared, enough to keep the one highway that's open from
being inundated. Many of the city's low-lying suburbs will
remain flooded for more than a week, but a local official said
the city as a whole had "dodged the bullet".

Longer term consequences
Now attention is shifting to the economic http://www.bbc.co.uk/news/world-asia-pacific-12116919

impact of the flooding on Australia's two most vital sectors, mining and agriculture.
Operations at some 40 mines have been interrupted and many of the railway lines that
transport coal to the ports have been severed. Queensland is responsible for more than
half of the country's coal exports. With farms flooded and crops ruined, the price of fresh
fruit and vegetables is also forecast to rise, by as much as 50%.
State Premier Anna Bligh predicted this disaster could have a global impact, partly because
Queensland supplies half of the world's coking coal for steel manufacturing. At least one
senior economist here thinks this could be Australia's most costly natural disaster, largely
because of the impact on exports.
40

Trying it out

Revisit your recommendation and rationale
- is it optimal?
- is it robust?
- which are the most important scenarios you want to use in the
discussions? why?

41

Managing the experts’ work in distributed
reasoning framework

Old situation New situation

What goes here?

Information flow
42

Keeping track of (partial) scenarios

Scenarios capture uncertainty
Requirements
Consistency and comparability
 Not mixing scenario values
Coherence:
 Keeping track of the scenario
construction

43

Consistency in the example
Combination of information Combination of information
about independent variables about related variables

 Changing the workflow mechanisms to
… keep track of partial scenarios
… correctly merge partial scenarios
44

An extract from the chemical emergency
management example

variables indicators variables FOCUS

transportation

police [manh]
infrastructure

infrastructure
source term*

(GVP/d, k€)

(GVP/d, k€)

# pp shelt &

# pp unshelt
population

firefighters
losses [k€]
population
alternative

presence*
leak size*

affected*

economic
indirectly
weather*

affected*
chemical

registry

registry

directly

building
objects
critical

[manh]
plume

& exp
firms

firms

exp
*
E&S1 NW none Cl_2 none none 750 0 5 0 0,33 5 0,67 15 0 0 247,50 123,75
E&S1 NW none Cl_2 none none 750 0 5 0 0,33 5 0,85 18 0 0 247,50 123,75
E&S1 NW med Cl_2 Big Area-big-1 2500 2 20 0 0,25 40 0,67 72,00 925,00 4262,50 437,50 218,75
E&S1 NW med Cl_2 Big Area-big-1 2500 2 20 0,1 0,6 40 0,85 72,00 1375,00 2687,50 1056,00 528,00
E&S1 NW med Cl_2 Big Area-big-1 2500 2 20 0,1 0,6 50 0,85 90,00 1375,00 2687,50 1056,00 528,00
E&S1 NW med Cl_2 Big Area-big-1 2500 2 22 0 0,25 48,00 0,67 86,40 925,00 4262,50 437,50 218,75
E&S1 NW med Cl_2 Big Area-big-1 2500 2 22 0,1 0,6 48,00 0,85 86,40 1375,00 2687,50 1056,00 528,00
E&S1 NW med Cl_2 Big Area-big-1 2500 2 22 0,1 0,6 60,00 0,85 108,00 1375,00 2687,50 1056,00 528,00
E&S1 NW large Cl_2 Big Area-big-2 2000 3 30 0 0,25 50 0,67 90,00 590,00 3935,00 312,50 156,25
E&S1 NW large Cl_2 Big Area-big-2 2000 3 30 0,1 0,6 50 0,85 90,00 950,00 2675,00 756,00 378,00
E&S1 NW large Cl_2 Big Area-big-2 2000 3 30 0,1 0,6 80 0,85 144,00 950,00 2675,00 756,00 378,00

... and this is just a small extract...
45

CHALLENGE #4

Handling combinatorics

46

Too many possible futures…

Given
Limited time, effort, available expertise
Need for a decision

Aim: exploring the space of possible developments

Combinatorics…
Too many scenarios!
What to do?

47

Scenario Management

During the construction
Selection of the most relevant partial
scenarios
Pruning of invalid scenarios
Update to take into account relevant new
information

Evaluation:
Partial scenario Selection of the most relevant scenarios
Selected partial Aggregation of results
scenario
Updated partial
scenario

48

Which scenarios are the most relevant?

Most scenario similarity measures:
distance of the variables‟ values

Our aim: Explore the space of evaluations
 Making risks and chances transparent
 Robustness

Definition of Scenario classes
 Based on the similarity of the evaluation
 Selection of a representative per class

49

Impact on exploration of scenario space exploiting
the network structures
1

0.9

UPDATED
0.8

0.7 ORIG
Evaluation

0.6 SEL
0.5

0.4

0.3

0.2

0.1

0

Scenario

50

Scenario Updates: Efficiency

400
Upper Bound of Duration [min]

350 Duration of update from
indicator variables to FOCUS
300
250 Duration of update to indicator
variables
200
150
100
50
0
Complete update Partial update all Partial update of
scenarios selected
Approach to update

51

How a distributed system can work in chemical
emergencies

Video available on:
http://www.pdc.dk/diadem/Video/DiademVideo.wmv

52

CHALLENGE #5

Supporting decision makers

53

How to develop good alternatives?

MCDA: workshops serve
Define the
- for the identification of Recommendation
Problem
decision criteria and feasible
countermeasures Sensitivity
Analysis

n
Con
Identify the

ctio
- as exercises Attributes

clus

odu
ing
her

ion

Intr
Pla
- for the identification of Mea nning
su
Gat ics
top
be t res to
responsibilities and authorities Choose an
ake
n Se
le
to implement a rapid response Alternative c
to tin
pi g
Specify
Performance

top g
the ndlin
c a

ic
Measures

Ha
How to support decision
makers in building better Weight Criteria
Identify the
alternatives and establish Analyse the
Alternatives

consensus in very Alternatives

uncertain situations?

54

How to handle trade-offs?

Preference models represent the preferences and value judgements of a
decision maker by
1. A model that scores each alternative against each individual attribute
 concerns all attributes
2. A model that compares the relative importance among the criteria to
obtain a ranking of alternatives
a. Elicitation of the relative importance (weights) of the criteria
b. Aggregation
 concerns the complete attribute tree

55

Back to the example attribute trees

How to compare the attributes?

1. do nothing

2. protection and
supplies
total
performance
3. evacuation

56

Some technical details: Value functions allow to score
each alternative against each individual attribute

Scores si(a) of the alternatives are measured in different units for the
different attributes
to make comparisons, map these scores to a scale ranging from 0 to 1
(where the “worst” and “best” possible outcomes correspond to 0 and 1
respectively) by defining value functions
si a : score of alternative a relative to attribute i
vi vi si a : value of the score of alternative a relative to attribute i

si a min si a # people protected
a
, if max si a highest value
max si a min si a a
a a

vi

max si a si a
a
, if max si a lowest value
max si a min si a a
a a

work effort (# workers)
57

Weights – Inter-criteria preferences

Different weighting procedures
The simplest way is the DIRECT weighting
In the SWING procedure, 100 points are first given to the most
important attribute; then, less points are given to the other attributes
depending on the relative importance of their ranges
The SMART method is similar, but the procedure starts from the least
important attribute (assigning 10 points to it) keeping it as the reference
In SMARTER, the weights are elicited directly from the ranking of the
alternatives
In AHP, the weights are determined by pairwise comparisons

58

Trying it out...

Go back to the attribute tree and the rationales you have developed.
- which are the most important criteria for you?
- can you establish clear preferences within your group (for weights and
value functions)?

59

Scenario selection: Exemplary results
Selected sources of uncertainty:
success of chlorine transfer
residual amount of chlorine in tank
weather Evaluation of Scenarios
1
Health
Effort
0.9
Society

0.8 results for best
and worst
Evaluation R(s)

0.7

0.6 scenarios
Evaluation R(s)

0.5

0.4

0.3

0.2

0.1

0
E S N E S N E S N E S N E S N E S N E S N E S N E S N E S N
Scenarios for Alternatives Evacuation (E), Sheltering (S) and Do nothing (N)
Scenarios for Alternatives Evacuation (E), Sheltering (S) and Do Nothing (N)

60

Aggregation of results:
how important is each scenario?

Definition of weights – but how?
direct elicitation from the decision-makers

According to the Evaluation
 Goal Attainment
 Trying to satisfice overall or partial goals (Simon, 1979)
 Deviation from equal weighting if these goals are not attained:
penalty functions
 According to risk aversion
 Risk aversion: relative importance of scenarios evaluated
worst/best (Yager, 2008)
 Determination of weights according to the scenarios„ ranking
61

Example: Results for varying levels of risk aversion

1 1
Evacuation
0.9 Sheltering
0.9
Do Nothing
0.8
Aggregated weights

0.8
0.7 aggregated weight of
worst evaluated scenarios
0.6 aggregated weight of
Result(alternative)

0.7 best evaluated scenarios
0.5

0.4
0.6

0.3

0.5 0.2

0.1
0.4
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.3
Risk level

0.2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Risk level

62

Interpreting the results: scenario reliability

Number of scenarios increases with growing uncertainty
 risk of overemphasizing some scenarios‟ results for structural reasons

Scenario Reliability
Modelling the relative uncertainty of scenarios:
uncertainty of the situation: comparison to other scenarios
uncertainty of the specific scenario
preferences of the decision makers
 easily manageable measure
 enables decision-makers to adapt scenario weights and overcome
cognitive biases

63

How to make alternatives better

1. How is the quality of an alternative measured? MCDA!
2. What can go well and what can go wrong? SBR!
An iterative approach
1. Identification of key weaknesses per alternative
2. Identification of better alternatives to address
these weaknesses
Analysis: how can these alternatives be combined?

So, all information is there. But...
... large numbers of scenarios and results
... visualisations not easy to interpret
 need for a clear and transparent explanation of results

64

Making sense of what you see

65

Communicating decisions under uncertainty

Evaluation of Scenarios
1
Health
Effort
0.9
Society

0.8

0.7

0.6

Evaluation R(s)
0.5

0.4

0.3

0.2

0.1

0
E S N E S N E S N E S N E S N E S N E S N E S N E S N E S N
Scenarios for Alternatives Evacuation (E), Sheltering (S) and Do nothing (N)

66

Generation of natural language reports

1. Content determination
Information about what? Type of report and
 variables: alternatives, outcomes, drivers, ... information
Questions that should be addressed? requirements
 relations: causes and effects, better or worse, ...
2. Discourse planning

3. Sentence generation

67


• variables Type of report and
• relations information
requirements

2. Discourse Planning
What can be said about the entities and their relations?
 determine types of individual messages Argumentation
How to combine the messages into an argumentation?
 relate and cluster messages into a tree structure


68


• variables Type of report and
• relations information
requirements

2. Discourse Planning
• types of individual messages
Argumentation
• tree structure
structure

How to express the message?
 choose of adequate text patterns Template System
What is the argument for this case?
completion of statements

69

From numbers to verbal expressions:
Semantic quantifiers

Aim: describe the quality of a decision
“substantially better”, “slightly worse”, ...
Alternative <name of alternative> performs <semantic quantifier> on
<objective> in the context of all available scenarios.
A relative approach
1. set of evaluated scenarios and relevant objectives
2. determine mean μ and standard deviation
3. set SQs

 Alternative evacuation performs very poor on effort in the context of all
available scenarios.
A benchmark approach: goal programming and satisfaction levels
Alternative evacuation has an acceptable performance with respect to
health in most scenarios.
70

Key weaknesses

1. What do the worst scenarios for an alternative have in common?
statistical approach: worst % for each alternative
benchmark approach: scenarios that violate threshold
 identify variables var1, ..., varn and their values
Alternative <name of alternative> performs <semantic quantifier> on
<objective> for all scenarios that assume <value of var1> for <var1>,...,
<value of varn> for <varn>.
2. How do other alternatives perform for the same / similar scenarios?
3. Identify better alternatives and describe significance in an SQ
Alternative <name of alternative2> performs <semantic quantifier> on
<objective> than <name of alternative> for the identified scenarios.

71

Finally...

Prepare for the
discussion, collect
the material you
need and choose the
representative...

... and then, find a
solution:
which strategic
measures should
be implemented
and where?

72

REFLECTIONS AND CONCLUSIONS

73

Conclusion

Integrated Scenario-Based MCDA
Distributed processing of relevant information
Consideration of interdependencies
Formalization using set and graph theory
Ensuring comparability
Scenario management: updating, selection, pruning
Respecting constraints and requirements in emergency management

Decentralised vs. centralised: Orchestrating emergence
Decentralised experts involved in workflow
Decision-centric management with overview

74

Reflections

1. What were the main challenges
in your team?

in the discussion?

2. Social media applications?

75

Thank you!
Contact
Tina Comes
comes@kit.edu

Questions?

76

References

Comes, T., Wijngaards, N. & Schultmann, F. (2012): Efficient Scenarios Updating in
Emergency Management. 9th International Conference on Information Systems for Crisis
Response and Management
Comes, T., Wijngaards, N., Maule, J., Allen, D. & Schultmann, F. (2012): Scenario
Reliability Assessment to Support Decision Makers in Situations of Severe Uncertainty.
2012 IEEE Conference on Cognitive Methods in Situation Awareness and Decision
Support
Comes, T., Hiete, M., Wijngaards, N. & Schultmann, F. (2011): Decision Maps: A
framework for multi-criteria decision support under severe uncertainty. Decision Support
Systems, 52(1), 108-118.
Comes, T., Conrado, C., Hiete, M., Wijngaards, N. & Schultmann, F. (2011): A distributed
scenario-based decision support system for robust decision-making in complex
situations. International Journal of Information Systems for Crisis Response and
Management, 3(4), 16-35.
Simon, H. (1979): Rational Decision Making in Business Organizations, The American
Economic Review, 69(4), 493-513.
Ronald R. Yager, “Using trapezoids for representing granular objects: Applications to
learning and OWA aggregation,” Information Sciences 178(2), 363-380.

77

Iscram summerschool12 decisions

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Iscram summerschool12 decisions

Similar to Iscram summerschool12 decisions (20)

Recently uploaded

Recently uploaded (20)

Iscram summerschool12 decisions