Colleen Roller's presentation from the Boston UPA mini conference 2011.

1. The Mind of the User
When 98% is more than 100%:
How number format affects judgment & decisions
Colleen Roller
UX Matters Columnist
On Decision Architecture

2. Do you know…
 How well do people understand and interpret
numeric data?
 Does the format of numeric data have an impact?
 How should we display numeric data?

3. This presentation
 Reviews research that reveals how
people perceive and use data
 Suggests UX design principles and
best practices

4. A study
 Rate the attractiveness of a simple gamble
(on a scale from 0-20)
 1st group (rating = 9.4)
 7 out of 36 chance to win $9
 2nd group (rating = 14.9)
 7 out of 36 chance to win $9
 29 out of 36 chance to lose 5 cents
Bateman/Dent/Peters/Slovic/Starmer -
2007

5. Judgment & decision making
 Must be able to attach meaning to data
 Meaning is determined via
 Context – a reference point or upper/
lower bound
 Comparison – against the context
The ease with which meaning can be derived
dictates the extent to which data will be used.

6. A study
 Which bowl did people prefer:
A. 100 beans, 7 of which are red
B. 10 beans, 1 of which is red
Denes-Raj/Epstein - 1994

7. A study
 Which disease is more dangerous?
A. Kills 1286 out of every 10,000 people
B. Kills 24 out of every 100 people
Yamagishi 1997

8. A study
 Will a mental patient commit a violent
act within 6 months of being
discharged from the hospital?
A. 20 out of every 100 patients commit a
violent act (41% refused to discharge)
B. 20% chance that the patient will commit
a violent act (21% refused to discharge)
Slovic/Monahan/MacGregor - 2000

9. Quiz
 Which is more easily understood:
A. A 30% chance of rain
B. A 3 in 10 chance of rain

10. 30% chance of rain
 Common misinterpretations:
 It will rain in 30% of the area
 It will rain 30% of the time
 It will rain on 30% on days like this
Gigerenzer/Edwards - 2003

11. Problems with probability
 What the doctor said:
 You have a 30% - 50% chance of
developing a sexual problem
 What the patient heard:
 In 30% - 50% of your sexual encounters,
something will go awry
Gigerenzer/Edwards - 2003

12. Summary
 People tend to
 Comprehend frequencies better than
probability/percent
 20 out of 100, rather than 20%
 Focus on the numerator (and ignore the
denominator)
 9 out of 100 is bigger than 1 out of 10
 Different expressions of equivalent data -
e.g., 30 out of 1000 is more than 3 out of 10

13. A study
 Purchase equipment for use in the
event of an airline crash landing:
A. Chance of saving 150 lives
B. Chance of saving 98% of 150 lives
Slovic et al. - 2002

14. 85% is more than 100%
Even 85% of 150 is more than “150”!
Slovic et al. - 2002

15. Calculations
Probability Frequency
 1% of car trips result in an  100 out of 10,000 car trips
accident. In 55% of the result in an accident.
trips that result in an Among the 100 trips that
accident, the driver is result in an accident, the
drunk. In 5% of the car driver is drunk in 55 of
trips that do not result in them. Among the 9900 car
an accident, the driver is trips that don’t result in an
drunk. If the driver is accident, the driver is drunk
drunk, what is the in 500 of them. How many
probability of an accident? car trips where the driver is
drunk result in an accident?

16. Study: Rates of return
 Allocate money across two investment
funds – stocks v. bonds
 Group A: shown 1-yr rates of return 63%
 Group B: shown 30-yr rates of return 81%
Benartzi/Thaler, 2001

17. Data on the page
61% Versus
75%
Hibbard/Slovic/Peters/Finucane - 2002

18. No neutral design
 What information – and how it is
presented – drives decision outcomes

19. Numeric ability
 Almost half of the general population
has difficulty with simple numeric tasks
National Adult Literacy Survey

20. Numeracy
 Those who are numeric
 Readily understand and use numeric
data effectively
 Those who are non-numeric
 Informed less by numbers, and more by
other non-numeric sources of info

21. Number format
Best Practices

22. Determine the right criteria
 Determine what decision criteria
people should be using
 Highlight them (salience)
 Make it easy to evaluate, compute, &
attach meaning

23. Frequency v. probability
 Convey absolute risks over relative
risks
 3 out of 1000 will have a stroke
is better than
 50% higher chance of stroke
 Don’t use decimals (.03)

24. Apples to apples comparisons
 When presenting various probabilities,
keep the denominator consistent
 20 out of 1000 compared to 1 out of 1000
is better than
 1 out of 50 compared to 1 out of 1000

25. Attach meaning
 Use labels to show standard of
performance
 Example: unacceptable, acceptable,
excellent
 Labels provide expert guidance and easy
mental processing

26. Mapping
 A higher number means better quality
 Reduces cognitive load
 Subtle, but influential

27. Quiz
 Which one results in better
comprehension and better choices:
A. Number of patients per registered nurse
B. Number of registered nurses per 100
patients

28. Variety of visual display

29. Consecutive v. random
Schapira/Nattinger/McHorney - 2001

30. Placement of solid dots

31. Conveying small risks
 To help people understand the
meaning of small risks
 Show context by providing a range of
probabilities and risks for comparison
 Example: being hit by a car v. x-rays v.
lighting v. asbestos

32. Emotion v. probability
 When consequences are marked by
strong emotion
 All or none – sensitive to the possibility
rather than the probability

33. Final thoughts
 Determine what info is most important
 What should people base the decision on?
 Design for meaning and ease
 Via context and comparison
 Facilitate easy computation
 Test – multiple methods
 Test drive, A/B testing, website metrics,
think aloud, observe/probe in usability testing

34. Questions/discussion
 Colleen Roller
UX Matters Columnist
on Decision Architecture
http://uxmatters.com/

35. Reference articles
 Simple Tools for Understanding Risks: From Innumeracy to Insight – G.
Gigerenzer & A. Edwards, 2003
 Numeracy and Decision Making – E. Peters et al., 2006
 Numeracy and the Perception and Communication of Risk – E. Peters, 2008
 Strategies for Reporting Health Plan Performance Information to Consumers:
Evidence from Controlled Studies – J. Hibbard, et al., 2002
 Risk as Analysis and Risk as Feelings: Some Thoughts about Affect, Reason,
Risk, and Rationality – P. Slovic et al., 2004
 Numeracy Skill and the Communication, Comprehension, and Use of Risk-
Benefit Information – E. Peters et al., 2007
 Reducing the Influence of Anecdotal Reasoning on People’s Health Care
Decisions: Is a Picture Worth a Thousand Statistics? - A. Fagerlin et al., 2005
 Bringing Meaning to Numbers: The Impact of Evaluative Categories on
Decisions – E. Peters et al., 2009
 When a 12.86% Mortality is More Dangerous than 24.14%: Implications for
Risk Communication – K Yamagishi, 1997