Slides from session given by Peter Mallow and Della Rucker on methods for evaluating economic impacts of policy choices at Ohio American Planning Association Conference, October 2011.

1. A more useful tool: Using
economic evaluation to make
better decisions
APA Ohio State Planning Conference
October 20 & 21, 2011
Dayton, Ohio

2. Agenda
• Introductions
• Definitions and goals of economic evaluations
• Economic evaluation vs. economic impact
• How to do an economic evaluation
• Discussion

3. What are the Goals of Economic Evaluations?
• Because all resources are scarce, economic
evaluations help us…..
– Identify which initiatives will offer the greatest
value
• “Value” means providing the most benefit per
unit of cost
• This is the so-called “efficiency criterion.”
Guarantees that the greatest aggregate benefit
will be provided for that cost.
– But efficiency has little to say about equity;
that is, about how the total benefits get
distributed
3

4. Why do we need to do economic evaluations of our
planning and policy choices?
• We know different choices have different
costs and impacts, but we don’t know what
those are automatically.
– As a result, we can either figure it out or
“guesstimate”
– Too often, we guesstimate – and get it
wrong.
• We can use economic evaluation methods as
a tool kit to make our real-world decision
making better.
4

5. How does this differ from an economic impact study?
• Economic IMPACT: essentially a
political tool
– Problem of counting indirect
impacts: who’s right?
– Not always balanced against cost
– Most useful for persuasion
– Losing effectiveness due to
overuse?
5

6. Therefore….
• Use an economic impact
study if are trying to
persuade people to do
something.
• Use an economic
evaluation if you are
trying to find answers
and make decisions.
6

7. Types of economic
evaluation

8. The Groundwork for Economic Evaluations
• Burden of Cost
– What impact will changing a policy have on people?
– Cost may be economic, social, or psychological cost to
individual, organization, or community.
– Example: changing how a city service is delivered.
– Who will be directly impacted?
– Who will be impacted as a result of the first set of
impacts?
– What will it require for the impacted people/orgs to make
up the difference, or avoid a loss of quality?
8

9. What Makes a Decision Hard?
• Urgency • Uncertainty
– How soon will it happen? – Do we know what’s really
going on?
– How soon do we have to
deal with it? – Do we know what the best
approach will be?
– Do we know if this has
• Conflicting objectives worked in the past?
– Cars vs. pedestrians? – What happens if we get it
– Commercial or residential? wrong?
– Quality of service vs. range
of access?
9

10. Cost-Effectiveness Analysis
• Compares the relative costs and outcomes (effects) of two or
more courses of action
– Costs are measured in monetary units and outcomes are
measured in effectiveness, such as deaths averted , park use
increased or VMT improved.
– Typically the CEA is expressed in terms of a ratio where the
denominator (bottom) is the impact from the potential action,
and the numerator (top) is the cost.
_COST_
OUTCOME
COST:OUTCOME
10

11. Cost-Effectiveness Analysis Example
Building a new park
Cost: expenditure required to build
the park according to design.
Benefit factors: number of people
expected to use the park, impact on
health of users (health costs, weight
loss, health care costs).
You’ll note that this is the tricky
part….can be done, but make
assumptions clear.
11

12. Cost-Benefit Analysis
• Cost-benefit analysis (CBA) differs
from CEA and CUA in that benefits
are measured in monetary units, just
like costs
• This feature allows one to compare
widely different types of
interventions using CBA, since all
benefits and costs are expressed in
monetary units.
• But with public policy issues,
converting outcomes to cash isn’t
always a good idea.
12

13. Conceptual Underpinnings of CEA
• Unlimited wants
• Finite resources
• Inevitability of choices
• Balance of benefits and costs
• Need for formal analysis
13

14. Solving the Resource Allocation Problem
• The approach: constrained optimization
• Two ways of framing the problem:
What allocation What allocation of
of resources will resources will
achieve the minimize our
greatest possible expenditures,
gains without subject to the
exceeding the requirement that
volume of we achieve at least
available some target level of
resources? benefit?
14

15. Finding the Cost-Effective Solution
• Identify mutually-exclusive interventions
– Eliminate strongly and weakly dominated interventions
More costly, less effective than some competing alternative
– Select the most effective program with a CE ratio less than or
equal to some threshold value
threshold value = decision maker’s willingness to pay for a unit of
benefit
15

16. Incremental Cost-Effectiveness Ratios
Incremental
Cost
+ -
Calculate
+ incremental
CE ratio
Dominant
Effectiveness
Incremental
Calculate
- Dominated incremental
CE ratio
16

17. Incremental CE Ratios
• The incremental CE ratio comparing P2 to P1:
Cost of P2 – Cost of P1
Effect of P2 –Effect of P1
This tells you how much more you would have to spend for each additional
unit of benefit you would gain if you choose a more costly, more effective
program rather than a less costly, less effective program
17

18. Using an Incremental CE Ratio to Guide Choice
slope = l
Incremental Cost
“BAD”
[CE ratio >l]
“GOOD”
[CE ratio <l]
Incremental Effect
18

19. Importance of Incremental Analysis
Tx Cost Effect
A $100,000 10 LM
10.001 LM
B $110,000
19

20. Importance of Incremental Analysis
Tx Cost Effect CE
A $100,000 10 LM $10,000/LM
10.001 LM
B $110,000 $10,999/LM
20

21. Importance of Incremental Analysis
Tx Cost Effect CE Incremental CE
A $100,000 10 LM $10,000/LM —
B $110,000 10.001 LM $10,999/LM $10,000,000/LM

22. Importance of Incremental Analysis
Tx Cost Effect CE Incremental CE
A $100,000 10 LM $10,000/LM —
10.001 LM
B $110,000 $10,999/LM $10,000,000/LM
Wrong Right
way to way to
calculate calculate
CE ratios! CE ratios!

23. Incremental Analysis With Multiple Options
• Example: Image three different street segments, and three different
paving options:
– Option 1: Pot Hole Repair
– Option 2: Walnut Street, chip and seal
– Option 3: Walnut Street, grind and repave
– Option 4: Main Street, chip and seal
– Option 5: Main Street, grind and repave
– Option 6: George Parkway, chip and seal
– Option 7: George Parkway, grind and repave
– Option 8: Vine Street, chip and seal
– Option 9: Vine Street, grind and repave
• Since each road carries a different volume of traffic, and since each
paving option has a different lifespan, each has different costs and
different length of benefit.
23

24. Costs and Effects
Strategy Cost ($) Life span/traffic volume ratio
1 40,288 39.08
2 44,786 42.56
3 45,944 43.04
4 47,046 43.01
5 48,596 43.60
6 54,628 43.20
7 56,812 43.83
8 58,082 43.80
9 61,119 44.62
24

25. Costs and Effects
65,000
9
60,000
8
7
55,000
Cost ($)
6
50,000
5
4
45,000 3
2
1
40,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
25

26. Eliminate Strongly Dominated Strategies
65,000
9
60,000
7
Cost ($)
55,000
50,000
5
3
45,000
2
1
40,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
26

27. Calculate CE Ratios
Strategy Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
7 56,812 43.83 (56,812-48,596)/(43.83-43.60) = 35,722
9 61,119 44.62 (61,119-56,812)/(44.62-43.83) = 5,452
27

28. Identify and Eliminate Weakly Dominated Strategies
Strat. Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
7 56,812 43.83 (56,812-48,596)/(43.83-43.60) = 35,722
9 61,119 44.62 (61,119-56,812)/(44.62-43.83) = 5,452
28

29. Identify and Eliminate Weakly Dominated Strategies
65,000
9
60,000
7
55,000
Cost ($)
50,000
5
3
45,000
2
1
40,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
29

30. Recalculate CE Ratios
Strategy Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
9 61,119 44.62 (61,119- 48,596)/(44.62-43.60) = 12,277
30

31. Recalculate CE Ratios
65,000
9
60,000
Cost ($)
55,000
50,000
5
45,000
3
2
1
40,000
39 40 41 42 43 44 45
Effect
31

32. Which Strategy to Use?
• Move down the list of undominated programs until the CE
ratio of the next program is “too high”
• What is threshold CE ratio?
– No correct answer
32

33. How to set up an economic
evaluation

34. Basic Steps in an Economic Evaluation
1. Define the problem
2. Identify the relevant alternatives
3. Estimate expected costs
4. Estimate expected effects
5. Compare expected costs and effects
6. Time preferences & discounting
7. Analyze uncertainty
8. Sources of data
34

35. Define the Problem
• Choice of perspective is particularly important
– Whose costs are considered relevant?
• Possible perspectives
– Local government
– Departments
– Nonprofits
– Subsets of residents
– Businesses
– Residents
• Whatever you choose, make it clear
35

36. Identify the Relevant Alternatives
• Possible comparators
– “Best” alternative
– Most widely used alternative
– “Do nothing”
• Appropriate comparator may
vary by location
• Potential for bias
36

37. Sources of Data
• Big challenge in public policy
– We don’t measure well
– Think through what you need
– Gather local data if possible
– Make clear what you are assuming
37

38. Calculate Expected Costs
• Associate resource utilization with
outcomes and events
• Associate costs with resource utilization
38

39. Direct Costs
• Construction & Development
• Operating Expenses
• Staff/staff time
• Advising/Consulting
39

40. Indirect Costs
• Impact on others
– Cash costs for service
replacement/augmen
tation
– Time costs to users
– Impacts on other
programs’ capacity to
meet objectives
40

41. Compare Costs and Effects
• Identify strongly and • Calculate incremental
weakly dominated cost-effectiveness
strategies ratios
41

42. Time Preference and Discounting
Time preference Common practice
• Prefer to receive
benefits now, defer • Discount costs and
costs to future benefits at same rate
• Present value of X • Most widely used rate:
occurring T years from 3% (typically in range
now 0%-10%)
= X/(1+r)T
42

43. Analyze Uncertainty
• Perform sensitivity analysis
– Helps to gauge the
reliability and robustness
of the results
– Particularly important for
public policy, given data
problems
• Identify ranges of values for
model inputs
• Recalculate economic
evaluation using these
different values
43

44. 44

45. Thank you !
Peter Mallow, AICP
Research Associate – S2 Statistical Solutions
PhD Student – University of Cincinnati
Peter.mallow@gmail.com
513.752.3845
Della Rucker, AICP, CEcD
Principal – Wise Economy Workshop
www.wiseeconomy.com
Della.rucker@wiseeconomy.com
513.288-6613
Twitter: @dellarucker
Facebook: Della Rucker Aicp Cecd
Also on Google + and LinkedIn
45