Benefit-Cost Analysis by malj

VIEWS: 2 PAGES: 29

• pg 1
```									    Uncertainty &
Decision Making

James K. Hammitt
Harvard Center for Risk Analysis
Outline
Uncertainty aversion & value of information
Representing uncertainty as probability
Policy evaluation
– Components of uncertainty
– Examples
• Diesel-vehicle emissions
• Mercury from power plants
Expert judgment
2
Aversion to Risk, Uncertainty,
Ambiguity, & Ignorance
Humans dislike absence of certainty
–   Risk: "objective" probabilities
–   Uncertainty: subjective probabilities
–   Ambiguity: unknown probabilities
–   Ignorance: unknown possible outcomes

Should we take greater precaution when risks
are more uncertain?
How should we describe uncertainty?
3
Perils of Prudence
(Nichols & Zeckhauser 1986)
Conservative assumptions, worst-case analysis,
uncertainty aversion can increase harm
Technology Deaths      Probability   Expected deaths
Uncertain  1           0.99
1,000       0.01           11
Certain    101         1.0           101

Using upper-bound risk estimates, Certain would
be preferred to Uncertain

4
Perils of Prudence
If decision is repeated for 10 pairs of technologies
(and risks are independent)
Technology        Deaths            Probability
Uncertain         10                0.904
< 1,010           0.996
Certain           1, 010            1.0

Policy of choosing Certain (with smaller upper-
bound risk) is almost sure to kill more people

5
Value of Information
For each of 10 technologies, learn true number of deaths
for ambiguous type
– Choose Uncertain if it causes 1 death
– Choose Certain otherwise
Choice                     Expected deaths
Uncertain (always)                 110
Certain (always)                 1,010
Perfect information                 20
Expected value of information       90 lives saved

6
Value(s) of Information
Increase chance of choosing decision that is best for
actual conditions
– "Expected value of information" in decision theory
Overcome burden of proof needed to depart from status
quo policy or default assumption
– Compensate for decision rule that does not maximize expected
value of outcome
Reassure decision makers and affected public that
decision is appropriate
– Enhance compliance, minimize opposition & legal challenges
– Incorporate compliance and challenges as factors in analysis?

7
Quantifying Uncertainty with Probability
Probabilities of health risks are subjective
– Often extrapolated from animal experiments or observational
human data
– Quantitative measure of degree of belief
– Individuals can have different probabilities for same event
There is no "true" or "objective" probability
All probabilities are subjective
– "Objective randomness" is not random but chaos (e.g., coin toss,
roulette wheel)
• Deterministic process
• Sensitively dependent on initial conditions (butterfly flapping wings
in China may cause hurricane in Atlantic)
– Insufficient information about initial conditions

8
Disagreement Among Experts
Individuals can hold different probabilities
– When evidence to choose among them is inadequate
As evidence accumulates
– Experts should update their probabilities
• "When somebody persuades me that I am wrong, I change my
mind. What do you do?" - John Maynard Keynes
– Ultimately, probabilities should converge
• Coin toss, roulette wheel
• "In the long run we are all dead."- John Maynard Keynes

9
Policy Outcomes
Use simulation model to combine multiple
inputs
– Inputs: releases to environment, fate & transport,
human exposure, dose-response function
– Outputs: adverse health events, benefits and costs
of model as probability distribution
Calculate probability distribution of output using
Monte Carlo analysis (or alternatives)

10
Components of Uncertainty
"Model uncertainty"
– Functional form
– Causality
"Parameter uncertainty"
– Sampling variation in data (estimation error)
– Relevance of data to application
May be helpful to distinguish, but can combine using
"super-model"
– Weighted sum of alternative models, weights are uncertain
parameters
Note: statistical confidence intervals are not sufficient;
exclude many important sources of uncertainty

11
Example: Low-Dose Extrapolation

Estimate risk at high dose, where risk is
measurable (e.g., 1/10, 1/100)
Extrapolate to risk at low dose
Extrapolation can be sensitive to choice
among models that fit observed data
equally well

12
0.5                                                      10-2
Dose Response                                          Low Dose
Extrapolation
Probability of Response

Excess Risk
10-5
0.25

10-8
X M    WL, G      P
0
0            75               150                         10-6        10-2            102

Dose d (ppm)
X – Linear Extrapolation      L – Logit Model
M – Multi-Stage Model         G – Gamma Multi-Hit Model
W – Weibull Model             P – Probit Model
Low-dose extrapolation for 2-acetylaminofluorene under several mathematical models.
13
Policy-Evaluation Examples
Retrofit diesel trucks & buses in Mexico
City
Benefits of reducing mercury emissions
from electric power plants

14
Diesel Retrofit: Benefit-Cost Model
Intake Fraction     Epidemiology        VSL

Valuation of
Pollutant        Exposure                 Health
Health
Emission        Concentration             Effects
Benefits
• Primary PM    • Primary PM             • Death
• SO2           • Ammonium
• HC              Sulfate
Control                        • Secondary
Organic PM                                      Net Benefits
Decision

•   Catalyzed DPF
•   Self-Regenerating DPF
Control Costs
•   Removable DPF
•   DOC
• Capital
• O/M
• Inspection                                  15
Annual Deaths Averted (per 1000 vehicles)
(Error bars show interquartile range)

12

deaths averted/yr   10

8

6

4

2

0
Bus       Truck>3T       Trailers

Catalyzed DPF         Active Regeneration DPF
Oxidation Catalyst
16
Net Benefits of Catalyzed Filter v. Alternatives
(US\$ millions per 1000 vehicles, model year ≥ 1994)
Self-regenerating DPF

Tractor Trailers

Trucks

Buses
Oxidation Catalyst (DOC)

Tractor Trailers

Trucks

Buses

-5   0   5   10     15     20      25
17
700

600                                             Relative importance
variables
Million USD

400
(Interquartile range, annual net
300                                             benefits of retrofitting all
vehicles with active
200                                             regeneration filters holding
other variables fixed at
100                                             medians)
0
te
on

se

st
l es

cy
ife

Ra

Co
cti

on

ien
lL
iab

ra

io n
sp
ca

ter
fic
ar

eF

Re
sti

Ef

Fil
iss
nV

ati
ak

n-

ter
Em
tai

St

tio
Int

Fil
er

tra
a
nc

of

en
lU

l ue

nc
Al

Co
Va

Variable(s) considered uncertain                            18
Mercury from Power Plants
Emissions    Deposition

IQ loss?
Heart attack?

Exposure
19
Summary of Benefits

20
Benefits of Reducing MeHg
Intake 10% (US)
1

0.8
Cumulative Probability

0.6

0.4

0.2

0
0     1   2   3            4           5           6           7   8   9        10
Total Benefits Both Genders (Billion \$/Year)

21
Importance of Variable
(Rank Correlation Coefficient)

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Plausibility of
Heart Attack
Causality

IQ Hair
Coefficient

Plausibility of
Neurotoxicity
Threshold

Blood/Intake
Coefficient

Earnings/IQ
Coefficient

Variables
Heart
Attack/Hair
Coefficient

Value of
Statistical
Relative importance of

Life
(Correlation of input and output)

Hair/Blood
Coefficient
22

Heart Attack
Time lag
Benefits & Sensitivity to Key Parameter

1.0
Plausibility 5%
Plausibility 25%
0.8
Plausibility 50%
Cumulative Probability

Plausibility 75%
0.6
Plausibility 95%

0.4

0.2

0.0
0       1             2             3             4           5
Expected Present Value of 10% Reduction in U.S. Mercury
Exposures (Billions of \$ per year)

23
Expert Judgment
Risk assessment models incorporate many
assumptions
– Choices usually made by modelers, informed by
scientific literature
– Meta-analysis can be used when literature is rich
Alternative (or complement): expert elicitation
– Experts provide probability distributions for key
parameters
– Rigorous, replicable process
• Selection of experts
• Preparation
• Interview
24
Key Elicitation Question
(Mortality Effect of PM2.5)
"What is your estimate of the
true percent change in
annual, all-cause mortality in
resulting from a permanent
1µg/m3 reduction in annual
average PM2.5"

5th, 25th, 50th, 75th, and 95th
percentiles of cumulative
density function

25
26
Source: EPA PM NAAQS RIA 2006
Source: EPA PM NAAQS RIA 2006   27
Performance: Expert Predictions of Ambient
Benzene Concentrations

Means                                                                     60
90th Percentiles
60

55                                                                                     55

6-day Average Concentration (ug/m )
3
6-day Average Concentration (ug/m )
3

50                                                                                     50

45                                                                                     45

40                                                                                     40

35                                                                                     35

30                                                                                     30

25                                    NHEXAS                                           25

20                                    Result                                           20

15
15

10
10

5
5

0
0
A   B   C       D    E   F   G
A   B   C     D      E   F   G
Expert                                                                                 Expert

28
Source: Walker et al. 2003
Conclusions
Outcomes of any policy alternative are
uncertain ex ante
Characterize uncertainty as probability
distributions