Docstoc

Benefit-Cost Analysis

Document Sample
Benefit-Cost Analysis Powered By Docstoc
					    Uncertainty &
   Decision Making

       James K. Hammitt
Harvard Center for Risk Analysis
                    Outline
Uncertainty aversion & value of information
Representing uncertainty as probability
Policy evaluation
  – Components of uncertainty
  – Examples
     • Diesel-vehicle emissions
     • Mercury from power plants
Expert judgment
                                         2
 Aversion to Risk, Uncertainty,
   Ambiguity, & Ignorance
Humans dislike absence of certainty
  –   Risk: "objective" probabilities
  –   Uncertainty: subjective probabilities
  –   Ambiguity: unknown probabilities
  –   Ignorance: unknown possible outcomes

Should we take greater precaution when risks
  are more uncertain?
How should we describe uncertainty?
                                               3
          Perils of Prudence
           (Nichols & Zeckhauser 1986)
Conservative assumptions, worst-case analysis,
 uncertainty aversion can increase harm
Technology Deaths      Probability   Expected deaths
Uncertain  1           0.99
           1,000       0.01           11
Certain    101         1.0           101

Using upper-bound risk estimates, Certain would
 be preferred to Uncertain

                                                  4
          Perils of Prudence
If decision is repeated for 10 pairs of technologies
   (and risks are independent)
Technology        Deaths            Probability
Uncertain         10                0.904
                  < 1,010           0.996
Certain           1, 010            1.0

Policy of choosing Certain (with smaller upper-
  bound risk) is almost sure to kill more people

                                                   5
           Value of Information
For each of 10 technologies, learn true number of deaths
  for ambiguous type
   – Choose Uncertain if it causes 1 death
   – Choose Certain otherwise
Choice                     Expected deaths
Uncertain (always)                 110
Certain (always)                 1,010
Perfect information                 20
Expected value of information       90 lives saved


                                                      6
       Value(s) of Information
Increase chance of choosing decision that is best for
   actual conditions
   – "Expected value of information" in decision theory
Overcome burden of proof needed to depart from status
  quo policy or default assumption
   – Compensate for decision rule that does not maximize expected
     value of outcome
Reassure decision makers and affected public that
  decision is appropriate
   – Enhance compliance, minimize opposition & legal challenges
   – Incorporate compliance and challenges as factors in analysis?

                                                                     7
Quantifying Uncertainty with Probability
Probabilities of health risks are subjective
   – Often extrapolated from animal experiments or observational
     human data
   – Quantitative measure of degree of belief
   – Individuals can have different probabilities for same event
There is no "true" or "objective" probability
All probabilities are subjective
   – "Objective randomness" is not random but chaos (e.g., coin toss,
     roulette wheel)
       • Deterministic process
       • Sensitively dependent on initial conditions (butterfly flapping wings
         in China may cause hurricane in Atlantic)
   – Insufficient information about initial conditions

                                                                                 8
Disagreement Among Experts
Individuals can hold different probabilities
   – When evidence to choose among them is inadequate
As evidence accumulates
   – Experts should update their probabilities
      • "When somebody persuades me that I am wrong, I change my
        mind. What do you do?" - John Maynard Keynes
   – Ultimately, probabilities should converge
      • Coin toss, roulette wheel
      • "In the long run we are all dead."- John Maynard Keynes


                                                                   9
  Quantifying Uncertainty About
        Policy Outcomes
Use simulation model to combine multiple
 inputs
   – Inputs: releases to environment, fate & transport,
     human exposure, dose-response function
   – Outputs: adverse health events, benefits and costs
Represent uncertainty about each component
 of model as probability distribution
Calculate probability distribution of output using
 Monte Carlo analysis (or alternatives)

                                                          10
   Components of Uncertainty
"Model uncertainty"
   – Functional form
   – Causality
"Parameter uncertainty"
   – Sampling variation in data (estimation error)
   – Relevance of data to application
May be helpful to distinguish, but can combine using
  "super-model"
   – Weighted sum of alternative models, weights are uncertain
     parameters
Note: statistical confidence intervals are not sufficient;
  exclude many important sources of uncertainty

                                                                 11
Example: Low-Dose Extrapolation

Estimate risk at high dose, where risk is
 measurable (e.g., 1/10, 1/100)
Extrapolate to risk at low dose
Extrapolation can be sensitive to choice
 among models that fit observed data
 equally well


                                            12
                           0.5                                                      10-2
                                     Dose Response                                          Low Dose
                                                                                           Extrapolation
Probability of Response




                                                                      Excess Risk
                                                                                    10-5
                          0.25




                                                                                    10-8
                                                                                            X M    WL, G      P
                            0
                                 0            75               150                         10-6        10-2            102

                                                                Dose d (ppm)
                                          X – Linear Extrapolation      L – Logit Model
                                          M – Multi-Stage Model         G – Gamma Multi-Hit Model
                                          W – Weibull Model             P – Probit Model
                          Low-dose extrapolation for 2-acetylaminofluorene under several mathematical models.
                                                                                                                  13
 Policy-Evaluation Examples
Retrofit diesel trucks & buses in Mexico
 City
Benefits of reducing mercury emissions
 from electric power plants




                                           14
     Diesel Retrofit: Benefit-Cost Model
                                   Intake Fraction     Epidemiology        VSL



                                                                       Valuation of
                   Pollutant        Exposure                 Health
                                                                         Health
                   Emission        Concentration             Effects
                                                                        Benefits
                    • Primary PM    • Primary PM             • Death
                    • SO2           • Ammonium
                    • HC              Sulfate
     Control                        • Secondary
                                      Organic PM                                      Net Benefits
     Decision

•   Catalyzed DPF
•   Self-Regenerating DPF
                                           Control Costs
•   Removable DPF
•   DOC
                                              • Capital
                                              • O/M
                                              • Inspection                                  15
Annual Deaths Averted (per 1000 vehicles)
       (Error bars show interquartile range)


                              12


          deaths averted/yr   10

                               8

                               6

                               4

                               2

                               0
                                         Bus       Truck>3T       Trailers

                              Catalyzed DPF         Active Regeneration DPF
                              Oxidation Catalyst
                                                                              16
   Net Benefits of Catalyzed Filter v. Alternatives
                               (US$ millions per 1000 vehicles, model year ≥ 1994)
Self-regenerating DPF




                           Tractor Trailers



                                   Trucks



                                    Buses
Oxidation Catalyst (DOC)




                           Tractor Trailers



                                   Trucks



                                    Buses


                                              -5   0   5   10     15     20      25
                                                                                      17
                      700

                      600                                             Relative importance
                      500                                             of uncertainty about
                                                                      variables
Million USD




                      400
                                                                      (Interquartile range, annual net
                      300                                             benefits of retrofitting all
                                                                      vehicles with active
                      200                                             regeneration filters holding
                                                                      other variables fixed at
                      100                                             medians)
                          0
                                                           te
                                                          on




                                                          se




                                                           st
                                                        l es




                                                          cy
                                                         ife



                                                      Ra




                                                      Co
                                                      cti




                                                     on




                                                    ien
                                                     lL
                                                   iab

                                                  ra




                                                io n
                                                 sp
                                                 ca




                                                ter
                                                fic
                                               ar

                                            eF




                                            Re
                                             sti




                                            Ef

                                            Fil
                                           iss
                          nV




                                         ati
                                         ak




                                         n-




                                        ter
                                      Em
                       tai




                                      St

                                     tio
                                     Int




                                    Fil
                    er




                                 tra
                                   a
                 nc




                                of

                               en
                 lU



                                 l ue

                            nc
              Al




                          Co
                               Va




                                   Variable(s) considered uncertain                            18
    Mercury from Power Plants
   Emissions    Deposition




IQ loss?
Heart attack?

                Exposure
                                19
Summary of Benefits




                      20
                                   Benefits of Reducing MeHg
                                       Intake 10% (US)
                          1



                         0.8
Cumulative Probability




                         0.6



                         0.4



                         0.2



                          0
                               0     1   2   3            4           5           6           7   8   9        10
                                                 Total Benefits Both Genders (Billion $/Year)


                                                                                                          21
                             Importance of Variable
                          (Rank Correlation Coefficient)




                                 0.00
                                 0.10
                                 0.20
                                 0.30
                                 0.40
                                 0.50
                                 0.60
                                 0.70
                                 0.80
                                 0.90
             Plausibility of
             Heart Attack
               Causality


                   IQ Hair
                  Coefficient

             Plausibility of
             Neurotoxicity
              Threshold


                 Blood/Intake
                  Coefficient


                 Earnings/IQ
                 Coefficient




     Variables
                     Heart
                  Attack/Hair
                  Coefficient

                    Value of
                   Statistical
                                                                                                 Relative importance of




                      Life
                                                           (Correlation of input and output)
                                                                                               uncertainty about variables




                  Hair/Blood
                  Coefficient
22




                 Heart Attack
                  Time lag
                         Benefits & Sensitivity to Key Parameter

                         1.0
                                                                                                 Plausibility 5%
                                                                                                 Plausibility 25%
                         0.8
                                                                                                 Plausibility 50%
Cumulative Probability




                                                                                                 Plausibility 75%
                         0.6
                                                                                                 Plausibility 95%


                         0.4


                         0.2


                         0.0
                               0       1             2             3             4           5
                                   Expected Present Value of 10% Reduction in U.S. Mercury
                                              Exposures (Billions of $ per year)

                                                                                                      23
             Expert Judgment
Risk assessment models incorporate many
  assumptions
   – Choices usually made by modelers, informed by
     scientific literature
   – Meta-analysis can be used when literature is rich
Alternative (or complement): expert elicitation
   – Experts provide probability distributions for key
     parameters
   – Rigorous, replicable process
      • Selection of experts
      • Preparation
      • Interview
                                                         24
      Key Elicitation Question
           (Mortality Effect of PM2.5)
"What is your estimate of the
true percent change in
annual, all-cause mortality in
the adult U.S. population
resulting from a permanent
1µg/m3 reduction in annual
average PM2.5"


5th, 25th, 50th, 75th, and 95th
percentiles of cumulative
density function



                                         25
                                26
Source: EPA PM NAAQS RIA 2006
Source: EPA PM NAAQS RIA 2006   27
                                           Performance: Expert Predictions of Ambient
                                                   Benzene Concentrations

                                                   Means                                                                     60
                                                                                                                                  90th Percentiles
                                      60

                                      55                                                                                     55




                                                                                       6-day Average Concentration (ug/m )
                                                                                       3
6-day Average Concentration (ug/m )
3




                                      50                                                                                     50


                                      45                                                                                     45


                                      40                                                                                     40


                                      35                                                                                     35


                                      30                                                                                     30


                                      25                                    NHEXAS                                           25


                                      20                                    Result                                           20


                                      15
                                                                                                                             15

                                                                                                                             10
                                      10

                                                                                                                             5
                                      5

                                                                                                                             0
                                      0
                                                                                                                                  A   B   C       D    E   F   G
                                           A   B   C     D      E   F   G
                                                       Expert                                                                                 Expert



                                                                                                                                                                   28
                                                                               Source: Walker et al. 2003
              Conclusions
Outcomes of any policy alternative are
 uncertain ex ante
Characterize uncertainty as probability
 distributions
Propagate uncertainties about model inputs
  using Monte Carlo analysis
Agreement on probabilities may not exist
Pattern of precautionary policies may be costly
                                                  29

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:2/12/2012
language:
pages:29