Get documents about "Introduction to Cost-effectiveness Analysis
Document Sample
Introduction to
Cost-effectiveness Analysis
Ming-Yu Fan, PhD
January 30, 2008
1
Outline
Increasing interest in C-E analysis
Incremental Cost-Effectiveness Ratio
(ICER)
Methods for constructing confidence
intervals
A simulation study
2
Increasing interest in CEA
On PubMed, the keyword “cost-effectiveness”
retrieves 49710 citations
The keyword “cost-effectiveness ratio” retrieves
3193 citations
1080 publications using “incremental cost-
effectiveness ratio”
(*) These numbers were obtained on 1/28/08. On 1/30/08, they are 49734, 3195,
and 1082 3
Measures for cost-effectiveness
Incremental Cost-Effectiveness Ratio (ICER)
ICER = (C1 - C2) / (E1 - E2)
(C1, E1) = (cost, effect) in the intervention/treatment group
(C2, E2) = (cost, effect) in the control/usual care group
Net Health Benefits (NHB)
NHB = E – C/λ
λ = a rate of substitution of dollars for health
INHB(λ) = NHB1(λ) - NHB2(λ)
4
ICER: C-E plane
ΔC
II Quadrant I Quadrant
(ΔE < 0, ΔC > 0) (ΔE > 0, ΔC > 0)
ICER < 0 ICER > 0
ΔE
III Quadrant IV Quadrant
(ΔE < 0, ΔC < 0) (ΔE > 0, ΔC < 0)
ICER > 0 ICER < 0
5
ICER = (C1 - C2) / (E1 - E2) = ΔC / ΔE
ICER – cont.
Best scenario: Quadrant IV intervention is
effective and cost-saving
Worst scenario: Quadrant II intervention
is worse than usual care and costs more
Most common scenario: Quadrant I
intervention is more effective than usual
care and costs more
6
Methodological challenge
Notation:
μC= E(ΔC), μE= E(ΔE)
VC= Var(ΔC), VE= Var(ΔE)
Cov= Covariance(ΔC, ΔE)
Expected value (mean) of a ratio does not have a close form
E(ΔC/ΔE) ≈ E(ΔC) / E(ΔE) = μC/μE
Variance of a ratio does not have a close form
Var(ΔC/ΔE) ≈ 2 V V Cov
C
2 E 2
C
E
2
C E
E C
Both approximations are based on Taylor’s expansion
7
Methodological challenge – cont.
Conventional 95% confidence interval:
[(mean - 1.96·se), (mean + 1.96·se)]
Normal distribution or large sample size
Good estimation of mean and variance
ICER:
Ratio is heavily skewed
Only approximated estimations of mean and
variance are available
8
Alternative confidence intervals
Bootstrap methods
Fieller’s method
Many simulation studies have shown
that these two (especially Fieller’s
method) yield best results
9
Bootstrap procedure
Sample the data With Replacement until the same
sample size is reached
Derive the statistics on the bootstrap sample (e.g.
mean, ICER)
Repeat the procedure for many times (e.g. 1000)
Construct the confidence interval based on the
statistics (e.g. using the 1000 ICERs)
10
Bootstrap - example
Original sample
ID Group DFD Cost DFD = depression-free-days
1 IV 240 9500
Intervention
2 IV 220 9000 N=6
3 IV 200 8500 Mean DFD / year = 190
Mean costs / year = 8000
4 IV 180 7500
5 IV 160 7000 Usual Care
6 IV 140 6500 N=4
Mean DFD / year = 130
7 UC 160 8300 Mean costs / year = 7700
8 UC 140 8000
ICER
9 UC 120 7400 = (8000-7700)/(190-130)
10 UC 100 7100 =5
11
Bootstrap sample # 1
Sampled
Bootstrapping IV and UC
Group DFD Cost samples separately
ID
3 IV 200 8500
Intervention
1 IV 240 9500 N=6
4 IV 180 7500 Mean DFD = 193
Mean costs = 8083
2 IV 220 9000
6 IV 140 6500 Usual Care
4 IV 180 7500 N=4
Mean DFD = 120
10 UC 100 7100 Mean costs = 7475
7 UC 160 8300
ICER
10 UC 100 7100
= (8083-7475)/(193-120)
9 UC 120 7400 = 8.3 12
Bootstrap sample # 2
Sampled
Bootstrapping IV/UC
Group DFD Cost samples separately
ID
2 IV 220 9000
Intervention
6 IV 140 6500 N=6
3 IV 200 8500 Mean DFD = 183
Mean costs = 7833
1 IV 240 9500
6 IV 140 6500 Usual Care
5 IV 160 7000 N=4
Mean DFD = 140
9 UC 120 7400 Mean costs = 7925
8 UC 140 8000
ICER
8 UC 140 8000
= (7833-7925)/(183-140)
7 UC 160 8300 = -2.1 13
Bootstrap example – cont.
Total number of possible bootstrap samples
= (66)·(44) = (46656)·(256) = 11,943,936
Total number of unique means/variances
= C 6 C 47 = 462·35 = 16,170
11
Repeat the same bootstrapping procedure for 1000 times, we
will obtain 1000 ICERs
These 1000 ICERs provide an approximate distribution of the
estimated ICER
We can make statistical inferences about the estimated ICER
using this approximate distribution
14
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity
0 .0 4
0 .0 2
0 .0
-10 -5 0 5 10 15 20
ICER
Bootstrap methods
Percentile:
Order the 1000 ICERs (from small to large)
Take the 25th and the 976th ICERs
The 95% confidence interval = [ICERb25, ICERb976]
Normal:
Derive the mean (μb) and standard deviation (σb) from the
1000 ICERs
μb = Σi(ICERbi) / 1000,
(σb)2 = Σi(ICERbi - μb)2 / (1000-1)
The 95% confidence interval =
[(μb - 1.96·σb), (μb + 1.96·σb)]
16
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity
0 .0 4
0 .0 2
0 .0
-10 -5 0 5 10 15 20
ICER
Bootstrap methods – cont.
Bootstrap-t:
Generate a t-statistic (tb = (ICERb – ICERs)/seb) within each
bootstrap sample then follow the percentile method using the t-
statistics
Better than the percentile method because the distribution of t-
statistic is closer to a normal distribution than the distribution of
ICERb
If the estimation of s.e. is biased, bootstrap-t might have worse
coverage rate than bootstrap-percentile method
Bias-corrected accelerated (BCa)
Instead of (α/2) and (1-α/2) (e.g. 0.025 and 0.975 for α = 0.05),
using α1 and α2 to get the percentiles
α1 and α2 are functions of bias-corrections and accelerated
scalars
18
Figure1: Bootstrap of Incremental Costs and Health Benefits
of IMPACT vs Usual Care
1500
1000
(In te r ve n tio n - U su a l C a r e )
500
In cr e m e n ta l C o sts
-5 0 0 0 -1 0 0 0
-1 5 0 0
-50 0 50 100 150
Incremental Depression-Free Days
(Intervention - Usual Care)
Katon et al. Arch Gen Psych. 2005
More about bootstrap
Why use bootstrap?
No need to assume the distribution family of the data
With today’s computing power, bootstrap can be done
very easily and quickly
Good asymptotic properties
Commonly chosen number of repetitions: 1000
The distribution of the bootstrap samples depends
on the original sample size, not so much on the
number of repetitions
20
Other bootstrap procedure
Parametric procedure (e.g. assume a
normal distribution)
Weighted bootstrap: assigning different
weights to different observations
Ex: smaller weight for outliers
Bayesian approach
26
Fieller’s method
Ratio is difficult to model
Mean and variance do not have a close form
Distribution is very skewed
Fieller suggested to transform the ICER into a
linear variable:
ICER = R = ΔC / ΔE
New statistic = S = ΔC – ΔE*R
The mean and variance of (ΔC – ΔE*R) are very
easy to derive
27
Fieller’s method – cont.
Notation:
Mean of (ΔC – ΔE*R) = μs = μΔC - μΔE*R
Standard deviation of (ΔC – ΔE*R) = σs
By Central Limit Theorem
[(ΔC – ΔE*R) – μs] / σs is normally distributed
The 95% confidence interval for the statistic can be
derived by
(C E R ) s
1 . 96
s
28
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity
0 .0 4
0 .0 2
0 .0
-10 -5 0 5 10 15 20
ICER
Fieller’s method – cont.
With simple algebra, the inequality can be re-written as
a·R2 + b·R + c ≤ 0
where
a = (ΔE)2 – 1.962 · σ2ΔE
b = 2 · [ΔE · ΔC – 1.962 · σΔEΔc ]
c = (ΔC)2 – 1.962 · σ2Δc
The upper and lower limits of the 95% confidence interval for
R (ICER) are the 2 boundaries that satisfy the inequality
The left hand side of the inequality represents a parabola
30
Fieller’s method - example
Y = 5(R^2) - 20R + 10 <= 0
40
20
0
-2 0
-5 0 5
R
31
Fieller’s method - complications
a·R2 + b·R + c ≤ 0
4 scenarios (and their solutions for R):
(1) a ≥ 0, (b2 – 4ac) ≥ 0 a close interval
(2) a < 0, (b2 – 4ac) ≥ 0 an open interval
(3) a ≥ 0, (b2 – 4ac) < 0 an empty set
(4) a < 0, (b2 – 4ac) < 0 the whole real line
32
(1) a >= 0 ; (b^2 - 4ac) >= 0 (2) a < 0 ; (b^2 - 4ac) >= 0
60
60
20
20
-2 0
-2 0
-6 0
-6 0
-5 0 5 -5 0 5
R R
(3) a >= 0 ; (b^2 - 4ac) < 0 (4) a < 0 ; (b^2 - 4ac) < 0
60
60
20
20
-2 0
-2 0
-6 0
-6 0
-5 0 5 -5 0 5
R R
Fieller’s method – cont.
Why use Fieller’s method
Several simulation studies have demonstrated that Fieller’s
method yield better results than other method
Easy to compute
Unique results of the same data (contrast to bootstrap)
When does Fieller’s method result in meaningless
confidence intervals?
Scenario (3) where a >= 0 and (b^2 – 4ac) < 0 never happens
When a ≥ 0, the confidence interval is a closed interval
34
Fieller’s method – cont.
What’s the interpretation for a ≥ 0?
a = (ΔE)2 – 1.962 · σ2ΔE ≥ 0
(ΔE/σΔE )2 ≥ 1.962
the incremental effect is statistically significant
When the incremental effect is not statistically
significant, it is not interesting clinical-wise to conduct cost-
effectiveness analysis. Statistically, it is appropriate to run analysis
using ICER and make inferences based on it
35
A simulation study
3 distribution families
(1) Both effect and cost follow a normal distribution
(2) Both effect and cost follow a log-normal distribution
(right skewed)
(3) Effect normal; Cost log-normal
For each distribution family, 162 different distributions:
Sample size: 50, 100, 400
Correlation coefficient between cost and effect: -0.5, 0.1, 0.5
Ratio of the variances (control / intervention): 1, 3
ICER: 2000, 10000, 50000
3 distances between the ICER and the origin
(relevant to the effect size)
36
Distance between ICER and origin
ΔC
ΔEB
B
ΔEA
A
ΔE
37
A and B have the same ICER
Normal Log-normal Mixed
Method Category Distribution Distribution Distribution
Taylor Coverage % 92.7 92.5 92.5
Fieller – close interval Coverage % 93.8 93.7 93.7
Fieller – all scenarios Coverage % 95.0 94.7 94.7
Close interval (85%) (84.8%) (84.8%)
Open interval (33%) (33.0%) (31.8%)
Whole real line (3%) (2.6%) (3.1%)
Bootstrap – percentile Coverage % 95.7 95.5 95.5
Bootstrap – normal Coverage % 95.8 95.7 95.7
Bootstrap – t Coverage % 83.9 83.7 83.8
Bootstrap - BCa1 Coverage % 92.4 92.2 92.2
Bootstrap - BCa2 Coverage % 92.3 92.0 92.0
Mixed Distribution
ICER 50000 2000 10000
Distance 1/4 4 1/4 4 1/4 4
Method N 50 400 50 400 50 400 50 400 50 400 50 400
Taylor Coverage % 75.4 88.8 91.1 95.0 96.0 95.3 94.3 94.8 92.9 95.3 94.6 94.9
Fieller – close interval Coverage % 72.5 95.0 96.9 95.0 94.9 95.0 94.0 94.9 95.7 94.9 94.6 94.9
Fieller – all scenarios Coverage % 95.0 94.8 94.9 95.0 94.6 95.0 94.0 94.9 94.8 94.9 94.6 94.9
Close interval 11.2 50.7 78.0 100.0 92.5 100.0 100.0 100.0 71.3 100.0 100.0 100.0
Open interval 87.8 49.5 22.0 4.5 23.2
Whole real line 1.7 3.6 7.0
Bootstrap – percentile Coverage % 95.8 97.6 97.6 95.0 95.2 94.9 93.8 94.9 96.9 94.8 94.5 94.7
Bootstrap – normal Coverage % 97.6 96.0 95.1 95.6 97.8 95.7 94.2 94.8 96.3 96.0 94.8 94.8
Bootstrap – t Coverage % 48.4 65.6 73.1 92.8 84.0 94.0 94.1 94.9 76.0 92.3 94.2 94.8
Bootstrap - BCa1 Coverage % 66.9 89.8 95.6 94.8 95.1 94.8 93.8 94.6 94.5 94.7 94.5 94.5
Bootstrap - BCa2 Coverage % 65.9 89.7 95.5 94.8 94.7 94.7 93.6 94.7 93.8 94.6 94.3 94.6
Problem with ICER
Can’t distinguish ICERs on quadrant I from quadrant III
Quadrant I: positive intervention effect, more costly
Quadrant III: negative intervention effect, cost saving
Can’t distinguish ICERs on quadrant II from quadrant IV
Quadrant II: negative intervention effect, more costly
Quadrant IV: positive intervention effect, cost saving
Negative ICERs are difficult to interpret
40
Problem with ICER – cont.
Example: 3 studies comparing to the same control group
A: ΔE = 10, ΔC = -1000
B: ΔE = 10, ΔC = -500
C: ΔE = 5, ΔC = -500
Pair-wise comparison:
A is better than B (same effect, more cost-saving)
B is better than C (same cost-saving, more effective)
A>B>C
ICER:
(A) ICER = -100; (B) ICER = -50; (C) ICER = -100
A=C>B
41
Summary
ICER is an intuitive measure for cost-effectiveness analysis
but is only appropriate when
The incremental effect is statistically significant from 0
Both incremental effect and incremental cost are positive
The available methods might not be appropriate for other
scenarios
With large samples, normal method, bootstrap-percentile
method, and Fieller’s method are all equally good
For small samples (or extremely skewed costs), my
preference is (1) Fieller’s method (2) bootstrap-percentile (3)
normal method
42
2008 Last Wednesday Health Services Methodology Seminar Series
DATE LECTURER TOPIC LECTURE ROOM
Jan. 30 Ming-Yu Fan Introduction to Cost-effectiveness Analysis HSB T-530
Feb. 27 Joan Russo Mediator and Moderator Analyses in Psychiatric Health Services Research BB 1640
Mar. 26 Marcia A. Ciol Sample Size Calculation BB 1640
April 30 Ming-Yu Fan Longitudinal Data Analysis and Survival Analysis BB 1640
May 28 Marcia A. Ciol Using Weights in Analysis of Multi-stage Surveys TBA
June 25 Ming-Yu Fan Missing Data Analysis – Multiple Imputation BB 1640
July & Aug No Seminar
Sept. 24 Jutta M. Joesch Analyses of Employment as Outcomes BB 1640
Oct. 29 Joan Russo ROC Curves and Diagnostic Statistics for Health Services Research BB 1640
Nov. 19* Jutta M. Joesch Propensity Score Matching BB 1640
Dec No Seminar
* November 26 is the day before Thanksgiving. The seminar is moved it to 11/19.
Lecturers:
Marcia A. Ciol, Ph.D. in Biostatistics, Research Associate Professor, Rehabilitation Medicine, marciac@u.washington.edu
Ming-Yu Fan, Ph.D. in Biostatistics, Research Assistant Professor, Psychiatry and Behavioral Sciences, myfan@u.washington.edu
Jutta M. Joesch, Ph.D. in Economics, Research Associate Professor, Psychiatry and Behavioral Sciences, joesch@u.washington.edu
Joan Russo, Ph.D. in Psychology, Associate Professor, Psychiatry and Behavioral Sciences, jerusso@u.washington.edu
Related docs
