Introduction to Cost-effectiveness Analysis

Shared by:
Categories
-
Stats
views:
99
posted:
11/20/2008
language:
English
pages:
43
Document Sample

```							Introduction to
Cost-effectiveness Analysis

Ming-Yu Fan, PhD
January 30, 2008

1
Outline
   Increasing interest in C-E analysis

   Incremental Cost-Effectiveness Ratio
(ICER)

   Methods for constructing confidence
intervals

   A simulation study
2
Increasing interest in CEA

   On PubMed, the keyword “cost-effectiveness”
retrieves 49710 citations

   The keyword “cost-effectiveness ratio” retrieves
3193 citations

   1080 publications using “incremental cost-
effectiveness ratio”

(*) These numbers were obtained on 1/28/08. On 1/30/08, they are 49734, 3195,
and 1082                                                                   3
Measures for cost-effectiveness

   Incremental Cost-Effectiveness Ratio (ICER)
ICER = (C1 - C2) / (E1 - E2)
   (C1, E1) = (cost, effect) in the intervention/treatment group
   (C2, E2) = (cost, effect) in the control/usual care group

   Net Health Benefits (NHB)
NHB = E – C/λ
   λ = a rate of substitution of dollars for health
   INHB(λ) = NHB1(λ) - NHB2(λ)

4
ICER: C-E plane
ΔC

(ΔE < 0, ΔC > 0)               (ΔE > 0, ΔC > 0)

ICER < 0                       ICER > 0

ΔE

(ΔE < 0, ΔC < 0)                (ΔE > 0, ΔC < 0)

ICER > 0                         ICER < 0

5
   ICER = (C1 - C2) / (E1 - E2) = ΔC / ΔE
ICER – cont.

   Best scenario: Quadrant IV  intervention is
effective and cost-saving

   Worst scenario: Quadrant II  intervention
is worse than usual care and costs more

   Most common scenario: Quadrant I 
intervention is more effective than usual
care and costs more
6
Methodological challenge
   Notation:
   μC= E(ΔC), μE= E(ΔE)
   VC= Var(ΔC), VE= Var(ΔE)
   Cov= Covariance(ΔC, ΔE)

   Expected value (mean) of a ratio does not have a close form
   E(ΔC/ΔE) ≈ E(ΔC) / E(ΔE) = μC/μE

   Variance of a ratio does not have a close form
   Var(ΔC/ΔE) ≈    2  V  V     Cov 
C
  2  E 2
C

       E
2
C  E 
 E   C

   Both approximations are based on Taylor’s expansion

7
Methodological challenge – cont.

   Conventional 95% confidence interval:
[(mean - 1.96·se), (mean + 1.96·se)]
   Normal distribution or large sample size
   Good estimation of mean and variance

   ICER:
   Ratio is heavily skewed
   Only approximated estimations of mean and
variance are available

8
Alternative confidence intervals

   Bootstrap methods

   Fieller’s method

   Many simulation studies have shown
that these two (especially Fieller’s
method) yield best results
9
Bootstrap procedure
   Sample the data With Replacement until the same
sample size is reached

   Derive the statistics on the bootstrap sample (e.g.
mean, ICER)

   Repeat the procedure for many times (e.g. 1000)

   Construct the confidence interval based on the
statistics (e.g. using the 1000 ICERs)

10
Bootstrap - example
   Original sample
ID   Group DFD   Cost          DFD = depression-free-days
1    IV   240    9500
   Intervention
2    IV   220    9000              N=6
3    IV   200    8500              Mean DFD / year = 190
   Mean costs / year = 8000
4    IV   180    7500
5    IV   160    7000          Usual Care
6    IV   140    6500              N=4
   Mean DFD / year = 130
7    UC   160    8300              Mean costs / year = 7700
8    UC   140    8000
   ICER
9    UC   120    7400           = (8000-7700)/(190-130)
10   UC   100    7100           =5
11
Bootstrap sample # 1
Sampled
   Bootstrapping IV and UC
Group DFD   Cost       samples separately
ID
3         IV   200    8500
   Intervention
1         IV   240    9500              N=6
4         IV   180    7500              Mean DFD = 193
   Mean costs = 8083
2         IV   220    9000
6         IV   140    6500          Usual Care
4         IV   180    7500              N=4
   Mean DFD = 120
10        UC   100    7100              Mean costs = 7475
7         UC   160    8300
   ICER
10        UC   100    7100
= (8083-7475)/(193-120)
9         UC   120    7400           = 8.3                     12
Bootstrap sample # 2
Sampled
   Bootstrapping IV/UC
Group DFD   Cost       samples separately
ID
2         IV   220    9000
   Intervention
6         IV   140    6500              N=6
3         IV   200    8500              Mean DFD = 183
   Mean costs = 7833
1         IV   240    9500
6         IV   140    6500          Usual Care
5         IV   160    7000              N=4
   Mean DFD = 140
9         UC   120    7400              Mean costs = 7925
8         UC   140    8000
   ICER
8         UC   140    8000
= (7833-7925)/(183-140)
7         UC   160    8300           = -2.1                    13
Bootstrap example – cont.
   Total number of possible bootstrap samples
= (66)·(44) = (46656)·(256) = 11,943,936

   Total number of unique means/variances
= C 6  C 47 = 462·35 = 16,170
11

   Repeat the same bootstrapping procedure for 1000 times, we
will obtain 1000 ICERs

   These 1000 ICERs provide an approximate distribution of the
estimated ICER

   We can make statistical inferences about the estimated ICER
using this approximate distribution
14
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity

0 .0 4
0 .0 2
0 .0

-10        -5        0           5        10       15   20

ICER
Bootstrap methods
   Percentile:
   Order the 1000 ICERs (from small to large)
   Take the 25th and the 976th ICERs
   The 95% confidence interval = [ICERb25, ICERb976]

   Normal:
   Derive the mean (μb) and standard deviation (σb) from the
1000 ICERs
   μb = Σi(ICERbi) / 1000,
   (σb)2 = Σi(ICERbi - μb)2 / (1000-1)
   The 95% confidence interval =
[(μb - 1.96·σb), (μb + 1.96·σb)]
16
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity

0 .0 4
0 .0 2
0 .0

-10        -5        0           5        10       15   20

ICER
Bootstrap methods – cont.
   Bootstrap-t:
   Generate a t-statistic (tb = (ICERb – ICERs)/seb) within each
bootstrap sample then follow the percentile method using the t-
statistics
   Better than the percentile method because the distribution of t-
statistic is closer to a normal distribution than the distribution of
ICERb
   If the estimation of s.e. is biased, bootstrap-t might have worse
coverage rate than bootstrap-percentile method

   Bias-corrected accelerated (BCa)
   Instead of (α/2) and (1-α/2) (e.g. 0.025 and 0.975 for α = 0.05),
using α1 and α2 to get the percentiles
   α1 and α2 are functions of bias-corrections and accelerated
scalars
18
Figure1: Bootstrap of Incremental Costs and Health Benefits
of IMPACT vs Usual Care
1500
1000
(In te r ve n tio n - U su a l C a r e )
500
In cr e m e n ta l C o sts

-5 0 0           0        -1 0 0 0
-1 5 0 0

-50              0                      50        100              150

Incremental Depression-Free Days
(Intervention - Usual Care)

Katon et al. Arch Gen Psych. 2005

   Why use bootstrap?
   No need to assume the distribution family of the data
   With today’s computing power, bootstrap can be done
very easily and quickly
   Good asymptotic properties

   Commonly chosen number of repetitions: 1000

   The distribution of the bootstrap samples depends
on the original sample size, not so much on the
number of repetitions
20
Other bootstrap procedure

   Parametric procedure (e.g. assume a
normal distribution)

   Weighted bootstrap: assigning different
weights to different observations
   Ex: smaller weight for outliers
   Bayesian approach

26
Fieller’s method
   Ratio is difficult to model
   Mean and variance do not have a close form
   Distribution is very skewed

   Fieller suggested to transform the ICER into a
linear variable:
   ICER = R = ΔC / ΔE
   New statistic = S = ΔC – ΔE*R

   The mean and variance of (ΔC – ΔE*R) are very
easy to derive

27
Fieller’s method – cont.
   Notation:
   Mean of (ΔC – ΔE*R) = μs = μΔC - μΔE*R
   Standard deviation of (ΔC – ΔE*R) = σs

   By Central Limit Theorem
   [(ΔC – ΔE*R) – μs] / σs is normally distributed

   The 95% confidence interval for the statistic can be
derived by

(C  E  R )   s
 1 . 96
s
28
Distribution of 1000 bootstrap ICERs: IMPACT study
0 .1 0
0 .0 8
0 .0 6
D e n sity

0 .0 4
0 .0 2
0 .0

-10        -5        0           5        10       15   20

ICER
Fieller’s method – cont.
   With simple algebra, the inequality can be re-written as

a·R2 + b·R + c ≤ 0

where
   a = (ΔE)2 – 1.962 · σ2ΔE
   b = 2 · [ΔE · ΔC – 1.962 · σΔEΔc ]
   c = (ΔC)2 – 1.962 · σ2Δc

   The upper and lower limits of the 95% confidence interval for
R (ICER) are the 2 boundaries that satisfy the inequality

   The left hand side of the inequality represents a parabola

30
Fieller’s method - example

Y = 5(R^2) - 20R + 10 <= 0
40
20
0
-2 0

-5   0                          5
R
31
Fieller’s method - complications

   a·R2 + b·R + c ≤ 0

   4 scenarios (and their solutions for R):
   (1) a ≥ 0, (b2 – 4ac) ≥ 0  a close interval
   (2) a < 0, (b2 – 4ac) ≥ 0  an open interval
   (3) a ≥ 0, (b2 – 4ac) < 0  an empty set
   (4) a < 0, (b2 – 4ac) < 0  the whole real line

32
(1) a >= 0 ;       (b^2 - 4ac) >= 0               (2) a < 0 ;       (b^2 - 4ac) >= 0

60

60
20

20
-2 0

-2 0
-6 0

-6 0
-5                  0               5             -5                 0               5
R                                                 R
(3) a >= 0 ;       (b^2 - 4ac) < 0                (4) a < 0 ;       (b^2 - 4ac) < 0
60

60
20

20
-2 0

-2 0
-6 0

-6 0

-5                  0               5             -5                 0               5
R                                                 R
Fieller’s method – cont.

   Why use Fieller’s method
   Several simulation studies have demonstrated that Fieller’s
method yield better results than other method
   Easy to compute
   Unique results of the same data (contrast to bootstrap)

   When does Fieller’s method result in meaningless
confidence intervals?
   Scenario (3) where a >= 0 and (b^2 – 4ac) < 0 never happens
   When a ≥ 0, the confidence interval is a closed interval

34
Fieller’s method – cont.
   What’s the interpretation for a ≥ 0?

   a = (ΔE)2 – 1.962 · σ2ΔE ≥ 0
 (ΔE/σΔE )2 ≥ 1.962
 the incremental effect is statistically significant

   When the incremental effect is not statistically
significant, it is not interesting clinical-wise to conduct cost-
effectiveness analysis. Statistically, it is appropriate to run analysis
using ICER and make inferences based on it

35
A simulation study
   3 distribution families
   (1) Both effect and cost follow a normal distribution
   (2) Both effect and cost follow a log-normal distribution
(right skewed)
   (3) Effect  normal; Cost  log-normal

   For each distribution family, 162 different distributions:
   Sample size: 50, 100, 400
   Correlation coefficient between cost and effect: -0.5, 0.1, 0.5
   Ratio of the variances (control / intervention): 1, 3
   ICER: 2000, 10000, 50000
   3 distances between the ICER and the origin
(relevant to the effect size)

36
Distance between ICER and origin
ΔC
ΔEB
B

ΔEA
A

ΔE

37
    A and B have the same ICER
Normal       Log-normal        Mixed
Method              Category         Distribution   Distribution   Distribution
Taylor                     Coverage %            92.7           92.5           92.5

Fieller – close interval   Coverage %            93.8           93.7           93.7

Fieller – all scenarios    Coverage %            95.0           94.7           94.7
Close interval      (85%)         (84.8%)        (84.8%)
Open interval      (33%)         (33.0%)        (31.8%)
Whole real line      (3%)          (2.6%)         (3.1%)

Bootstrap – percentile Coverage %                95.7           95.5           95.5

Bootstrap – normal         Coverage %            95.8           95.7           95.7

Bootstrap – t              Coverage %            83.9           83.7           83.8

Bootstrap - BCa1           Coverage %            92.4           92.2           92.2

Bootstrap - BCa2           Coverage %            92.3           92.0           92.0
Mixed Distribution
ICER                          50000                                    2000                                   10000
Distance                 1/4                  4                  1/4                  4                  1/4                  4
Method                   N            50           400      50        400     50           400     50         400     50           400     50         400
Taylor                     Coverage %          75.4         88.8    91.1       95.0    96.0         95.3   94.3        94.8    92.9         95.3   94.6        94.9

Fieller – close interval   Coverage %          72.5         95.0    96.9       95.0    94.9         95.0   94.0        94.9    95.7         94.9   94.6        94.9

Fieller – all scenarios    Coverage %          95.0         94.8    94.9       95.0    94.6         95.0   94.0        94.9    94.8         94.9   94.6        94.9
Close interval   11.2         50.7    78.0       100.0   92.5     100.0      100.0       100.0   71.3     100.0      100.0       100.0
Open interval    87.8         49.5    22.0               4.5                                     23.2
Whole real line   1.7                                     3.6                                     7.0

Bootstrap – percentile Coverage %              95.8         97.6    97.6       95.0    95.2         94.9   93.8        94.9    96.9         94.8   94.5        94.7

Bootstrap – normal         Coverage %          97.6         96.0    95.1       95.6    97.8         95.7   94.2        94.8    96.3         96.0   94.8        94.8

Bootstrap – t              Coverage %          48.4         65.6    73.1       92.8    84.0         94.0   94.1        94.9    76.0         92.3   94.2        94.8

Bootstrap - BCa1           Coverage %          66.9         89.8    95.6       94.8    95.1         94.8   93.8        94.6    94.5         94.7   94.5        94.5

Bootstrap - BCa2           Coverage %          65.9         89.7    95.5       94.8    94.7         94.7   93.6        94.7    93.8         94.6   94.3        94.6
Problem with ICER
   Quadrant I: positive intervention effect, more costly
   Quadrant III: negative intervention effect, cost saving

   Quadrant II: negative intervention effect, more costly
   Quadrant IV: positive intervention effect, cost saving

   Negative ICERs are difficult to interpret

40
Problem with ICER – cont.
   Example: 3 studies comparing to the same control group
   A: ΔE = 10, ΔC = -1000
   B: ΔE = 10, ΔC = -500
   C: ΔE = 5, ΔC = -500

   Pair-wise comparison:
 A is better than B (same effect, more cost-saving)
 B is better than C (same cost-saving, more effective)
A>B>C

   ICER:
 (A) ICER = -100; (B) ICER = -50; (C) ICER = -100
A=C>B
41
Summary
   ICER is an intuitive measure for cost-effectiveness analysis
but is only appropriate when
   The incremental effect is statistically significant from 0
   Both incremental effect and incremental cost are positive

   The available methods might not be appropriate for other
scenarios

   With large samples, normal method, bootstrap-percentile
method, and Fieller’s method are all equally good

   For small samples (or extremely skewed costs), my
preference is (1) Fieller’s method (2) bootstrap-percentile (3)
normal method

42
2008 Last Wednesday Health Services Methodology Seminar Series
DATE          LECTURER                                             TOPIC                                 LECTURE ROOM

Jan. 30      Ming-Yu Fan        Introduction to Cost-effectiveness Analysis                                  HSB T-530

Feb. 27      Joan Russo         Mediator and Moderator Analyses in Psychiatric Health Services Research       BB 1640

Mar. 26      Marcia A. Ciol     Sample Size Calculation                                                       BB 1640

April 30     Ming-Yu Fan        Longitudinal Data Analysis and Survival Analysis                              BB 1640

May 28       Marcia A. Ciol     Using Weights in Analysis of Multi-stage Surveys                                TBA

June 25      Ming-Yu Fan        Missing Data Analysis – Multiple Imputation                                   BB 1640

July & Aug                                                             No Seminar

Sept. 24     Jutta M. Joesch    Analyses of Employment as Outcomes                                            BB 1640

Oct. 29      Joan Russo         ROC Curves and Diagnostic Statistics for Health Services Research             BB 1640

Nov. 19*     Jutta M. Joesch    Propensity Score Matching                                                     BB 1640

Dec                                                                No Seminar

* November 26 is the day before Thanksgiving. The seminar is moved it to 11/19.
Lecturers:

Marcia A. Ciol, Ph.D. in Biostatistics, Research Associate Professor, Rehabilitation Medicine, marciac@u.washington.edu

Ming-Yu Fan, Ph.D. in Biostatistics, Research Assistant Professor, Psychiatry and Behavioral Sciences, myfan@u.washington.edu

Jutta M. Joesch, Ph.D. in Economics, Research Associate Professor, Psychiatry and Behavioral Sciences, joesch@u.washington.edu

Joan Russo, Ph.D. in Psychology, Associate Professor, Psychiatry and Behavioral Sciences, jerusso@u.washington.edu

```
Related docs