No Slide Title

Document Sample
No Slide Title Powered By Docstoc
					Interim Analyses with Multiple
      Primary Endpoints
  Application to an HIV Vaccine Trial

  Devan V. Mehrotra* and Xiaoming Li
    Merck Research Laboratories
    * devan_mehrotra@merck.com

 PhRMA Adaptive Trials Lecture Series
          June 13, 2008
                       Outline

•   Motivation (CMI-based vaccines)
•   POC efficacy trial design
•   Interim analysis strategies
•   Implementation and results
•   Concluding remarks




                                      2
                    Motivation
• Vaccines for HIV, malaria, TB, etc., are being
  specifically designed to elicit cell mediated immune
  (CMI) responses: T cells that can detect and kill
  pathogen-infected cells.
• In concept, vaccine-induced CMI responses may:
  Prevent persistent clinical infection
                 and/or
  Reduce the pathogen load in those who become
  infected despite vaccination, thereby slowing or
  stopping disease progression.


                                                     3
              Cell Mediated Immunity: Mechanism
Source: Mims CA, Playfair JHL, Roitt IM, et al (1993). "Medical Microbiology”, page 6.9, Mosby, London.




                                                                                                      4
CMI-Based Vaccines: Efficacy Evaluation
              Focus on HIV Vaccines

CMI-based vaccines: new, large uncertainty 
prudent to conduct a focused test-of-concept (TOC)
efficacy trial with a lower but clinically relevant
hurdle than a traditional phase III trial.

Three key trial design challenges:
1. Primary efficacy endpoint/hypothesis
2. Statistical method for establishing efficacy
3. Accelerating GO/NO GO to phase III

                                                  5
     Primary Efficacy Endpoint/Hypothesis

 For an antibody-based vaccine, a phase III efficacy
  trial would use infection (INF) as the primary endpoint.

 Example: Hnull: VEINF > 30% vs. Halt: >30%   [Test of super efficacy]


                    infection rate for VACCINE
 where VEINF    1
                    infection rate for PLACEBO
 If true VEINF  50%, for 90% power, 1-tailed  = 2.5%,
 400 infections would be needed to establish efficacy.

 For a CMI-based vaccine, infection and viral load (VL)
  should be dual primary endpoints in a TOC trial.

                                                                         6
            POC Efficacy Trial (continued)
                    Data Set-Up
                           Vaccine         Placebo

   No. randomized              Nv             Np
 No. infected (HIV+)           nv              np

 Proportion infected      pv  nv / N v p p  n p / N p

                             y1( v )       y1( p ) 
Viral load set-point of
                                                   
  infected subjects                        
   (log10 copies/ml)         ynvv ) 
                                (
                                             yn p ) 
                                                (
                                           p 
                                                          7
        Primary Efficacy Endpoint/Hypothesis (cont’d)

• Appropriate hypotheses for a TOC trial:
  H1:VEINF = 0% and H2:VL = 0
              versus
  VEINF > 0% and/or VL > 0   [Test for any efficacy]


  VL = true difference in mean viral load among
          infected subjects (placebo – vaccine)
• Test of concept is successful if H1 or H2 can be
  rejected with joint confidence of ≥ 95%.
• What is an efficient statistical method for testing
  the CMI concept?
                                                        8
    #2. Statistical Method for Efficacy Evaluation

• Single analysis of a composite BOI endpoint [Method A]
  > Popular method (Chang, Guess, Heyse; 1994)
  > Define burden-of-illness (BOI) = viral load for
    infected subject and zero for uninfected subject.
  > Use a single statistical test to compare the BOI
    per randomized subject between vaccine and
    placebo groups.
• Separate analyses of INF and PL endpoints [Method B]
  > Use a separate statistical test for the infection
    and viral load endpoints.
  > Pay a statistical cost for getting two chances to
    establish vaccine efficacy (multiplicity adjustment
    to keep the false positive risk ≤ 5%).
                                                      9
     Pathogen Load Comparison: Selection Bias

• Problem
  The viral load comparison in Method B is restricted
  to subjects that are selected based on a post-
  randomization outcome (infection)  potential for
  selection bias due to imbalanced covariates.
• Solution
  - Adjust the viral load comparison for plausible
    levels of selection bias. [Details omitted]
  - If the adjusted test is statistically significant,
    then vaccine effect on viral load is credible.
    References
    Gilbert, Bosch, Hudgens (Biometrics, 2003)
    Mehrotra, Li, Gilbert (Biometrics, 2006)
    Shepherd, Gilbert, Mehrotra (The American Statistician, 2007)
                                                                10
No. of Events (Infections) Needed for TOC Trial
                                                 =5%, 80% power


           VL               VEINF          Unadjusted                  Adjusted*
     copies/mL                               Method B                   Method B                   Method A
        1.0                    0%               28                         30                       > 250
        1.0                   30%               27                         34                         74
         0                    60%               47                         48                         44
     * adjusted for potential selection bias in viral load comparison; calculations assume pathogen load SD=0.8



Method B (separate analysis of each endpoint) is notably more
efficient than method A (single analysis of BOI endpoint).


~ 30 infections required if VEINF ~ 0% and VL ~ 1.0 [more likely]
~ 50 infections required if VEINF ~ 60% and VL ~ 0 [less likely]




                                                                                                                  11
                                                 #3. Accelerating GO/NO GO for phase III
                                          Do interim analysis when viral load endpoint has 80% power
                                           HIV Vaccine Trial: at 30 HIV infections assuming VL=1.0 copies/ml
                                   100
                                                           I: VEINF = 0%
                                   90                      II: VEINF = 60%

                                   80
Total Number of HIV-1 Infections




                                                                                                                                                  I
                                   70

                                   60

                                   50                                                                                                             II
                                   40

                                   30         Enrollment
                                              |-------------------|
                                   20

                                   10                                                                                      30 infections
                                                                                                                           50 infections
                                    0

                                             0    3    6    9    12   15     18   21   24   27 30 33 36 39 42 45 48 51 54 57 60
                                                                                          Time (months)
                                        Above example: N = 750 per arm, ~ 21 months enrollment, placebo infection rates of ~2% per year, and
                                        dropout rates of 10%, 5%, 5% for 1st, 2nd, 3rd year, respectively.                                   12
                        POC Efficacy Trial: Interim Analysis
• Goal: To establish POC and advance to phase III as soon as possible.
• Interim efficacy analysis after 30 HIV infections.
• Final efficacy analysis at 50 HIV infections, if necessary.
  Potential Results and                                                   VIRAL LOAD (VL)
    Outcomes of the                              Strongly                                                            Strongly
                                                                                In-Between
    Interim Analysis                             Negative                                                            Positive
    I
            Strongly                                                                                            Phase III: GO
   N                                                STOP                     Continue to 50 inf.
            Negative                                                                                          No more inf. needed
   F
   E
                                                                                                                Phase III: GO
   C     In-Between                        Continue to 50 inf.               Continue to 50 inf.
                                                                                                               Continue to 50 inf.
   T
    I
            Strongly                        Phase III: GO                     Phase III: GO                     Phase III: GO
   O
            Positive                      No more inf. needed              [Continue to 50 inf?]              No more inf. needed
   N
VL = viral load, inf. = infection endpoint. Note: no multiplicity penalty is required at the final analysis if it includes only one endpoint.


      We considered four -spending interim analysis strategies. 13
                         Interim Analysis Strategy A
                                      Overall  = 0.05

Interim Analysis (30 events)   Final Analysis (50 events)
 “Futility” look only        Test H1 and H2
 No alpha spent               = 0.05 (shared via the
                               Hochberg adjustment)
H1: no effect on infection endpoint, H2: no effect on viral load endpoint


“Futility” considerations at interim analysis (30 events)
If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE.

Note: Cannot get an accelerated GO for phase III since no
efficacy boundary has been formally crossed. (Could use
conditional power, but the associated calculations are very
complicated.)

                                                                            14
                        Interim Analysis Strategy B
                                   Overall  = 0.05

 Interim Analysis (30 events)    Final Analysis (50 events)
 Do not test H1               Test H1
 Test H2                      If p1 < 0.025 then reject H1,
 If p2 < 0.025 then reject H2, otherwise accept H1
 otherwise accept H2
   H1: no effect on infection endpoint, H2: no effect on viral load endpoint



“Futility” considerations at interim analysis (30 events)
If p1 > 0.5 then STOP (accept H1), otherwise CONTINUE.

Note: This is a simple Bonferroni approach, but it provides
only one chance to formally test the viral load endpoint!
                                                                               15
                       Interim Analysis Strategy C
                                   Overall  = 0.05
  Interim Analysis (30 events)    Final Analysis (50 events)
 Test H1                       If both endpoints are in the
 If p1 < 0.00025 then reject H1 analysis, test H1 and H2,  =
 otherwise continue              0.03322 shared via the
 Test H2                        Hochberg adjustment
 IF p2 < 0.025 then reject H2,  If only one endpoint is in the
 otherwise continue              analysis, test it using  =
                                 0.03322
   H1: no effect on infection endpoint, H2: no effect on viral load endpoint

“Futility” considerations at interim analysis (30 events)
If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE.
Note: If a hypothesis is rejected at the interim, it is not tested
again later; additional data is used descriptively only.
                                                                               16
                       Interim Analysis Strategy D
                                    Overall  = 0.05

Interim Analysis (30 events)    Final Analysis (50 events)
 Test H1 and H2              If both endpoints are in the
  = 0.030 shared via the     analysis, test H1 and H2,  =
  Hochberg adjustment          0.03184 shared via the
 If both endpoints are        Hochberg adjustment
  significant then STOP,      If only one endpoint is in the
  otherwise CONTINUE           analysis, test corresponding
                               hypothesis using  = 0.03184
   H1: no effect on infection endpoint, H2: no effect on viral load endpoint

“Futility” considerations at interim analysis (30 events)
If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE.
Note: If a hypothesis is rejected at the interim, it is not
tested again later; additional data is used descriptively only.
                                                                               17
                                Simulation Results
                                        Power (%)
                               At Interim (30 events)             Overall* (50 events)
  Scenario      Endpoint
                               A      B     C      D            A      B       C     D
 VEINF=0%
                 V. Load       0      83       83      77      96      83       96      95
  δVL = 1.0
            Infection          0      0        1       16      36      25       27      30
 VEINF=30%
             V. Load           0      82       82      77      97      82       96      95
  δVL = 1.0
             Either            0      82       82      80      98      87       97      97
            Infection          0      0        11      67      94      88       90      91
 VEINF=60%
             V. Load           0      73       73      73      96      73       94      92
  δVL = 1.0
             Either            0      73       78      91      99      98       99      99
 probability of rejection at either the interim analysis or the final analysis; 10K simulations


 Note: We chose strategy C based on the presumption that a
 CMI-based vaccine is more likely to reduce viral load rather
 than prevent infection.
                                                                                       18
                Implementation and Results
• Prototype TOC design used for 2 HIV vaccine trials:
  > STEP (Merck vaccine; Americas, Caribbean, Australia; Merck & NIH funded)
  > Phambili (Merck vaccine; S. Africa; NIH funded)
• Interim results of STEP showed Merck vaccine was
  not efficacious based on per-protocol analysis (30
  HIV infections: 19 vaccine, 11 placebo)  NO-GO!
• Vaccine failed, but TOC design hailed as a success
  for providing NO-GO in a resource-efficient manner.
• Impact of STEP interim results:
  > Grounded ~ 12 HIV vaccine trials worldwide.
  > Recalibrated scientific discussion on utility of
    CMI-based HIV vaccines, use of viral vectors.

                                                                          19
              Concluding Remarks
• In concept, our proposed interim analysis strategy
  can be used for any trial with  2 primary
  endpoints and/or  2 treatment groups (e.g.,
  multiple doses versus a control).
  Benefits: accelerate programmatic decisions, cut
  costs, optimize label, etc., through optimal
  spending/allocation of alpha and beta.
• Further statistical improvements are possible
  based on a recent “adaptive alpha allocation”
  method for multiple endpoints (Li and Mehrotra,
  Stats in Med, 2008, in press).
• More research is needed in this area.
                                                     20
                Appendix
Theoretical derivations of the critical alphas for
              strategies C and D




                                                     21
         Deriving the Critical Alphas for Strategy C

Notation:
   H infI ) : the infection hypothesis is not rejected at the interim analysis
      (


   H infI ) : the infection hypothesis is rejected at the interim analysis
      (


   H infF ) : the infection hypothesis is not rejected at the final analysis
      (


   H infF ) : the infection hypothesis is rejected at the final analysis
      (


   H inf : the infection hypothesis is not rejected at either the interim
               or final analysis
   H inf : the infection hypothesis is rejected at either the interim or
               final analysis
   H vlI ) , H vlI ) , H vlF ) , H vlF ) , H vl, and H vl are defined similarly
      (         (        (         (


    1 (.) : inverse standard normal cumulative distribution function

Overall Type I error rate is given by the following probability
calculated under the null hypothesis:
   Pr( H inf  H vl )  Pr( H inf )  Pr( H vl )  Pr( H inf  H vl )
                                                                         22
       Deriving the Critical Alphas for Strategy C (continued)

We use N (0,1) as the approximate the null distribution for the test
statistics for both endpoints.

Let I  Pr(H vl )  Pr(H vlI ) )  Pr(H vlI )  H vlF ) ), where Pr(H vlI ) )   vlI ) , and
                                  (               (           (                         (         (


Pr( H vlI )  H vlF ) )  Pr( H vlI )  H infI )  H vlF ) )  Pr( H vlI )  H infI )  H vlF ) )
      (         (               (          (         (               (          (         (



                                    infI )  Pr( Z vlI )  ( vlI ) ), Z vlF )  ( ( F ) ))  Pr( H vlI )  H infI )  H vlF ) ),
                                       (             (           (          (                            (          (         (




I  Pr( H vl )  Pr( H vlI ) )  Pr( H vlI )  H vlF ) )   vlI )   inf)  Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))
                       (               (         (           (          (I          (            (          (



                       Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))  Pr( Z inf)   ( inf) ), Z inf )   ( ( F ) ))
                              (            (          (                              (I          (I        (F



                                                                       (F )
                       Pr( Z     (I )
                                  vl       ( ), Z(I )
                                                    vl
                                                           (F )
                                                           vl      (         ))  Pr( Z inf)   ( inf) ), Z inf )   ( ( F ) ))
                                                                                           (I          (I        (F

                                                                         2
                    vlI )   inf)  [ ( F )  Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))]
                      (          (I                     (            (          (



                       [ ( F )  Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))]  [ ( F )  Pr( Z inf)   ( inf) ), Z inf )   ( ( F ) ))]
                                         (            (          (                                          (I          (I        (F



                          (F )                                                     (F )
                      [        Pr( Z   ( ), Z   (
                                             (I )
                                             vl
                                                             (I )
                                                             vl
                                                                     (F )
                                                                     vl          ))]
                           2                                                 2
                          [1   ( F )   inf)  Pr( Z inf)   ( inf) ), Z inf )   ( ( F ) ))].
                                             (I           (I          (I        (F




                                                                                                                                                    23
            Deriving the Critical Alphas for Strategy C (continued)

Similarly,
II  Pr( H inf )  Pr( H infI ) )  Pr( H inf )  H infF ) )   inf )  Pr( H inf )  H infF )  H vlI ) )  Pr( H inf )  H infF )  H vlI ) )
                          (                (I        (            (I            (I        (         (                (I        (         (



                      inf )   vlI )  Pr( Z inf )   ( inf ) ), Z inf )   ( ( F ) ))  Pr( H inf )  H infF )  H vlI ) )
                         (I       (              (I           (I         (F                            (I        (         (



                      inf )   vlI )  [ ( F )  Pr( Z inf )   ( inf ) ), Z inf )   ( ( F ) ))]
                         (I       (                         (I           (I         (F



                         [ ( F )  Pr( Z inf )   ( inf ) ), Z inf )   ( ( F ) ))]  [ ( F )  Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))]
                                            (I           (I         (F                                       (            (          (



                             (F )                                                   (F )
                        [         Pr( Z   ( ), Z   (
                                                (I )
                                               inf
                                                               (I )
                                                              inf
                                                                        (F )
                                                                       inf             ))]
                              2                                                    2
                             [1   ( F )   vlI )  Pr( Z vlI )   ( vlI ) ), Z vlF )   ( ( F ) ))].
                                               (             (            (          (




III  Pr( H vl  H inf )  Pr( H vlI ) , H infI ) )  Pr( H vlI ) , H inf ) , H infF ) )  Pr( H vlI ) , H vlF ) , H infI ) )  Pr( H vlI ) , H inf ) , H vlF ) , H infF ) )
                                 (          (               (          (I        (               (         (          (               (          (I       (          (



          vlI )   inf )   vlI )  [ ( F )  Pr( Z inf )  ( inf ) ), Z inf )  ( ( F ) ))]
            (          (I       (                         (I          (I         (F



              inf )  [ ( F )  Pr( Z vlI )  ( vlI ) ), Z vlF )  ( ( F ) ))]
                 (I                      (           (          (



             [ ( F )  Pr( Z inf )  ( inf ) ), Z inf )  ( ( F ) ))]  [ ( F )  Pr( Z vlI )  ( vlI ) ), Z vlF )  ( ( F ) ))].
                                (I          (I         (F                                      (           (          (




The overall type I error rate is:
           Pr( H inf  H vl )  Pr( H inf )  Pr( H vl )  Pr( H inf  H vl )
                          = I + II – III
                            (a function of  inf ) ,  vlI ) , and  ( F ) )
                                              (I       (
                                                                                                                                                                  24
       Deriving the Critical Alphas for Strategy C (continued)

Pr( Z vlI )  a, Z vlF )  b) above can be found using the well-known result:
      (            (




               Z vlI )       0 1   
                         ~ N   , 
                  (
                                          
               Z (F )         0    1 
               vl                    
              and
               Z inf )       0 1   
                         ~ N   , 
                   (I
                                                   
               Z (F )         0    1 
               inf                            
                                   n
              where   event at int erim .
                                    nevent at final
Finally, to control the overall type I error rate at , for a chosen  inf ) and  vlI ) , we
                                                                        (I         (


can calculate  ( F ) using a simple search routine.

Note: Similar derivation for strategy D.



                                                                                        25

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:9/20/2012
language:English
pages:25