Document Sample

Interim Analyses with Multiple Primary Endpoints Application to an HIV Vaccine Trial Devan V. Mehrotra* and Xiaoming Li Merck Research Laboratories * devan_mehrotra@merck.com PhRMA Adaptive Trials Lecture Series June 13, 2008 Outline • Motivation (CMI-based vaccines) • POC efficacy trial design • Interim analysis strategies • Implementation and results • Concluding remarks 2 Motivation • Vaccines for HIV, malaria, TB, etc., are being specifically designed to elicit cell mediated immune (CMI) responses: T cells that can detect and kill pathogen-infected cells. • In concept, vaccine-induced CMI responses may: Prevent persistent clinical infection and/or Reduce the pathogen load in those who become infected despite vaccination, thereby slowing or stopping disease progression. 3 Cell Mediated Immunity: Mechanism Source: Mims CA, Playfair JHL, Roitt IM, et al (1993). "Medical Microbiology”, page 6.9, Mosby, London. 4 CMI-Based Vaccines: Efficacy Evaluation Focus on HIV Vaccines CMI-based vaccines: new, large uncertainty prudent to conduct a focused test-of-concept (TOC) efficacy trial with a lower but clinically relevant hurdle than a traditional phase III trial. Three key trial design challenges: 1. Primary efficacy endpoint/hypothesis 2. Statistical method for establishing efficacy 3. Accelerating GO/NO GO to phase III 5 Primary Efficacy Endpoint/Hypothesis For an antibody-based vaccine, a phase III efficacy trial would use infection (INF) as the primary endpoint. Example: Hnull: VEINF > 30% vs. Halt: >30% [Test of super efficacy] infection rate for VACCINE where VEINF 1 infection rate for PLACEBO If true VEINF 50%, for 90% power, 1-tailed = 2.5%, 400 infections would be needed to establish efficacy. For a CMI-based vaccine, infection and viral load (VL) should be dual primary endpoints in a TOC trial. 6 POC Efficacy Trial (continued) Data Set-Up Vaccine Placebo No. randomized Nv Np No. infected (HIV+) nv np Proportion infected pv nv / N v p p n p / N p y1( v ) y1( p ) Viral load set-point of infected subjects (log10 copies/ml) ynvv ) ( yn p ) ( p 7 Primary Efficacy Endpoint/Hypothesis (cont’d) • Appropriate hypotheses for a TOC trial: H1:VEINF = 0% and H2:VL = 0 versus VEINF > 0% and/or VL > 0 [Test for any efficacy] VL = true difference in mean viral load among infected subjects (placebo – vaccine) • Test of concept is successful if H1 or H2 can be rejected with joint confidence of ≥ 95%. • What is an efficient statistical method for testing the CMI concept? 8 #2. Statistical Method for Efficacy Evaluation • Single analysis of a composite BOI endpoint [Method A] > Popular method (Chang, Guess, Heyse; 1994) > Define burden-of-illness (BOI) = viral load for infected subject and zero for uninfected subject. > Use a single statistical test to compare the BOI per randomized subject between vaccine and placebo groups. • Separate analyses of INF and PL endpoints [Method B] > Use a separate statistical test for the infection and viral load endpoints. > Pay a statistical cost for getting two chances to establish vaccine efficacy (multiplicity adjustment to keep the false positive risk ≤ 5%). 9 Pathogen Load Comparison: Selection Bias • Problem The viral load comparison in Method B is restricted to subjects that are selected based on a post- randomization outcome (infection) potential for selection bias due to imbalanced covariates. • Solution - Adjust the viral load comparison for plausible levels of selection bias. [Details omitted] - If the adjusted test is statistically significant, then vaccine effect on viral load is credible. References Gilbert, Bosch, Hudgens (Biometrics, 2003) Mehrotra, Li, Gilbert (Biometrics, 2006) Shepherd, Gilbert, Mehrotra (The American Statistician, 2007) 10 No. of Events (Infections) Needed for TOC Trial =5%, 80% power VL VEINF Unadjusted Adjusted* copies/mL Method B Method B Method A 1.0 0% 28 30 > 250 1.0 30% 27 34 74 0 60% 47 48 44 * adjusted for potential selection bias in viral load comparison; calculations assume pathogen load SD=0.8 Method B (separate analysis of each endpoint) is notably more efficient than method A (single analysis of BOI endpoint). ~ 30 infections required if VEINF ~ 0% and VL ~ 1.0 [more likely] ~ 50 infections required if VEINF ~ 60% and VL ~ 0 [less likely] 11 #3. Accelerating GO/NO GO for phase III Do interim analysis when viral load endpoint has 80% power HIV Vaccine Trial: at 30 HIV infections assuming VL=1.0 copies/ml 100 I: VEINF = 0% 90 II: VEINF = 60% 80 Total Number of HIV-1 Infections I 70 60 50 II 40 30 Enrollment |-------------------| 20 10 30 infections 50 infections 0 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 Time (months) Above example: N = 750 per arm, ~ 21 months enrollment, placebo infection rates of ~2% per year, and dropout rates of 10%, 5%, 5% for 1st, 2nd, 3rd year, respectively. 12 POC Efficacy Trial: Interim Analysis • Goal: To establish POC and advance to phase III as soon as possible. • Interim efficacy analysis after 30 HIV infections. • Final efficacy analysis at 50 HIV infections, if necessary. Potential Results and VIRAL LOAD (VL) Outcomes of the Strongly Strongly In-Between Interim Analysis Negative Positive I Strongly Phase III: GO N STOP Continue to 50 inf. Negative No more inf. needed F E Phase III: GO C In-Between Continue to 50 inf. Continue to 50 inf. Continue to 50 inf. T I Strongly Phase III: GO Phase III: GO Phase III: GO O Positive No more inf. needed [Continue to 50 inf?] No more inf. needed N VL = viral load, inf. = infection endpoint. Note: no multiplicity penalty is required at the final analysis if it includes only one endpoint. We considered four -spending interim analysis strategies. 13 Interim Analysis Strategy A Overall = 0.05 Interim Analysis (30 events) Final Analysis (50 events) “Futility” look only Test H1 and H2 No alpha spent = 0.05 (shared via the Hochberg adjustment) H1: no effect on infection endpoint, H2: no effect on viral load endpoint “Futility” considerations at interim analysis (30 events) If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE. Note: Cannot get an accelerated GO for phase III since no efficacy boundary has been formally crossed. (Could use conditional power, but the associated calculations are very complicated.) 14 Interim Analysis Strategy B Overall = 0.05 Interim Analysis (30 events) Final Analysis (50 events) Do not test H1 Test H1 Test H2 If p1 < 0.025 then reject H1, If p2 < 0.025 then reject H2, otherwise accept H1 otherwise accept H2 H1: no effect on infection endpoint, H2: no effect on viral load endpoint “Futility” considerations at interim analysis (30 events) If p1 > 0.5 then STOP (accept H1), otherwise CONTINUE. Note: This is a simple Bonferroni approach, but it provides only one chance to formally test the viral load endpoint! 15 Interim Analysis Strategy C Overall = 0.05 Interim Analysis (30 events) Final Analysis (50 events) Test H1 If both endpoints are in the If p1 < 0.00025 then reject H1 analysis, test H1 and H2, = otherwise continue 0.03322 shared via the Test H2 Hochberg adjustment IF p2 < 0.025 then reject H2, If only one endpoint is in the otherwise continue analysis, test it using = 0.03322 H1: no effect on infection endpoint, H2: no effect on viral load endpoint “Futility” considerations at interim analysis (30 events) If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE. Note: If a hypothesis is rejected at the interim, it is not tested again later; additional data is used descriptively only. 16 Interim Analysis Strategy D Overall = 0.05 Interim Analysis (30 events) Final Analysis (50 events) Test H1 and H2 If both endpoints are in the = 0.030 shared via the analysis, test H1 and H2, = Hochberg adjustment 0.03184 shared via the If both endpoints are Hochberg adjustment significant then STOP, If only one endpoint is in the otherwise CONTINUE analysis, test corresponding hypothesis using = 0.03184 H1: no effect on infection endpoint, H2: no effect on viral load endpoint “Futility” considerations at interim analysis (30 events) If p1 > 0.5 and p2 > 0.5 then STOP, otherwise CONTINUE. Note: If a hypothesis is rejected at the interim, it is not tested again later; additional data is used descriptively only. 17 Simulation Results Power (%) At Interim (30 events) Overall* (50 events) Scenario Endpoint A B C D A B C D VEINF=0% V. Load 0 83 83 77 96 83 96 95 δVL = 1.0 Infection 0 0 1 16 36 25 27 30 VEINF=30% V. Load 0 82 82 77 97 82 96 95 δVL = 1.0 Either 0 82 82 80 98 87 97 97 Infection 0 0 11 67 94 88 90 91 VEINF=60% V. Load 0 73 73 73 96 73 94 92 δVL = 1.0 Either 0 73 78 91 99 98 99 99 probability of rejection at either the interim analysis or the final analysis; 10K simulations Note: We chose strategy C based on the presumption that a CMI-based vaccine is more likely to reduce viral load rather than prevent infection. 18 Implementation and Results • Prototype TOC design used for 2 HIV vaccine trials: > STEP (Merck vaccine; Americas, Caribbean, Australia; Merck & NIH funded) > Phambili (Merck vaccine; S. Africa; NIH funded) • Interim results of STEP showed Merck vaccine was not efficacious based on per-protocol analysis (30 HIV infections: 19 vaccine, 11 placebo) NO-GO! • Vaccine failed, but TOC design hailed as a success for providing NO-GO in a resource-efficient manner. • Impact of STEP interim results: > Grounded ~ 12 HIV vaccine trials worldwide. > Recalibrated scientific discussion on utility of CMI-based HIV vaccines, use of viral vectors. 19 Concluding Remarks • In concept, our proposed interim analysis strategy can be used for any trial with 2 primary endpoints and/or 2 treatment groups (e.g., multiple doses versus a control). Benefits: accelerate programmatic decisions, cut costs, optimize label, etc., through optimal spending/allocation of alpha and beta. • Further statistical improvements are possible based on a recent “adaptive alpha allocation” method for multiple endpoints (Li and Mehrotra, Stats in Med, 2008, in press). • More research is needed in this area. 20 Appendix Theoretical derivations of the critical alphas for strategies C and D 21 Deriving the Critical Alphas for Strategy C Notation: H infI ) : the infection hypothesis is not rejected at the interim analysis ( H infI ) : the infection hypothesis is rejected at the interim analysis ( H infF ) : the infection hypothesis is not rejected at the final analysis ( H infF ) : the infection hypothesis is rejected at the final analysis ( H inf : the infection hypothesis is not rejected at either the interim or final analysis H inf : the infection hypothesis is rejected at either the interim or final analysis H vlI ) , H vlI ) , H vlF ) , H vlF ) , H vl, and H vl are defined similarly ( ( ( ( 1 (.) : inverse standard normal cumulative distribution function Overall Type I error rate is given by the following probability calculated under the null hypothesis: Pr( H inf H vl ) Pr( H inf ) Pr( H vl ) Pr( H inf H vl ) 22 Deriving the Critical Alphas for Strategy C (continued) We use N (0,1) as the approximate the null distribution for the test statistics for both endpoints. Let I Pr(H vl ) Pr(H vlI ) ) Pr(H vlI ) H vlF ) ), where Pr(H vlI ) ) vlI ) , and ( ( ( ( ( Pr( H vlI ) H vlF ) ) Pr( H vlI ) H infI ) H vlF ) ) Pr( H vlI ) H infI ) H vlF ) ) ( ( ( ( ( ( ( ( infI ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) )) Pr( H vlI ) H infI ) H vlF ) ), ( ( ( ( ( ( ( I Pr( H vl ) Pr( H vlI ) ) Pr( H vlI ) H vlF ) ) vlI ) inf) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) )) ( ( ( ( (I ( ( ( Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) )) Pr( Z inf) ( inf) ), Z inf ) ( ( F ) )) ( ( ( (I (I (F (F ) Pr( Z (I ) vl ( ), Z(I ) vl (F ) vl ( )) Pr( Z inf) ( inf) ), Z inf ) ( ( F ) )) (I (I (F 2 vlI ) inf) [ ( F ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))] ( (I ( ( ( [ ( F ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))] [ ( F ) Pr( Z inf) ( inf) ), Z inf ) ( ( F ) ))] ( ( ( (I (I (F (F ) (F ) [ Pr( Z ( ), Z ( (I ) vl (I ) vl (F ) vl ))] 2 2 [1 ( F ) inf) Pr( Z inf) ( inf) ), Z inf ) ( ( F ) ))]. (I (I (I (F 23 Deriving the Critical Alphas for Strategy C (continued) Similarly, II Pr( H inf ) Pr( H infI ) ) Pr( H inf ) H infF ) ) inf ) Pr( H inf ) H infF ) H vlI ) ) Pr( H inf ) H infF ) H vlI ) ) ( (I ( (I (I ( ( (I ( ( inf ) vlI ) Pr( Z inf ) ( inf ) ), Z inf ) ( ( F ) )) Pr( H inf ) H infF ) H vlI ) ) (I ( (I (I (F (I ( ( inf ) vlI ) [ ( F ) Pr( Z inf ) ( inf ) ), Z inf ) ( ( F ) ))] (I ( (I (I (F [ ( F ) Pr( Z inf ) ( inf ) ), Z inf ) ( ( F ) ))] [ ( F ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))] (I (I (F ( ( ( (F ) (F ) [ Pr( Z ( ), Z ( (I ) inf (I ) inf (F ) inf ))] 2 2 [1 ( F ) vlI ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))]. ( ( ( ( III Pr( H vl H inf ) Pr( H vlI ) , H infI ) ) Pr( H vlI ) , H inf ) , H infF ) ) Pr( H vlI ) , H vlF ) , H infI ) ) Pr( H vlI ) , H inf ) , H vlF ) , H infF ) ) ( ( ( (I ( ( ( ( ( (I ( ( vlI ) inf ) vlI ) [ ( F ) Pr( Z inf ) ( inf ) ), Z inf ) ( ( F ) ))] ( (I ( (I (I (F inf ) [ ( F ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))] (I ( ( ( [ ( F ) Pr( Z inf ) ( inf ) ), Z inf ) ( ( F ) ))] [ ( F ) Pr( Z vlI ) ( vlI ) ), Z vlF ) ( ( F ) ))]. (I (I (F ( ( ( The overall type I error rate is: Pr( H inf H vl ) Pr( H inf ) Pr( H vl ) Pr( H inf H vl ) = I + II – III (a function of inf ) , vlI ) , and ( F ) ) (I ( 24 Deriving the Critical Alphas for Strategy C (continued) Pr( Z vlI ) a, Z vlF ) b) above can be found using the well-known result: ( ( Z vlI ) 0 1 ~ N , ( Z (F ) 0 1 vl and Z inf ) 0 1 ~ N , (I Z (F ) 0 1 inf n where event at int erim . nevent at final Finally, to control the overall type I error rate at , for a chosen inf ) and vlI ) , we (I ( can calculate ( F ) using a simple search routine. Note: Similar derivation for strategy D. 25

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 0 |

posted: | 9/20/2012 |

language: | English |

pages: | 25 |

OTHER DOCS BY 2l0d49AD

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.