Sequential analysis in clinical trials and epidemiological studies[1]

Sequential analysis in clinical trials and epidemiological studies Ingeborg van der Tweel Centre for Biostatistics Utrecht University 25-11-2004 Sequential methods in statistics “ … the whole of life is sequential, for our future actions are conditioned to some extent by our past experience …” (Wetherill and Glazebrook, 1986) Outline Sequential analysis in • clinical trials • epidemiological studies • quality control • safety monitoring Clinical Trial recruitment and inclusion sequentially  response available sequentially  analysis sequentially Interim analysis after each new response or group of responses an interim analysis is performed  enough evidence to stop the trial or continue the trial  continuous sequential or group sequential analysis Why interim analyses? • Ethics: • Safety: superiority of a treatment inferiority of a treatment / toxicity of a treatment = costly therapy = no clinically relevant difference in effect between treatments • Economy: ICH E9 (International guideline): Statistical principles for clinical trials 4.5 “The goal of an interim analysis is to stop the trial early = if the superiority of the treatment under study is clearly established, = if the demonstration of a relevant treatment difference has become unlikely or = if unacceptable adverse effects are apparent.” Effect of interim analyses number of analyses 1 2 3 4 5 10 total significance level  0.05 0.08 0.11 0.13 0.14 0.19 Effect of interim analyses The more often one analyses the accumulating data, the greater the chance of eventually and wrongly detecting a difference, and thus drawing incorrect conclusions from the trial. Increase in overall significance level  decrease in nominal significance level  per interim analysis Critical Z values per interim analysis (overall =0.05) …. Pocock ---- Haybittle-Peto ___ O‟Brien-Fleming RST vs boundaries • Repeated Significance Tests (Pocock, OBF, HP) = fixed number and timing of the interim analyses • More flexible (alpha-spending function) design by Lan & DeMets • Most flexible is the boundaries approach (Wald, Whitehead) A. Wald (1947) • Sequential Probability Ratio Test (SPRT): likelihood of the results, given H1 true Ln  likelihood of the results, given H0 true  stop and reject H0 as soon as Ln  A  stop and accept H0 as soon as Ln  B A ~ (1-)/ B ~ / (1- )  continue as long as B < Ln < A A (hypothetical) example • Cross-over trial on treatment of hypertension • standard treatment S and new treatment N • probability of superiority of N over S:  = 0.7 • under H0:  = 0.5 • one-sided  =0.05 power 1-  =0.80 A (hypothetical) example nr of pat. nr of pref. for N 1 1 2 2 3 2 4 3 5 3 6 3 7 4 8 5 9 5 10 5 11 5 12 6 13 6 A (hypothetical) example after n=13 patients H0 can be accepted J. Whitehead (University of Reading, UK) • flexible boundaries approach: continuous and group sequential testing • • SPRT, TT, …. (one- and two-sided) outcome: dichotomous, Normal, survival, … • • H0:  = 0 vs H1:   R test statistics V (amount of information) and Z (effect size) The (hypothetical) example • Z = Y – n0 = observed – expected Y : number of preferences for N n : total number of patients with a preference 0: probability of a preference for N under H0 Z = Y – n/2 under H0 • V = n 0(1- 0) = n/4 under H0 • R = log(OR) = log{(0.7x0.5)/(0.3x0.5)} = 0.847 The (hypothetical) example: a (truncated) SPRT after n=13 patients H0 can be accepted The (hypothetical) example: a triangular test (TT) after n=14 patients H0 can be accepted Important  on average, less observations necessary to come to a decision with the same type I and type II errors  sample size is not fixed beforehand, but stochastic An example • Amyotrophic Lateral Sclerosis (ALS): – progressive weakness in skeletal muscles leading to death due to respiratory failure – 50% of the patients do not survive 3 years – no medical treatment available yet • Trial to compare placebo to dietary supplement: – possible positive effect on muscle strength ALS-example (continued) • death from any cause, tracheostomy and persistent assisted ventilation were considered events for survival analysis • cumulative survival after 16 months: – with placebo: 60% – with dietary supplement:  80% • one-sided =0.05 power 1-=0.90 fixed total number of events  56 fixed total number of patients  190 ALS-example (continued) • Fixed sample analysis can not take place before all patients are included, followed-up and analyzed. • Total duration = duration of the inclusion period + 16 months after inclusion of the last patient ALS-example (continued) Ethical considerations: • minimize the burden for ALS-patients • total trial duration as short as possible • number of included patients as small as possible Enough evidence? STOP ALS-example (continued) • Hazard Ratio = log(0.8) / log(0.6) = 0.437 R = -log(HR) = 0.828 • Z = observed number of events in the control group – expected number of events under H0 • V  (number of events)/4 • Z²/V = logrank test statistic (1:1 randomisation) ALS-example (continued) ALS-example (continued) • Lower boundary was crossed after 34 events in 164 included patients  accept H0 • Eventually 175 patients included: placebo: 30 / 87 died supplement: 28 / 88 died • Adjusted HR = 0.78 (95% CI: 0.47; 1.48)  No real gain in number of patients, but in follow-up time Epidemiological studies Why a sequential analysis? • efficiency and economical aspects: = quality control = savings in money, time, biological samples, ... = screening of new hypotheses Epidemiological studies • the DOM project: a breast cancer screening programme in Utrecht and surroundings • biological material stored in a biobank • regional cancer registration  cases • large number of interesting hypotheses vs limited number and amount of biological samples  efficient statistical methods needed data OL vd Hel et al (1998) 1:1 matching 1:3 matching cohort-nested case-control study relating exposure to the MTHFR-gene to the occurrence of rectal cancer (2=0.05, power 1-=0.80, R=0.5) data OL vd Hel et al (2002) non-hierarchical GxE-interaction, breast cancer P(G)=0.30, P(E)=0.30, 2=0.05, 1-=0.80, OR=2 Nfixed = 308, Nseq = 5x44 = 220  29% gain data OL vd Hel et al (2002) hierarchical GxE-interaction breast cancer P(G)=0.30, P(E)=0.30, 2=0.05, 1-=0.80, OR=2 Nfixed = 674, Nseq = 9x44 = 396  41% gain Quality control  MR de Leval et al. „Analysis of a cluster of surgical failures‟ J Thorac Cardiovasc Surg (1994)107:914-924  104 neonatal arterial switch operations for transposition of the great arteries  1 † in the first 52 patients  then nrs 53, 55, 59, 63, 64, 67, 68 †  H0: p=0.02 versus H1: p=0.05  = 0.05, power = 0.80 Quality control (MR de Leval et al (1994)) 2 1 0 -1 -2 -3 ALARM -4 (alpha=0.05) ALERT (alpha=0.20) -5 -6 50 52 54 56 58 60 62 64 66 68 70 number of patients Safety monitoring • Example: = RCT in head injury = active drug vs placebo = evaluation of functional status after 6 months = safety monitoring of mortality within 21 days = placebo 21-day mortality rate 0.35 If exp. 21-day mortality rate is larger than 0.52, it is undesirable to continue the trial Safety monitoring cum. nr of patients active placebo died entered died entered 0 16 4 14 1 33 7 27 4 49 12 44 6 64 14 58 8 76 17 77 14 118 26 117 18 139 27 134 20 153 28 150 29 190 38 184 33 230 43 223 date of interim 07/02/1994 27/04/1994 09/05/1994 26/08/1994 30/11/1994 05/05/1995 28/06/1995 04/08/1995 21/11/1995 17/05/1996 Safety monitoring Newer statistical methods W.G. Cochran (1959): “In the sequential trial, at the beginning the doctor is forced to make decisions about the desired sensitivity of the trial which he can dodge in the fixed-size trial. But to make an intelligent choice of the number of patients in a fixed-size trial, the same questions, or similar ones must be answered. When sequential trials are better explained and better understood, they should not be any harder than fixedsize trials.” In Quantitative methods in human pharmacology and therapeutics, ed D.R. Laurence, pp 119-143, London: Pergamon. Conclusion Sequential design and analysis of clinical trials and epidemiological studies could be considered more often for • ethical reasons: = superiority or inferiority of a treatment = toxicity • efficiency and economical reasons: = quality control = savings in money, time, biological samples, ... = screening of new hypotheses PEST 4.4 (2004) Some literature • Lewis JA. Statistical principles for clinical trials (ICH E9). An introductory note on an international guideline. Stat in Med 1999 (18) 1903-1942 Whitehead J. The design and analysis of sequential clinical trials. Chichester: Wiley 1997 (rev. 2nd ed.) PEST 4.4 Operating Manual. MPS Research Unit. Reading: the University of Reading 2004 Wetherill GB, Glazebrook KD. Sequential methods in statistics, 3d ed. London: Chapman and Hall, 1986. • • • Some literature • Todd S, Whitehead A, Stallard N, Whitehead J. Interim analyses and sequential designs in phase III studies. Br J Clin Pharmacol 2001 (51) 394-399 Van der Tweel I, Schipper M. Sequentiële analysen in klinisch en epidemiologisch onderzoek. NTvG 2002 (146) 2348-2352 (in Dutch) Groeneveld GJ, Veldink JH, Van der Tweel I, …, van den Berg LH. A randomized sequential trial of creatine in amyotrophic lateral sclerosis. Ann Neurol 2003 (53) 437-445 Van der Tweel I. Application and efficiency of sequential tests in matched casecontrol studies. PhD thesis. Utrecht University. March 2nd, 2004 Baksh M.F. Sequential tests of association with applications in genetic epidemiology. PhD thesis. University of Reading. 2002 • • • • Some literature • Baksh MF, Todd S, Whitehead J, Lucini MM. Design considerations in the sequential analysis of matched case-control data. Stat in Med (2004) to appear Van der Tweel I, Schipper M. Sequential tests for gene-environment interactions in matched case-control studies. Stat in Med (2004) to appear •