Relationship between coffee and pancreatic cancer

Issues in Randomization Laura Lee Johnson, Ph.D. Statistician National Center for Complementary and Alternative Medicine Introduction to the Principles and Practice of Clinical Research Tuesday, October 18, 2005 Biostatistics • Randomization • Hypothesis Testing • Sample Size and Power • Survival Analysis Objectives • Intuitive understanding of the statistics used in clinical research • Understand and perform some simple but useful analyses & sample size calculations • Learn how to collaborate effectively with statisticians Objectives: Randomization Lecture • • • • Reasons for randomization Randomization theory and mechanisms Types of randomized study designs Compare randomized experimental studies to nonrandomzied observational studies • Nonrandomized experimental studies Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Statistical Software, Books, Articles Words I Might Use • • • • • Phase I, II, IIb, III, IV Trial Model: y = β0 + β1x1 + β2x2 + ε Covariate Effect Modifier Confounder Confounding • Two or more variables • Known or unknown to the researchers • Confounded when their effects on a common response variable or outcome are mixed together Confounding Example • Relationship between coffee and pancreatic cancer, BUT • Smoking is a known risk factor for pancreatic cancer • Smoking is associated with coffee drinking but it is not a result of coffee drinking. What is confounding? • If an association is observed between coffee drinking and pancreatic cancer – Coffee actually causes pancreatic cancer, or – The coffee drinking and pancreatic cancer association is the result of confounding by cigarette smoking. How to handle confounding • If you know something is a possible confounder, in the data analysis use – Stratification, or – Adjustment • Fear the unknown! Study Design Taxonomy • Treatment vs. Observational • Prospective vs. Retrospective • Longitudinal vs. Cross-sectional • Randomized vs. Non-Randomized • Blinded/Masked or Not – Single-blind, Double blind, Unblinded Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Stat Software, Books, Articles Randomization: Definition • Random Allocation – known chance receiving a treatment – cannot predict the treatment to be given • Eliminate Selection Bias • Similar Treatment Groups ONE Factor is Different • Randomization tries to ensure that ONE factor is different between two or more groups. • Observe the Consequences • Attribute Causality Types of Randomization • Standard ways: – Random number tables (see text) – Computer programs • NOT legitimate – Birth date – Last digit of the medical record number – Odd/even room number Types of Randomization • Simple • Blocked Randomization • Stratified Randomization Simple Randomization • Randomize each patient to a treatment with a known probability – Corresponds to flipping a coin • Could have imbalance in # / group or trends in group assignment • Could have different distributions of a trait like gender in the two arms Block Randomization • Insure the # of patients assigned to each treatment is not far out of balance • Variable block size – An additional layer of blindness • Different distributions of a trait like gender in the two arms possible Stratified Randomization • A priori certain factors likely important (e.g. Age, Gender) • Randomize so different levels of the factor are balanced between treatment groups • Cannot evaluate the stratification variable Stratified Randomization • For each subgroup or strata perform a separate block randomization • Common strata – Clinical center, Age, Gender • Stratification MUST be taken into account in the data analysis! When to Randomize? • When the treatment must change! • SWOG: 1 vs. 2 years of CMFVP adjuvant chemotherapy in axillary node-positive and estrogen receptor-negative patients. – JCO, Vol 11 No. 9 (Sept), 1993 Randomize at the Time Trial Arms Diverge • SWOG randomized at beginning of treatment • Discontinued treatment before relapse or death – 17% on 1 year arm – 59% on 2 year arm – Main reason was patient refusal Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Stat Software, Books, Articles Types of Randomized Studies • • • • • Parallel Group Sequential Trials Group Sequential trials Cross-over Factorial Designs Parallel Group • Randomize patients to one of k treatments • Response – Measure at end of study – Delta or % change from baseline – Repeated measures – Function of multiple measures Ideal Study - Gold Standard • Double blind • Randomized • Parallel groups Sequential Trials • Not for a fixed period • Terminates when – One treatment shows a clear superiority or – It is highly unlikely any important difference will be seen • Special statistical design methods Group Sequential Trials • Popular • Analyze data after certain proportions of results available • Early stopping – If one treatment clearly superior – Adverse events • Careful planning and statistical design Crossover Trial • E.g. 2 treatments: 2 period crossover • Use each patient as own control • Must eliminate carryover effects – Need sufficient washout period Factorial Design • Each level of a factor (treatment or condition) occurs with every level of every other factor • Selenomethionine and Celecoxib Gastroenterology 2002; 122:A71 Placebo Placebo Placebo Celecoxib Selenium Placebo Selenium Celecoxib Incomplete Factorial Trial • • • • Nutritional Intervention Trial (NIT) 4x4 incomplete factorial A,B,C,D Did not look at all possible interactions – Not of interest (at the time) – Sample size prohibitive Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Stat Software, Books, Articles Observational • Can ONLY show Association Experimental • Can show Association and Causality • Well done randomization means unknown confounders should not create problems • You will never know all the possible confounders! Observational Studies • Cohort Study – Follow a group for a while – Cardiovascular Health Study • Case-Control Study – Groups with or without outcome – Determine who was exposed to risk factor Observational Studies • Cross-sectional – Collect a representative sample – Simultaneously classify by outcome and risk factor Outcome Disease No disease Y Risk Factor N Observational Studies are Useful • May be only alternative – Smoking in humans – What happens in free living people (Cardiovascular Health Study) • May be cheaper and faster than a trial Do not always agree • HRT – Observational trials – WHI – Publication bias? Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Stat Software, Books, Articles Nonrandomized Experimental Studies • No control group – Early in investigation • Concurrent control “group” – Treatment assignment not by randomization • Historically controlled – Missing/poor data – Non-comparability of groups No placebo/control = problems • Patients tend to do better by receiving some treatment, even placebo or standard of care (soc) • Comparing a patient on treatment to baseline does not take this into account Additional Problems • Researchers tend to interpret findings in favor of the new treatment – Investigator bias • Impossible to distinguish the effect of time from treatment effects – Confounding Human Assumptions and Concurrent Control Groups • Newer = better • Systematic allocation is unreliable and many times NOT systematic – Bias – Manipulation • No randomization  impossible to establish if comparable groups Historical Control Study • Small patient pool – Pediatrics – Cancer research • Responses compared to controls from previous studies. • Only half the patients • No “placebo exposure” Historical Control Problems • Serious bias for assessing treatment efficacy • Controls not a good comparison group Historical Controls and Time • Treatments, technology, patient care changed over time • Patient population characteristics have changed over time Non-randomized Phase II design problems • • • • Placebo effect Investigator bias Unblinded treatment Regression to the mean – Natural reduction in disease activity over time Observational Studies • Why can observational studies only find a weaker degree of connection? – Subject to confounding – Can correct for what you know, but nothing to be done about the unknown • Sometimes it is unethical to do a randomized trial (e.g. smoking) Causation vs. Association • Causation – Established by experimental studies and clinical trials • Association – Observation studies can merely find association between a risk factor and an response Outline • • • • • • Introductory Statistical Definitions What is Randomization? Randomized Study Design Experimental vs. Observational Non-Randomized Study Design Stat Software, Books, Articles Statistical Resources • Software • Books • Articles, etc. Software • Most is expensive and some have yearly license fees – NIH (through CIT) many times has the software for free or cheaper than retail • Some is hard to use, some is easy Software: Programming Options • S-PLUS (Windows/UNIX): Strong academic and NIH following; extensible; comprehensive – www.insightful.com • R (Windows/Linux/UNIX/Mac): GNU; similar to S-PLUS – www.r-project.org – www.bioconductor.org S+ and R • Produce well-designed publicationquality plots • Code from C,C++, Fortran can be called • Active user communities Statistical Calculators • www.stat.ucla.edu – “Statistical Calculators” Other Software • STATA (Windows/Mac/UNIX) – Good for general computation, survival, diagnostic testing – Epi friendly – GUI/menu and command driven – Active user community – www.stata.com Other Software • SAS (Windows/UNIX) – Command driven – Difficult to use, but very good once you know how to use it – Many users on the East coast – www.sas.com • SPSS, EpiCure, many others Books • Statistical Rules of Thumb by Gerald van Belle • Epidemiology by Leon Gordis • The Statistical Evaluation of Medical Tests for Classification and Prediction by Margaret Sullivan Pepe More Books • Hosmer and Lemeshow books • Statistical Reasoning in Medicine: The Intuitive P-Value Primer by Lemuel Moye • Designing Clinical Research: An Epidemiologic Approach, edited by Stephen Hulley And More Books • Data Monitoring Committees in Clinical Trials: A Practical Perspective by Ellenberg, Fleming, DeMets. • Fundamentals of Clinical Trials by Friedman, Furberg, DeMets Articles • British Medical Journal: Statistics Notes http://www.sghms.ac.uk/depts/ phs/staff/jmb/pbstnote.htm • Statistics in Medicine • NEJM: Equivalence trials –October 16, 1997 Questions?