VIEWS: 33 PAGES: 42 POSTED ON: 5/15/2011 Public Domain
Clinical Investigation and Outcomes Research Statistical Issues in Designing Clinical Research Marcia A. Testa, MPH, PhD Department of Biostatistics Harvard School of Public Health 1 Objective of Presentation • Introduce statistical issues that are critical for designing a clinical research study and developing a research protocol, with a special focus on • Power and sample size – Readings: Textbook, Designing Clinical Research, Chapter 6, Estimating Sample Size and Power: Applications and Examples and Chapter 19, Writing and Funding a Research Proposal. 2 Research Proposal • Carefully planning the analytical and statistical methods is critical to any clinical research study. • An outline of the main elements of a research proposal are listed in Table 19.1 of your textbook. • Two very important components of the “Research Methods” section are “Measurements” and “Statistical Issues”. 3 Measurement and Statistical Components of the Research Proposal • Measurements – you first must define: – Main predictor/independent variables (intervention, if an experiment) – Potential confounding variables – Outcome/dependent variables • Statistical Issues – you should outline: – Approach to statistical analyses – Hypothesis, sample size and power 4 Power and Sample Size • Depends upon: – measurements and study hypotheses – statistical test used on primary outcome – study design – variability and precision of the dependent measure – alpha (type 1 error) – effect size – number of hypotheses that you want to test 5 Types of Errors Confidence 6 What is power analysis? • Statistical power: – the probability of correctly identifying a trend or effect (Being correct that there is a trend or effect) • Statistical confidence: – the probability of not identifying a false trend or effect (false alarm) (Being correct that there is no trend) 7 Why is power analysis useful in research planning? • Clinical research is primarily concerned with detecting improvements or worsening due to interventions or risk factors. • Power analysis answers the question: “How likely is my statistical test to detect important clinical effects given my research design?” 8 Elements of power analysis • Variability (stochastic noise in the data) Beyond our control • Sample Size (accumulated information) • time horizon (e.g.,survival analysis) – sampling frequency – replication Within our control – Confidence level/statistical test 9 Dealing with Variability • Variability is often a barrier to detection • Minimizing variability is often the goal • Choose variables with a high signal to noise ratio • Caution: these variables may be less sensitive to change • Sample within a more homogeneous population • Caution: greater homogeneity often means we are limiting the inferences we can make. At the extreme we would have highly reliable results that are for the most part clinically irrelevant 10 The Balancing of Cost and Power Power Curve high optimal use of resources C 100 B effective but Sample Power inefficient use of resources A low return on investment low 0 small large Sample Size Low Cost High Cost 11 Limitations of power analysis • Power analysis is only as good as the information you provide: – How appropriate is the statistical test? – How accurate are estimates of variability? • Power analysis can’t tell you: – How much power is enough? – What’s a meaningful change? 12 How much power is enough? • There is no universal standard • What is more important? • Not missing a trend? Power > Confidence • Reporting a false trend? Confidence > Power • Usual range for confidence and power: 80-95% 13 What’s a meaningful change? 100 100 Power = 95% for declines = -17% 80 80 Example: You want Power (%) 60 60 to be able to detect the withdrawal 40 40 (decline in participation) from a 20 20 diet and exercise program under “usual care”. -20 -15 -10 -5 0 Annual Rate of Decline (%) effect size 14 What’s a meaningful change? 100 100 80 80 Power = 80% for decline = -13% Power (%) 60 60 40 40 20 20 -20 -15 -10 -5 0 Annual Rate of Decline (%) effect size 15 What’s a meaningful change? 100 100 80 80 Power (%) 60 60 Power = 60% for decline = -10% 40 40 20 20 -20 -15 -10 -5 0 Annual Rate of Decline (%) effect size 16 Is a 17% annual withdrawal rate clinically meaningful? • Example – Start with 100 patients No. of Year individuals 1 100 17% withdrawal 2 83 after one year 3 69 4 57 After 5 years, more than 50% of your original 5 47 population has withdrawn for the program 17 What is a meaningful change? • Most people would concur that a withdrawal of 17% per year from a diet and exercise is large enough to be considered clinically meaningful. • However, how meaningful are smaller withdrawal rates (13%, 10%, 5% 1%) ? • This can not be answered using a formula. • The answer will depend on the research objectives and clinical objectives, and the research budget. 18 1. Chose Statistical Hypothesis • Set up Null Hypotheses: Examples 1. Compare sample group mean to a known value 0 – Mean of group = Known population mean (H0 : 0 ) vs (HA : 0 ) 2. Compare two sample group means – Mean Group (1) = Mean Group (2) (H0 : 1 2 ) (HA : 1 2 ) Note – because you are testing “not equal” in the alternative hypothesis () you have selected a “two-tailed test”. 19 2. Chose Statistical Test • There are many statistical tests that are used in clinical research, however, for this presentation we will restrict ourselves to the following: Outcome/Dependent Variable Predictor/Independent Variable Dichotomous Continuous Dichotomous Chi-squared test t-test Continuous t-test Correlation Coefficient 20 3. Chose Alpha Level and Effect Size • Alpha = 0.05 – probability of rejecting the null when the null is true = 5% – You will conclude that there was a difference 5% of the time when there really was no difference • You would like to detect a difference of X units or higher (effect size) in one group as compared to the other 21 4. Need SD of the Dependent Variable • Use historical data if available • Use the sample data from a feasibility study (e.g. 15 subjects) • If you have no data to serve as a reference, you have to make an educated guess. Here’s a trick if your data is mound shaped and approximately normal. – Choose a representative low and high from your clinical experience, take the difference and divide by 4. – = ((high) – (low))/4 = SD estimate 22 5. Calculate a Standard Effect Size • Effect size/standard deviation = standardized effect size • Choose the error – Remember Power = 1 - , so a type 2 error of 0.20 yields a power of 0.80 – Power is the probability of failure to reject the null hypothesis when the null hypothesis is false concluding no difference when there really is a difference. 23 Power and Sample Size Example Continuous Glucose Monitoring Diabetes Study 24 CGM Study Two-group Comparison • How many subjects do we need to be able to detect a difference in CGM mean daily glucose between patients on Lantus and Apidra insulin versus Premix analogue insulin? – Before you can answer this question, you must gather some more information. 25 Break down the problem • CGM glucose at Week 12 = dependent variable of interest • Want to compare two groups – each group has different patients • Simple independent t-test • Need SD of daily glucose • Need to specify how large an effect you want to detect 26 Data from feasibility study Week 12 Data 27 CGM Study Two-group Comparison • Compare Lantus & Apidra to Premix at 12 weeks • Feasibility data available on 15 patients • Independent t test will be used • Alpha = 0.05, beta = 0.20, 2-tailed test • Power = 0.80 – Null: Mean L & A = Mean Premix (H0 : 1 2 ) (HA : 1 2 ) 28 CGM Study Two-group Comparison • SD from 15 patient feasibility study = 33 29 Estimating Sample Size of CGM Study • Alpha = 0.05 for 1-sided, 0.025 for 2-sided test • Beta = 0.20, hence, power = 0.80 • Clinically meaningful effect = 10 mg/dL difference (based upon clinical judgement) • SD CGM glucose = 33 (from feasibility study) • Standardized effect = 10/33 = 0.30 • Check Appendix 6A in textbook for power • Table 6A says you need 176 subjects per treatment group for a total of 352 subjects. 30 http://www.epibiostat.ucsf.edu/biostat/sampsize.html This is a directory of where you can find sample size and power programs 31 Useful Power Calculator Website http://www.stat.uiowa.edu/~rlenth/Power/ 32 Online Power/Sample Size Power = 0.8, detect ES Power = 0.9, detect ES = 0.3 (10 mg/dL) = 0.35 (11.6 mg/dL) N = 175 per group N = 175 per group 33 Online Power/Sample Size Power = 0.8, detect ES = 0.5 Power = 0.8, detect ES = 1.57 (16.5 mg/dL) (52 mg/dL) 34 Sample size = 64/group Sample size = N1 = 7, N2 = 8 CGM Study Paired Comparison • Useful for longitudinal assessments • CGM Study – You want to detect a decrease between Week 12 and Week 24 of 10 mg/dL • You only have one group of patients, but they are measured on two separate occasions (Week 12 and Week 24). 35 15 patient feasibility study Wk 0 Wk 12 Wk 24 What is the mean glucose, parameter for the subjects at Week 12 versus Week 24? For simplicity, we are going to use the single value summary mean glucose levels at Wk 12 and Wk 24. 36 Power and Sample Size for Paired t-test Power = 0.8, detect ES = 0.30 Need 92 subjects or “pairs” (Wk 12 and Wk 24) data. Remember with two independent groups we needed 175 subjects per group for a total of 350 subjects. When patients serve as their own control, you need “fewer” subjects to detect an equivalent effect size (ES) with the same power. 37 HRV Study Correlation and Multiple Regression • Single-Group Study – Session 1 – Signal 1 HRV – Session 1 – Signal 2 BP – Demographic variables = Age, Gender – Clinical characteristics = Disease Status • Suppose you want to look at associations between HRV, BP, demographic and clinical characteristics -- use bivariate correlation coefficient for 2 variables of multiple regression R2 multiple predictors. 38 Power and Sample Size for Correlations (H0: r = 0) Power = 0.0.80, r = 0.3, ES = R2 Power = 0.97, r = 0.4, ES = R2 = = 0.09, Sample size = 85 0.16, Sample size = 85 Only 1 “regressor” or predictor 39 Power and Sample Size for Correlations (H0: r = 0) Power = 0.80, r = 0.3, ES = R2 = Power = 0.80, r = 0.3, ES = R2 = 0.09, Sample size = 139, if number 0.09, Sample size = 177, if number of ipredictor variables = 5 of predictor variables = 10 40 Power and Sample Size for Test of Two Proportions You want to detect a difference between two proportions. Example: How many patients do you need in each group to detect a difference in the numbers of patients who adhere to diet and exercise at the end of 5 years. Old Program = 0.5 Adhere New Program= 0.7 Adhere Alpha = 0.05, Power = 0.8. You will need 103 individuals in each group. 41 Final Points • Design your study such that you will have a sufficient number of subjects to be able to detect the effects that are clinically meaningful (high power). • If you have a limited budget, and you can not afford to increase your sample size to the necessary levels, and lowering the variability is not feasible, you should consider alternative designs and hypotheses rather than proceeding with a study design with low power. 42