Moderators of Treatment Effects in the General Medicine Literature: Looking for Improvement
Nicole Bloser, MHA, MPH University of California, Davis June 5, 2007
1
Background
Parallel group randomized controlled trials (RCTs) are the cornerstone of evidence based medicine Result: Average of usually immeasurable individual effects
Some benefit, some harmed Heterogeneity of treatment effects
2
Background
Examining treatment impact in similar individuals or subgroups
N-of-1 clinical trial Prospective stratification with a multivariable risk index Subgroup analysis
3
Background
Subgroup analysis – problems
Low power Multiple testing
Not all subgroup analysis is the same
P-value within subgroup ≠ interaction analysis Interaction analysis is the correct way to examine moderators of treatment effects (MTEs)
4
Background
MTEs necessary to maximize benefit and minimize harm MTEs often not examined (<50% of RCTs report interaction analysis) Studies that examine MTE reporting:
Majority in cardiovascular literature None since revised CONSORT statement (2001)
5
Research Objective
We sought to identify current practice in evaluating moderators of treatment effects (MTEs) and to elucidate trends Persistent low rate of analyses would suggest
Missed opportunities Slower progress towards personalized medicine
6
Study Design
Systematic review
Annals, BMJ, JAMA, Lancet, NEJM Odd months, 1994, 1999, 2004
Randomized controlled trials Unit of randomization as the individual All independently reviewed and coded by two investigators Adjudication by a third
7
Study Selection
4863 articles from initial search
Exclude: All articles that were not clinical trials. N=4,322
N=541, Random sample of N=379 selected
Include
N=303 articles
Exclude
N=76 articles Reasons for exclusion: Not RCT (N=61 trials)
319 trials included
9 trials excluded
77 trials excluded
Unit of randomization not individual patient 8 (N=25 trials)
Methods
Trials were coded as having
MTE analysis (utilizing a formal test for heterogeneity) Subgroup analysis only (no formal test) Neither
Chi-square test used for bivariate comparisons Multiple logistic regression used to identify predictors of MTE analysis
9
Trial Characteristics
n (%) Journal of Publication Annals BMJ JAMA Lancet NEJM Year of Publication 1994 1999 2004 Condition Studied Cardiovascular Cancer Infectious Disease Psych / Neuro Other 30 (9) 47 (15) 50 (16) 101 (32) 91 (29) 91 (29) 106 (33) 122 (38) 74 (23) 42 (13) 70 (22) 25 (8) 108 (34)
10
Trial Characteristics
38% of trials had authors from North America (US and Canada) 95% utilized a parallel group design Study sample size ranged from 6-41,000 (Median: 262, IQR: 101-708)
11
MTE and Subgroup Reporting
Analysis reported None Subgroup without statistical comparison MTE
n (%) 139 (45) 88 (28) 92 (29)
For those trials reporting MTE analysis: 43 (47%) reported on one covariate 24 (26%) reported on 2-4 covariates 17 (18%) reported on 5-10 covariates 7 (7%) reported on 11-19 covariates
12
Covariates examined for MTE
Among those trials that reported MTE, major covariates examined included:
Individual Risk Factors Gender Age Study Site Co-occurring treatment Co-morbidities Race/Ethnicity % 56% 36% 29% 29% 25% 21% 15%
Only one trial reported MTE analysis using a composite multivariable risk index.
13
Bivariate analysis
Journals published in North America (Annals, JAMA) and first authors writing from North America were more likely to publish trials with MTE analysis Sample size (p<0.0001 for trend)
Quintile 1: 14%, Quintile 5: 52%
More MTE analysis reported in each of the three successive time periods (p=0.047)
14
Logistic Regression
Prediction of MTE analysis Included in model: study year, journal clinical condition, first author’s region, and sample size Journal and sample size were significant
Reference categories: BMJ, Quintile 1 JAMA: 4.4 (1.4, 13.5), Annals: 4.2 (1.2, 15.1) Quintile 5: 7.5 (2.9, 19.3)
15
Limitations
Limited number of trials reviewed MTE results are published elsewhere MTE examined, but not reported due to nonsignificance
16
Conclusions
Missed opportunities for MTE analysis abound
Conservative reaction that stifles hypothesis generation Impairs recognition of patient strata Impedes future research
When subgroups are reported, the reporting is not done correctly in many (~50%) cases. This could lead to erroneous conclusions.
17
Conclusions
NIH guidelines regarding subgroup specific results recommends reporting both significant and non-significant results, yet only half of the trials reported any MTE analysis. In the face of broad NIH mandates for inclusion of subjects by race/ethnicity, the low proportion of trials examining race/ethnicity as a treatment effect modifier is puzzling.
18
Policy Implications
MTE analysis critical to future research The genomic revolution is only going to increase the desire to individualize treatment effects Kraemer et al. argue that exploratory moderator analysis is critical for designing future confirmatory studies
Significant exploratory effects are later used as guidance for future stratification
Kraemer, HC, E Frank, and DJ Kupfer, Moderators of treatment outcomes: clinical, research, and policy importance. Jama, 2006. 296(10): p. 1286-9. 19
Policy Implications
Rigorous and routine exploratory MTE analysis is necessary and should be encouraged Standards are essential for developing practice guidelines that are appropriate to the needs of complex patients
20
Research Team
University of California, Davis University of California, Los Angeles
Richard Kravitz, MD, MSPH - PI Elizabeth Yakes, MS
Naihua Duan, PhD - PI Diana Liao, MPH Kiavash Nikkhou
21
Thank you
22