EPI-214: Lecture 1 Designing a Systematic Review (Meta-analysis) Dejana Braithwaite Assistant Professor UCSF Department of Epidemiology and Biostatistics 8 steps of a systematic review 1 Formulate research question Lecture 1 2 Develop review protocol 3 Initiate search strategy 4 Apply inclusion /exclusion criteria 5 Quality appraisal 6 Data abstraction 7 Analysis Lectures 1, 2 & 3 8 Interpret findings Bent et al. 2004 What’s a Systematic Review? “A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review.” Cochrane Collaboration …and meta-analysis? Statistical combination of >= 2 studies to produce single estimate of effect of exposure Rapid growth Pubmed search up to 3/29/2010: Systematic review >1.5 million hits Meta-analysis >40,000 hits up to 1/1/1990: Systematic review 300,994 hits Meta-analysis 845 The Cochrane Collaboration International systematic review initiative • Archie Cochrane’s vision led to the opening of the first Cochrane centre (in Oxford, UK) in 1992 and the founding of the Cochrane Collaboration in 1993 Source: http://www.cochrane.org/cochrane/archieco.htm Meta-analysis news coverage Sattar N, et al. Statins and risk of incident diabetes: a collaborative meta-analysis of randomised statin trials. Lancet. 2010 Feb 27;375(9716):735-42. Epub 2010 Feb 16. Systematic Review in context Traditional Systematic Meta- review Review analysis Author An A team A team individual Search IndividualBased on Based on strategy a protocol a protocol Summary Author’s Can be Summary Conclusio judgement qualitative statistical n techniques Narrative review Uses informal, unsystematic and subjective methods Searching quality and synthesis not described Disadvantage may have preconceived biases and may overestimate value of some studies Systematic review Driven by evidence-based medicine movement and Cochrane collaboration Advantages: Reduces bias Replicable Resolves controversy between conflicting findings Provides reliable basis for decision making Use of meta-analysis as a prelude to clinical trials Define pre-trial expected effect sizes sample size estimation Determine effect estimates in key subgroups (e.g. based on gender, race/ethnicity or age) Identify sources of heterogeneity in prior studies Addressing these sources in design phase of new trial Use of meta-analysis in study designs that are not clinical trials Observational studies (e.g. case control, cohorts, cross-sectional prevalence studies, etc.) Studies evaluating diagnostic tests (sensitivity, specificity, predictive value) “IPD” = individual patient data studies Qualitative studies (meta-ethnography) Meta-analyses IPD* Systematic reviews *IPD= individual participant data Resources required for systematic reviewing Can be time consuming Team science (to reduce bias) Bibliographic software (e.g. Endnote) Statistical software (if appropriate) Step 1 Formulate research question FINER criteria for research question Feasible Interesting Novel Ethical Relevant Hulley S, et al. 2001 Designing Clinical Research Components of a research question (PICOT) Patient: Disease or condition Stage, severity Demographic characteristics (age, gender, etc.) Intervention (or “Exposure”): Type of intervention or exposure Dose, duration, timing, route, etc. Comparison: Absence of risk or treatment Placebo or alternative therapy Outcome: Risk or protective Dichotomous or continuous Type: mortality, morbidity, quality of life, etc. PICOT Type of Study: RCTs Cohort Case-control Cross-sectional All Formulation of an etiology question Exposure Outcome Is smoking a risk factor for lung cancer? Patient Exposure Are people who smoke regularly at a greater risk of developing lung cancer as compared to those who do not smoke? + cohort & case-control studies Outcome Comparison Formulation of a diagnosis question Test (intervention) Outcome Is MRI a good screening test for breast cancer? Test (intervention) Outcome Is MRI a more sensitive and specific test in diagnosing breast cancer as compared to mammography among high risk women? Comparison Step 2 Develop review protocol Protocol Background Objectives Pre-determined selection criteria Planned search strategy Planned data abstraction Proposed method of synthesis of findings Establishment of an advisory group Step 3 Initiate search strategy Where to locate studies Pubmed CINAHL Web of Science EMBASE PsychINFO Additional sources to identify studies for systematic reviews Reference lists of retrieved articles Manual searching of relevant publications Experts in the field Corresponding or first authors of published studies identified for the systematic review Issues to consider Publication bias Search bias Pubmed citation example Title: Interaction between 5-HTTLPR genotype, stressful life events and depression Search terms: Life stress Life event Depression Depress Serotonin transporter 5-HTTLPR Interaction Moderation Risch et al. JAMA 2010 Step 4 Apply inclusion /exclusion criteria Inclusion/exclusion criteria P - Population I - Intervention C - Comparison (if necessary) O - Outcome T - Type of study (if necessary) Subject headings OR Textwords To find studies using all of the PICO elements: P and I and C and O (and T) Exclusion criteria Keep log of excluded studies Note reasons for exclusion Have eligibility checked by more than one reviewer Develop strategy to resolve disagreements Search strategy example Risch et al. JAMA 2010 Step 5 Quality appraisal Principles of quality appraisal Quantitative studies Internal Validity allocation bias, confounding, attrition, statistical analysis, intervention integrity, withdrawals and dropouts External Validity (generalizability or applicability) Are there quality appraisal tools? QUORUM for trials Moher et al. Improving the quality of reports of meta-analyses of randomized controlled trials: The QUORUM statement. Lancet 1999;354:1896-1900 MOOSE for observational designs Stroup et al. Meta-analysis of observational studies in epidemiology. JAMA 2000;283:2008-12 Qualitative research Checklists available to assess the quality of qualitative research. E.g. CASP appraisal tool for qualitative research (http://www.phru.nhs.uk/casp/qualitat.htm) Recruit SELECTION BIAS participants ALLOCATION BIAS Allocate to intervention and control groups Intervention group Control group CONFOUNDING INTEGRITY OF Implement intervention INTERVENTION Implement intervention INTENTION-TO-TREAT Follow-up participants Follow-up participants WITHDRAWALS/ DROP OUTS Measure outcomes BLINDING OUTCOME Measure outcomes ASSESSORS DATA COLLECTION Analyze outcomes METHODS Analyze outcomes STATISTICAL ANALYSIS Step 6 Data abstraction Data abstraction Design and pilot data abstraction form Consider >1 reviewer Consider blinding of observers to authors, institutions and journals Data abstraction elements publication details study design population details (n, characteristics) intervention details setting outcomes and findings Summary of study characteristics - example Risch et al. JAMA 2009 Data abstraction examples http://www.cochrane.org Step 7 Analysis Synthesizing the Evidence NARRATIVE SYNTHESIS primary studies explained qualitatively and summarized META-ANALYSIS findings summarized and then combined statistically Is there heterogeneity? No Yes Meta-analysis Narrative Deal with synthesis heterogeneity? (e.g. subgroup analysis) Guidelines for narrative synthesis Not available yet Step 8 Interpret findings Interpretation of results Strength of the evidence Explanations of effectiveness Applicability Trade-offs between benefits and harms Implications for practice Strength of the evidence • Describe the overall strength of the evidence • The quality of the evidence • The size and consistency of the results (particularly important in population based studies where a small effect at the community level may have much significance than the effect of comparable size at the individual level) • Describe biases and limitations of the review process (e.g. difficulties in locating articles, inability to translate non-English articles) 8 steps of a systematic review Bent et al. 2004 1 Formulate research question Lecture 1 2 Develop review protocol 3 Initiate search strategy 4 Apply inclusion /exclusion criteria 5 Quality appraisal 6 Data abstraction 7 Analysis Lectures 1,2 & 3 8 Interpret findings Section 1 Assignment Developing a Protocol for a Systematic Review Homework: Write a 1-2 page protocol to address the clinical problem described in the assignment.
Pages to are hidden for
"Systematic Review"Please download to view full document