Hierarchy of Clinical Evidence
Systematic Reviews Metaanalysis Double-blind Randomized Controlled Trials Cohort Studies Case-Control Studies Cases Report/Series Physiologic Studies Intuition or Beliefs
Design of a cohort study
TIME direction of inquiry
people population
without the
exposed
disease no disease
disease not exposed “at risk”
disease no disease
Design of a case-control study
TIME direction of inquiry
Start with:
Exposed
Not exposed Population Exposed Not exposed controls (people without disease)
cases (people with disease)
Questions to ask when an association is reported in the literature
(eg estrogen and CHD)
Explanation Association Bias in selection or measurement Chance Confounding Cause Yes No Finding
Likely
Unlikely Yes No Cause
Case report?
Association and cause
Finding Association
Explanation
Bias in selection or measurement Chance Confounding Cause
Yes
No
Likely
Unlikely Yes No Cause
Case series?
Association and cause
Explanation Association Bias in selection or measurement Chance Confounding Cause P value Yes No Finding
Likely
Unlikely Yes No Cause
Association and cause
Explanation Association Bias in selection or measurement Chance Confounding Cause Yes No Finding
Likely
Unlikely Yes No Cause
Criteria for causation
1. Is there evidence from true experimentation in humans? 2. Is the association strong? 3. Is the association consistent from study to study? 4. Is the temporal association correct? 5. Is there a dose-response gradient? 6. Does the association make biological sense? 7. Is the association specific? (Adapted from Bradford Hill)
Randomization
1. Guarantees equal probability of receiving control/experimental treatment to all participants (removes investigator bias) 2. Protects against imbalances in known and unknown confounders 3. Provides basis for statistical analysis
Hierarchy of study methods for clinical decision-making
Systematic reviews
Is observational evidence equivalent to experimental evidence?
Benson NEJM 2000;342:1877 Concato NEJM 2000;342:1887
In others- NO !
In some cases YES
Clinical trials are selective!
-Select group gets in - chance decides who gets treatment; treatment effect decides who does better
Observational evidence is also selective: self-selection of exposure: this may decide who does better
How much of Medicine is Evidence-Based ?
Matzen P. Ugeskr laeger 2003;165:1431-5 • General Internal Medicine - 50% • Psychiatry- 65% • Others (surgery, general practice, dermatology) - less Lai Br. J Ophthal . 2003;4:385-90: • 42.9% of patient interventions were based on evidence from RCT, metaanalysis or systematic reviews (23% on no evidence)
Major Disadvantage of RCTs: Selectivity
Clinical trials
Should not be performed unless there is a realistic chance of providing a valid/reliable answer to a welldefined medical question
But…SHEP study
Of 447,921 (100%) identified
31,960 (11.6%) met initial criteria
4,736 (1.03%) randomized
0 1.4 10.1
Comorbidity: General Pop vs. Subgroup
25 20
10 5 0
Men, gen’l pop Wom., gen’l pop SHEP pop DM
5
CVD 4
3 CHF 2 MI
Angina 1
% of Pop
% of Pop
15
Coke tastes better…….!
Than what?
Coke tastes better…….!
CAVEATS in using RCT evidence to guide patient management
1. Tendency to extend application of new treatments to patient groups other that those for whom data exist
2. Extrapolation of data to agents of the same class but untested for specific indication
Don’t drown in the evidence
ACP J Club, Bandolier, POEMS, clinical evidence, Cochrane Reviews, Clinical guidelines clearinghouse
Use predigested sources
Good luck!