of Medical Literature
What is Evidence-Based Medicine?
“Evidence-based medicine is the integration of best
research evidence with clinical expertise and patient
values” (Sackett & Straus)
1. Information overload
2. Keeping current with literature
3. Our clinical performance deteriorates with
time (“the slippery slope”)
4. Traditional CME does not improve clinical
5. EBM encourages self directed learning process
which should overcome the above shortages
Medical evidence increasing at epidemic rates:
we all need EBP skills to keep up-to-date
2,000 articles / day
approx 75 new
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
The research-to-awareness gap
Medical Articles per Year
55 per per day
Trials MEDLINE BioMedical
2 4 6 8 10 12
Years after graduation
THE SLIPPERY SLOPE
The Prognosis of Ignorance is Poor
Worse with “duration in practice”
Steps in EBM practice
1. Formulate clinical problems in answerable questions
2. Search the best evidence: use internet or other on-
line database for current evidence
3. Critically appraise the evidence for VIA
Validity (was the study valid?)
Importance (were the results clinically
Applicability (could we apply to our patient?)
4. Apply the evidence to patient
5. Evaluate our performance
(Determination of disease or problem)
(What causes the disease)
(Prediction of the outcome of the disease)
(Intervention necessary to help the patient)
>25,000 journals worldwide
>2 million published articles per year
Many published articles have methodological
(including statistical) flaws – even in most respected
Not all results can be applied due to many reasons,
a.o. dissimilarities of study subjects with our patients
Limited time of physicians: focus on articles relevant to
your clinical practice
All articles should be
critically appraised for: VIA
Validity : was the study valid?
Importance : were the results clinically
Applicability : were the valid and
important results can be
applied to my patients?
For educational purposes,
students are encouraged
to consider all parts of the report,
from title to list of references,
before specifically appraise the content.
Anatomy & physiology of research reports:
Introduction - Why did I start?
Methods - What did I do?
Results - What did I find?
Discussion - What do they mean?
What is the main purpose
of critical appraisals?
To determine that the study was valid and the results
were important and can be applied to our patients
How can we determine validity?
Methods! (and Results)
How can we know that the results were important?
Your clinical judgment!
How can we know that the results can be applied to
Make sure that your patients are similar to those in
the study (You should assume that your patients are
part of the target population (domain) of the study
General approach to asses validity of
the study: RAMMbo
R = Recruitment
A = Allocation
M = Maintenance
M = Measurements
Relation of PICO to RAMMbo
RCT (Pragmatic trials): Validity
Were the study participants randomized?
Was the randomization technique described?
Was the randomization table concealed?
Were the characteristics of the subjects similar at
the start of the intervention?
Were all participants given equal treatment apart
from the intervention?
Were all relevant outcomes considered?
Were the results analyzed correctly?
RCT (Pragmatic trials): Importance
Calculate: EER, CER, RRR, ARR, NNT
C c d
EER = a/(a+b) RRR = (CER-EER)/CER
CER = c/(c+d) ARR = CER-EER
NNT = 1/ARR
Were the participations similar to your
May be intuitively concluded or use f (factor
indicating how much severe your patient
compared to the study participation in terms of
DIAGNOSTIC TEST: Validity
Was independent and blind comparison to
gold standard applied?
Was the diagnostic test include spectrum of
disease similar to your real practice?
Was the gold standard applied regardless
of the diagnostic result?
Diagnostic Test: Importance
Calculate: Sensitivity, specificity, predictive
values, likelihood ratios
Se = a/(a+c) LR+ = se/(1-sp)
Sp = d/(b+d) LR - = (1-se) /sp
PPV = a/(a+b) Posttest odds = Pretest odds x LR+
NPV = d/(c+d)
Diagnostic Test: Applicability
Were the participations similar to your
Is the diagnostic test applicable, acceptable,
and affordable in your setting?
Will the result of the test help your patient?
Prognostic Studies: Validity
Was the inception cohort assembled in usual point
of course of the disease?
Was the follow-up sufficient & complete?
Were outcome criteria applied in blind fashion?
Was there any validation in other group of
Was subgroup analysis performed after
adjustment for prognostic factors?
Prognostic studies: Importance
Calculate: Relative risk (RR) and 95% CI
+ a b
- c d
RR > 1: risk factor
RR = a/(a+b) : c/(c+d) RR < 1 : protective factor
RR = 1 : not a risk factor
Prognostic studies: Applicability
Were the patients similar to yours?
Will the evidence make a clinically
important impact for the care of your
Is it the SR / Meta-analysis you are interested?
Does it include Methods describing how to find
Does it include description of assessment of
Were the results consistent from study to study?
Are the valid results clinically important?
Inspect individual studies: OR / RR/mean
difference - point estimates and confidence
Inspect combined OR / RR / mean difference
Comments if any
Is your patient similar to the patients in the
How big is the benefit?
calculate from the OR
select the OR of best study
Critically appraisals the evidence
Be selective and critical in your reading
To decide quickly whether a study warrants attention
look at the abstract and answer 7 questions (PP-
Problem: is it a problem I see in my practice?
Patient population: does the study’s patient population
look like my patient population?
Intervention: what is the intervention, and is it realistic
in my setting?
Comparison: what is the intervention being compared
to, and is it a reasonable comparison?
Outcomes: would the outcomes matter to my patients?
Number: how many patients were in the study?
Statistics: how does the study present its finding
number needed to treat: how many pts you need to
treat for one pts to benefit.
Three common myths/mistakes in
reviewing the literature
It is in a “big name” journal it must be good
Many dangerous DOE’s have been printed in NEJM, JAMA,
Annals of ____ etc.
The author is a big name, or well published, or from
an impressive institution
My article is newer than your article
Newer is often NOT better
Three quick and easy tips for critical
appraisal of the literature
Look for relevance first
• Almost always quicker & easier to determine relevance than
to do the hard work of determining validity
• If it is not relevant – keep looking
If you cannot figure out statistics – assume they are
• Look at the other important aspects of the study
• And keep working on BASIC understanding of stats
Let someone else do the hard work for you
• Look in high quality evidence based sources first
Implement information into practice
Integrate information with patients values and
Demographics, age, socioeconomics, fear, etc.
Evidence may point to surgery as better
treatment but patient refuses
This does NOT mean EBM is out the window
Your job is to understand the magnitude of benefit and the
level of evidence, then translate into useable information for
Evaluate the implementation
Effectiveness and efficacy the evidence on your own
And also how’s these steps improve your…
framing the question
critical appraisal skills
patient centered approach