Document Sample
xEBM 005 APPRAISALS Powered By Docstoc

 Critical Appraisals
of Medical Literature
What is Evidence-Based Medicine?
“Evidence-based medicine is the integration of best
research evidence with clinical expertise and patient
             values” (Sackett & Straus)

                      Best research

            Patient               Clinical
            values               expertise
               WHY EBM?
1. Information overload
2. Keeping current with literature
3. Our clinical performance deteriorates with
   time (“the slippery slope”)
4. Traditional CME does not improve clinical
5. EBM encourages self directed learning process
   which should overcome the above shortages
    Medical evidence increasing at epidemic rates:
      we all need EBP skills to keep up-to-date
2,000 articles / day

  approx 75 new
  trials published
      every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
       The research-to-awareness gap

                                                                  per day
Medical Articles per Year



                            1000000                  1,500
                                        55 per      per day
                                      Trials     MEDLINE      BioMedical

Relative                                            $
% of

                   2   4       6        8      10       12

                           Years after graduation

                   THE SLIPPERY SLOPE
The Prognosis of Ignorance is Poor

          Worse with “duration in practice”
           Steps in EBM practice
1. Formulate clinical problems in answerable questions
2. Search the best evidence: use internet or other on-
   line database for current evidence
3. Critically appraise the evidence for     VIA
        Validity       (was the study valid?)
        Importance (were the results clinically
        Applicability (could we apply to our patient?)
4. Apply the evidence to patient
5. Evaluate our performance
             Main area
  (Determination of disease or problem)

        (What causes the disease)

(Prediction of the outcome of the disease)

(Intervention necessary to help the patient)
>25,000 journals worldwide
>2 million published articles per year
Many published articles have methodological
(including statistical) flaws – even in most respected
Not all results can be applied due to many reasons,
a.o. dissimilarities of study subjects with our patients
Limited time of physicians: focus on articles relevant to
your clinical practice
        All articles should be
   critically appraised for: VIA

Validity        :   was the study valid?
Importance      :   were the results clinically
Applicability   :   were the valid and
                    important results can be
                    applied to my patients?
       For educational purposes,
        students are encouraged
   to consider all parts of the report,
     from title to list of references,
before specifically appraise the content.
Anatomy & physiology of research reports:

  Introduction    - Why did I start?
  Methods         - What did I do?
  Results         - What did I find?
  Discussion      - What do they mean?
       What is the main purpose
        of critical appraisals?
To determine that the study was valid and the results
were important and can be applied to our patients
How can we determine validity?
  Methods! (and Results)

How can we know that the results were important?
  Your clinical judgment!

How can we know that the results can be applied to
our patients?
  Make sure that your patients are similar to those in
   the study (You should assume that your patients are
   part of the target population (domain) of the study
General approach to asses validity of
        the study: RAMMbo
         R   = Recruitment
         A   = Allocation
         M   = Maintenance
         M   = Measurements
                blinded?

                objective?
 Relation of PICO to RAMMbo

Patient/Problem   Recruitment

Intervention      Allocation
Comparison        Maintenance

Outcome           Measurements
                   blinded?

                   objective?
 RCT (Pragmatic trials): Validity
Were the study participants randomized?
Was the randomization technique described?
Was the randomization table concealed?
Were the characteristics of the subjects similar at
the start of the intervention?
Were all participants given equal treatment apart
from the intervention?
Were all relevant outcomes considered?
Were the results analyzed correctly?
RCT (Pragmatic trials): Importance
     Calculate: EER, CER, RRR, ARR, NNT

             Success    Failure

              a            b

     C        c            d

    EER = a/(a+b)      RRR = (CER-EER)/CER
    CER = c/(c+d)      ARR = CER-EER
                       NNT = 1/ARR
        RCT: Applicability
Were the participations similar to your
May be intuitively concluded or use f (factor
indicating how much severe your patient
compared to the study participation in terms of
prognostic factor)
Was independent and blind comparison to
gold standard applied?
Was the diagnostic test include spectrum of
disease similar to your real practice?
Was the gold standard applied regardless
of the diagnostic result?
       Diagnostic Test: Importance
 Calculate: Sensitivity, specificity, predictive
 values, likelihood ratios

                +           -

                    a       b
                    c       d

        Se = a/(a+c)    LR+ = se/(1-sp)
        Sp = d/(b+d)    LR - = (1-se) /sp
        PPV = a/(a+b)   Posttest odds = Pretest odds x LR+
        NPV = d/(c+d)
Diagnostic Test: Applicability

Were the participations similar to your
Is the diagnostic test applicable, acceptable,
and affordable in your setting?
Will the result of the test help your patient?
   Prognostic Studies: Validity
Was the inception cohort assembled in usual point
of course of the disease?
Was the follow-up sufficient & complete?
Were outcome criteria applied in blind fashion?
Was there any validation in other group of
Was subgroup analysis performed after
adjustment for prognostic factors?
     Prognostic studies: Importance
     Calculate: Relative risk (RR) and 95% CI
                      +          -

            +        a      b
            -        c      d

                                RR > 1: risk factor
   RR = a/(a+b) : c/(c+d)       RR < 1 : protective factor
                                RR = 1 : not a risk factor
Prognostic studies: Applicability

  Were the patients similar to yours?
  Will the evidence make a clinically
  important impact for the care of your
      Meta-analysis: validity
Is it the SR / Meta-analysis you are interested?
Does it include Methods describing how to find
Does it include description of assessment of
individual validity?
Were the results consistent from study to study?
   Meta-analysis: Importance
Are the valid results clinically important?
  Inspect individual studies: OR / RR/mean
  difference - point estimates and confidence
  Inspect combined OR / RR / mean difference
Comments if any
  Meta-analysis: Applicability
Is your patient similar to the patients in the
How big is the benefit?
      calculate from the OR
      select the OR of best study
 Critically appraisals the evidence

Be selective and critical in your reading

To decide quickly whether a study warrants attention
 look at the abstract and answer 7 questions (PP-
Problem: is it a problem I see in my practice?
Patient population: does the study’s patient population
look like my patient population?
Intervention: what is the intervention, and is it realistic
in my setting?
Comparison: what is the intervention being compared
to, and is it a reasonable comparison?
Outcomes: would the outcomes matter to my patients?
Number: how many patients were in the study?
Statistics: how does the study present its finding 
number needed to treat: how many pts you need to
treat for one pts to benefit.
Three common myths/mistakes in
     reviewing the literature
It is in a “big name” journal it must be good
  Many dangerous DOE’s have been printed in NEJM, JAMA,
  Annals of ____ etc.
The author is a big name, or well published, or from
an impressive institution
My article is newer than your article
  Newer is often NOT better
    Three quick and easy tips for critical
         appraisal of the literature
   Look for relevance first
     •   Almost always quicker & easier to determine relevance than
         to do the hard work of determining validity
     •   If it is not relevant – keep looking
   If you cannot figure out statistics – assume they are
     •   Look at the other important aspects of the study
     •   And keep working on BASIC understanding of stats
   Let someone else do the hard work for you
     •   Look in high quality evidence based sources first
Implement information into practice

 Integrate information with patients values and
 Patient-centered care
   Demographics, age, socioeconomics, fear, etc.
 Evidence may point to surgery as better
 treatment but patient refuses
   This does NOT mean EBM is out the window
   Your job is to understand the magnitude of benefit and the
   level of evidence, then translate into useable information for
   the patient
  Evaluate the implementation
Effectiveness and efficacy the evidence on your own
And also how’s these steps improve your…
  framing the question
  search term
  search location
  critical appraisal skills
  patient understanding
  patient centered approach

Shared By: