Docstoc

EBM-primer

Document Sample
EBM-primer Powered By Docstoc
					 Clinical Problem Solving
an introduction to Evidence-Based Medicine basics
         Lecture overview
• Objectives
• EBM skills for practicing medicine
  – Asking
  – Acquiring
  – Assessing
  – Applying
                  Objectives
• Define evidence-based medicine (EBM)
• Explain why we use EBM
   – Compare with expert-based medicine
   – How are we misled by:
      • Surrogate outcomes
      • Personal observation
      • Pathophysiologic reasoning
• Describe the tools of EBP
• Construct a well-built clinical question
      What is Evidence-Based Medicine (EBM)?



“Using the best available evidence for
making decisions about health care”
   What is important to read in the
         medical literature

Things that
   1.   Have patient oriented outcomes
   2.   Answer a patient-care question
   3.   Might change your practice
   4.   Are on a topic you have been following
   5.   People are talking about and you want to know more
   6.   You find interesting POEM or DOE
        •   Patient-oriented evidence that matters vs
            disease-oriented evidence
           “Intro”: EBM (I)
“Evidence-based medicine (EBM) requires the
  integration of the best research evidence
  with our clinical expertise and our patient’s
  unique values and circumstances”
                        EBM, 2006, Straus et al
Why learn EBM / EBP?
 “Good education teaches us
   to become both producers
         of knowledge
              &
discerning consumers of what
  other people claim to know.”
Helps you find the
truth in face of
pharmaceutical
marketing


        Cal Ripkin, Jr. is
        not hypertensive
        and is not taking
           PRINIVIL




    Its always in the fine print.
                     Value of Learning EBM:
                         Short-Term Trial
• A controlled trial of teaching critical appraisal of clinical
  literature conducted among medical students
• Experimental group of students worked with clinical tutors
  who had
    – Taken course in clinical appraisal
    – Evaluated specialty-specific articles on diagnostic tests and
      treatments
• Control group of students worked with usual clinical tutors
Bennett et al. JAMA. 1987;257:2451-2454.
               Value of Learning EBM:
                Short-Term Trial (cont)
• Students in experimental group made greater number
  of correct diagnostic and treatment decisions and were
  better able to justify their decisions
• Students in control group were more likely to make
  incorrect decisions after their tutorial than before it
   – Students in the control group had become more accepting of
     recommendations from authority figures
   Bennett et al. JAMA. 1987;257:2451-2454.
                       The Patient
• Patient is a 27-year-old woman with severe right lower
  quadrant pain.
   – initial peri-umbilical pain x 2 days migrating yesterday to
     current site.
• Loss of appetite. No vomiting, diarrhea; no bowel
  movement
• no known infectious exposure/
  suspicious ingestions, or recent travel
Standard medical practice for hot, moist diseases
       Louis’ Study of Bloodletting
Day of 1st
bleeding


                               Averages



Duration of
illness            Number of
                   bleedings
 Pierre Louis (1787-1872)
Inventor of the “numeric method” and the “method of bservation”




                              Discovered in 1828 that
                              patients who were bled
                              did worse than those who
                              weren’t
     Many advances in medicine with
            uncontrolled use

• PCN for life-threatening disease
• Insulin for type I diabetes
• Treatment of malignant hypertension
        Traditional Guides to Medical Practice

• Pathophysiology and pharmacology
   – Foundation of medical practice
   – Do what “makes sense”
• Expert opinion
   – In training: learning at the bedside from the master clinician
   – In practice: lectures and seminars with thought leaders
• Clinical experience
   – Successes, outcomes, and adverse events
     in our own practice
      Cardiac Arrhythmia Suppression Study



1498 subjects with suppressible arrhythmias post-MI
                  RANDOMIZED




     Treatment                     Placebo

                     Mortality
    7.7%                            3.0%
  Problems With the Traditional Approach
• Physiology may not predict clinical response
   – Beta-adrenergic blockade in heart failure
   – Encainide for post-MI arrhythmia
   – Estrogen replacement for cardioprotection
• Expert opinion
   – Only as good as the expert
   – May be affected by biases and conflicts of interest
• Clinical experience
   – Dramatic clinical experiences may unduly influence our practice
     patterns
   – May not take account of recent medical literature

   Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
      Clinical Practice. Chicago, IL: American Medical Association; 2001.
             Paradigms of Medicine

       Expert Based                  Evidence Based
Pathophysiological reasoning   Clinical Studies

Personal observation           Best evidence available

Expert based guidelines        Evidence based guidelines
              Seven alternatives to EBM
                 Humorous approach from BMJ

      Basis              Marker                     Unit
Evidence          RCT                   Odds ratio
Eminence          White hair            Optical dentistry
Vehemence         Level of stridency    Decibels
Eloquence         Smoothness            Adhesion score
Providence        Religious fervor      International units of
                                        piety
Diffidence        Level of gloom        Sighs
Nervousness       Litigation phobia     Bank Balance
                  level
Confidence*       Bravado               No sweat

                                          BMJ 1999;319:1618-1618
         Integrates Evidence With

• Clinical expertise
    – Experience
    – Judgment
•   Patient values and preferences
•   Quality of life
•   Costs
•   Other important factors
Clinical                Patient Values
Expertise               and
                        Preferences
              Best
            Available
            Evidence

Quality                     Costs
of Life
Focus: Treatment & Diagnosis
                        The Patient
• Patient is a 27-year-old woman with severe right lower
  quadrant pain.
   – initial peri-umbilical pain x 2 days migrating yesterday to
     current site.

• Loss of appetite.
• No vomiting, diarrhea; no bowel movement
• no known infectious exposure/ suspicious ingestions, or
  recent travel
                  Patient exam

VS BP 120/78 P 16 RR 12 T 98.8
Chest CTA. CV RRR s M/R/G
ABD: NML exam x decreased bowel tones and definite
  right lower quadrant pain, specifically at McBurney’s
  point.
no heptomegaly nor splenomegaly (enlarged liver or
  spleen). She has no rebound pain or involuntary
The Five “A’s”
1. Ask the right question
2. Acquire the evidence
3. Appraise the evidence
4. Apply the evidence
5. Assess its impact
                           Concern:

• Case discussion: 27 year old woman with right lower
  quadrant (RLQ) abdominal pain
• Background information available from textbooks-
   – What typically presents as RLQ pain
   – What is the clinical course of the different diagnoses
   – Specifically, what is typical presentation of appendicitis
• Foreground information
   – How good is a CT scan for appendicitis?
Formulating the Question
     Formulating the Question
• An ideal question:
  – Focused enough to be answerable
  – Pertinent to clinical scenario
  – Framed as
    Population receiving an
    Intervention (test or treatment) [as Compared to
    other test/treatment or placebo] associated with
    Outcome (disease or improvement)
         PICOS
P roblem/population
I   ntervention

C   omparison

O   utcome

S   tudy design
     Examples of tough questions

• Should I screen men for prostate cancer?
• Who is a good candidate for hormone
  replacement therapy?
• Are angiotensin receptor blockers now first-line
  for hypertension?
     Examples of better questions
• Would a PSA test reduce mortality in a 65 year-old
  asymptomatic man?
• What is the reduction in fracture risk associated
  with hormone replacement therapy?
• Is losartan more effective than atenolol at
  preventing cardiovascular events in middle-aged
  hypertensive diabetic women?
                      PICOS

PICOS for confirmatory diagnosis of appendicitis
P:    27 year old woman with symptoms suggestive of
      appendicitis
I:    CT Scan
C:    Ultrasound
O:    Accurate diagnosis without undue delay
S:     ??
               Important Outcomes
• Patient Oriented Outcomes:
  outcomes patients actually care about
  – Death (overall or disease-specific)
  – Heart attacks, strokes, amputations, bed sores, broken hips, renal
    failure, etc.
  – Ability to perform activities of daily living
                         Versus
• Disease oriented outcomes:
  – Biochemical, physiologic, pharmacologic, or laboratory measures
      Comparing DOE and POE
                                       Patient-Oriented
                     Disease-Oriented    Evidence that
     Example              Evidence          Matters                           Comment
Antiarrhythmic       Drug X  PVCs    Drug X increases                    POE contradicts
Therapy              on ECG           mortality                           DOE
Type 2 Diabetes      Aggressive Tx    Aggressive Tx                       POE contradicts
                     with insulin or  does not reduce                     standard teaching
                     oral agents      mortality or
                     can keep BS low prevent most
                                      complications
Prostate             PSA screening    Does PSA                            DOE exists, but
Screening            detects prostate screening                          POE is unknown
                     cancer early     mortality?

                  Shaughnessy AF, Slawson DC. Getting the Most from Review Articles: A Guide for
                  Readers and Writers. American Family Physician 1997 (May 1);55:2155-60.
       Background versus foreground
               information
• Case discussion: 27 year old woman with right lower
  quadrant (RLQ) abdominal pain
• Background information available from textbooks-
   – What typically presents as RLQ pain
   – What is the clinical course of the different diagnoses
   – Specifically, what is typical presentation of appendicitis
• Foreground information
   – How good is a CT scan for appendicitis?
        Steps of EBM-5 A’s

•   Ask
•   Acquire
•   Appraise
•   Apply
•   Assess
  “Finding Evidence”: Sources (I)
• Primary research database (articles)
  – PubMed (aka MEDLINE), Pyschlit, CCTR
• Secondary research databases (synthesis)
  – Cochrane Library, Clinical Evidence, InfoPOEMS,
    UpToDate
• Tertiary resources (meta search engines,
  databases of databases)
  – TRIP+ (Translating Research Into Practice),
    PrimeEvidence
       “Finding Evidence”: Sources
• PubMed
    – 16 million peer reviewed biomedical articles indexed (note can use PubMed
      limits to search on particular populations, study types, etc.)
• Cochrane Library
    – ~3000 clinical systematic reviews (gold standard database)
• Clinical Evidence
    – ~2500 tsystematic reviews of treatment classified by likelihood of benefit
• InfoPOEMS (www.infopoems.com)
    – ~3000 regularly updated entries, Patient Oriented Evidence the Matters
      (POEM), 100+ journals monitored
• UpToDate
    – 70,000 pages, evidence based clinical information resource, ~3000 authors,
      350+ journals monitored, peer reviewed
• TRIP+
    – Meta-search of 55 sites of evidence based information
       “Finding Evidence”: Searching
1.   Convert clinical question to searchable question (e.g. PICOS)

2.   Choose the database you want to search (e.g. PubMed)

3.   Apply filters to restrict your search (e.g. PubMed limits linked to
     PICOS such as gender, age, study type limits)

4.   Assess result (e.g. using systematic review worksheet)

5.   Decide if you have enough information to make a decision

6.   If not then refine steps 1-3 until you either have an answer or decide
     there isn’t enough evidence to make an evidence based decision
Appraising the Evidence
                        Assess the Evidence
• Is the study valid?
   – Validity is defined as relative freedom from bias and confounding
     factors
• What are the results?
   – What is the outcome and how was it measured?
   – What is the magnitude of the effect?
   – Are the results statistically significant?
• Do the results apply to my patient?
   – Does my patient resemble those in the study?
   – Were all outcomes relevant to my patient evaluated?
   – Are there other factors (eg, cost, availability) that limit applicability to
     my patient?
   Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice.
      Chicago, IL: American Medical Association; 2001
                          Objectives
• Understand difference between observational and
  experimental studies
• For 2 major study designs (randomized controlled trial and
  cohort study) describe
   –   How the study is designed
   –   Advantages and disadvantages of design
   –   How to assess validity
   –   How to assess results
   –   How to assess applicability
     Experimental vs Observational Studies

• In experimental studies, the investigator controls subjects’
  exposure to intervention
   – Example: randomized controlled trial (RCT)
• In observational studies, investigator does not control the
  exposure; it occurs naturally or is initiated by patients or their
  physicians
   – Examples: cohort study, case-control study

   Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
      Practice. Chicago, IL: American Medical Association; 2001.
                            RCTs
Generally held to be the optimal
methodology for determining benefit
or harm                   Treatment
                                                                           Outcome

   Eligible
                  Randomization
   Patients

                                                 Control                   Outcome




              Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
              Clinical Practice. Chicago, IL: American Medical Association; 2001.
                   RCTs: Advantages
• Treatment and control groups are likely to
  have similar distribution of known and
  unknown prognostic factors (potential
  confounders)
• Outcomes are determined prospectively in a
  standardized, systematic fashion
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
   Clinical Practice. Chicago, IL: American Medical Association; 2001.
                     RCTs: Disadvantages
• Costly to perform
• Size limitations make detection of rare events
  difficult (eg, adverse medication effects)
• Eligibility restrictions may reduce applicability to
  real patients
• Cannot be ethically performed if exposure is
  expected to cause harm (eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice.
   Chicago, IL: American Medical Association; 2001.
                    RCTs: Disadvantages
• Costly to perform
• Size limitations make detection of rare events
  difficult (eg, adverse medication effects)
• Eligibility restrictions may reduce applicability
  to real patients
• Cannot be ethically performed if exposure is
  expected to cause harm (eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL:
    American Medical Association; 2001.
      Assessing the Validity of RCTs
• Was randomization concealed?
• Were patients analyzed in groups to which they were
  randomized?
• Were patients in treatment & control groups similar
  with respect to prognostic factors?
• Were patients, clinicians, outcome assessors, and
  data analysts aware of allocation?
• Were groups treated equally?
• Was follow-up complete?
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago,
    IL: American Medical Association; 2001.
      Assessing the Results of an RCT
• Magnitude of result: How large was the treatment
  effect?
     – Relative risk and odds ratio
     – Absolute risk reduction and number needed to treat (NNT)
• Statistical significance
     – P value
     – Confidence interval: How precise was estimate of treatment
       effect?
• Clinical significance
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001.
     Assessing the Results of an RCT
• Magnitude of result: How large was the treatment
  effect?
     – Relative risk and odds ratio
     – Absolute risk reduction and number needed to treat (NNT)
• Statistical significance
     – P value
     – Confidence interval: How precise was estimate of
       treatment effect?
• Clinical significance
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001.
Calculating the Risk Ratio and Number
        Needed to Treat (NNT)
   Treatment                     100 have the                         Risk = 0.1, or 10%
   (n = 1000)                    outcome                              (100/1000)



   Control                       120 have the                         Risk = 0.12, or 12%
   (n = 1000)                    outcome                              (120/1000)


                 Risk ratio = 0.1/0.12 = 0.83, or 83%

                 Absolute risk reduction = 0.12 - 0.1 = 0.02, or 2%

                 NNT = 1/0.02 = 50


         Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
         Practice. Chicago, IL: American Medical Association; 2001.
 Assessing the Applicability of an RCT
• Were the study patients similar to
  my patient?
     – Eligibility criteria
     – “Table 1” data (baseline characteristics)
• Were all clinically important outcomes
  considered?
• Are the likely treatment benefits worth the
  potential harm and costs?
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice.
   Chicago, IL: American Medical Association; 2001.
                Cohort Studies
• Similar to RCTs, except that assignment
  to intervention is not random
                                              Exposed                        Outcome

   Eligible         Choice or
   Patients         Happenstance

                                               Not                           Outcome
                                               Exposed



              Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
              Clinical Practice. Chicago, IL: American Medical Association; 2001.
           Cohort Studies: Advantages
• Outcomes are determined prospectively in a
  standardized, systematic fashion
• Often includes a larger, more diverse
  population than those eligible for or included in
  RCTs
• Can be used to assess effects of harmful
  exposures (eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001
    Cohort Studies: Disadvantages
• Costly to perform
• Size limitations make detecting rare events
  difficult
• Exposure and control groups are likely to
  differ in factors that may affect outcomes
• Control of confounding through statistical
  analysis may be inadequate
  Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
  Practice. Chicago, IL: American Medical Association; 2001.
             Assessing the Validity of a
                   Cohort Study
• Were the exposed and control groups similar in
  all known determinants of outcome?
     – Did the analysis adjust for potential differences?
• Were the outcomes measured in the same way
  in the groups being compared?
• Was follow-up sufficiently complete?
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice.
   Chicago, IL: American Medical Association; 2001.
               Assessing the Results of a
                     Cohort Study
• How strong is the association between exposure and
  outcome?
     – Risk ratio or odds ratio
     – Absolute risk increase or number needed to harm (NNH)
• Statistical significance
     – P value
     – Confidence interval: How precise was estimate of risk?
• Clinical significance
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001.
             Assessing the Applicability
                 of a Cohort Study
• Were the study patients similar to
  the patient under consideration in
  my practice?
• Should I attempt to stop the exposure?

Guyatt et al. Users' Guides A Manual for Evidence-Based Clinical Practice. Chicago, IL:
   American Medical Association; 2001 to the Medical Literature:.
                     Case-Control Studies
• In contrast to RCTs and cohort studies,
  participants are selected based on the presence
  of the outcome rather than the exposure
• Exposure status is determined retrospectively

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001.
Case-Control Studies: Design

Select Subjects:         Cases                               Controls
                         (diseased)                          (nondiseased)




Observe:           Exposed       Not Exposed             Exposed          Not Exposed




               Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
               Clinical Practice. Chicago, IL: American Medical Association; 2001.
               Case-Control Studies:
                   Advantages
• Much more efficient for investigation
  of rare outcomes
• Take less time to perform than RCTs
  or cohort studies

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based
   Clinical Practice. Chicago, IL: American Medical Association; 2001.
                   Case-Control Studies:
                      Disadvantages
• Retrospective assessment of exposure may be
  inadequate (recall bias)
• Can be performed only after outcomes have occurred
  (ie, after damage has already occurred)
• Selection of appropriate controls may be difficult
• Control of confounding through statistical analysis may
  be inadequate

Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical
   Practice. Chicago, IL: American Medical Association; 2001.
        Steps of EBM-5 A’s

•   Ask
•   Acquire
•   Appraise
•   Apply
•   Assess
Applying EBM
Clinical                Patient Values
Expertise               and
                        Preferences

              Best
            Available
            Evidence

Quality                     Costs
of Life
    Integrates Evidence With

• Clinical expertise
    – Experience
    – Judgment
•   Patient values and preferences
•   Quality of life
•   Costs
•   Other important factors
     Integrates Evidence With

• Clinical expertise
    – Experience
    – Judgment
•   Patient values and preferences
•   Quality of life
•   Costs
•   Other important factors
                     “Therapy Review” -
VALIDITY
•    Clearly focused question?
•    Randomization
•    Blinding- subjects, providers, investigators
•    Groups similar at start and treated the same throughout?
•    Followed in randomized groups and accounted for at end? (intention to
     treat)
•    Enough subjects to minimize chance differences?

REUSLTS AND PRECISION
1.   What are results? How presented?
2.   Certainty & precision? (95% CI’s)

APPLICABILITY
1.   Can the results be applied to my patient?
2.   All important outcomes addressed?
3.   Should there by change in policy?
      “Therapy”: Intention to treat

• Subjects are analyzed in the groups they
  were randomized to.
  –   Maintains randomization
  –   Better reflects real world outcomes
  –   Measures efficacy (“Will this work?”)
  –   Detects issues about intervention other than
      effectiveness “In the best possible
      circumstances, do they work?”
             “Therapy”: Bias

• Randomization helps lessen patient bias
  – Self-selection

• Blinding helps lessen patient and
  investigator bias
    “Therapy”: What Are the Results ?
• RR OR RRR
• ARR
• NNT / NNH
• P value/ CI
• Clinically significant?
    “Therapy”: What Are the Results ?

• RR OR RRR
• ARR
• NNT / NNH
• P value/ CI
• Clinically significant?
   “Therapy”: Expressing Results
Risk = outcome event rate

       = number having event
         number receiving the intervention


Relative risk = risk in intervention group
  (RR)            risk in control group


Relative risk reduction (RRR) = 1 - RR
  “Therapy”: Expressing Results


Absolute risk reduction (ARR)
 = difference in risk (control – intervention)
“Therapy”: Expressing Results

Number-needed-to-treat (NNT) = 1/ARR

  in words: the number of patients who
  need to be treated to prevent one
  outcome event from occurring in
  specified time
      “Therapy”: Example

• An oncology trial testing a new
  treatment with 4-year follow-up for
  mortality
• experimental treatment: 30%
• control group:             50%

• What are the RR, RRR, ARR, NNT?
                   “Therapy”: Example
RR = risk of death in experiment/control groups
      = 30%/50% = 0.6 or 60%
RRR = 1 - RR = 1-0.6 = 0.4 or 40%
ARR       = risk of death in control – experimental groups
      = .50 -.30 = 0.2 or 20%
NNT =1/ARR = 1 ÷ 0.2
      = 5 patients treated with the experimental therapy to prevent one death at 4
  years
     “Therapy”: Relative Versus
         Absolute Benefits
• Consider
  – July 3, 2002: Worldcom stock rose 120%
    (relative increase)
  – The stock rose from: $0.10  $0.22 (absolute
   increase)
“Therapy”: Relative risk reduction versus
          absolute risk reduction

• Baseline risk   10/100        5/100
  RRR = 50%       ARR = 5%     NNT = 20

• Baseline risk   1/100         0.5/100
  RRR = 50%       ARR = 0.5%   NNT = 200


• Baseline risk   0.1/100       0.05/100
  RRR = 50%       ARR = 0.05% NNT = 2000
       “Therapy”: RRR Lipid Trials
     4S WOSCOPS CARE AFCAPS
           for acute myocardial infarction
RRR 27           31           25             40
(%)
NNT 19           42           40             435
(5 year)
   “Therapy”: 95% Confidence Interval
• Any statistic only an estimate of the “true value” of that
  statistic.
• Confidence Interval (CI) gives range within which that “true
  value” probably lies.
• 95% CI - if we repeated the experiment with similar
  populations an infinite number of times, the results would fall
  within the CI 95% of the time. 95% certain that the “true
  value” will fall within the 95% CI range.
• CI gives us an idea of the precision of the result, since the
  narrower the CI is, the more certain we can be that the
  experimental value is close to the “true value”.
• And, generally, the larger the sample size, the narrower the CI
• CI =idea of significance, e.g.
    – If the 95% CI for the ARR includes 0, no difference between the
      experimental and control groups.

    – If the 95% CI for the RRR or Odds Ratio includes 1,
    – no difference between the experimental and control groups.

• Similar to P values (e.g., P<0.05) =statistically significant

• CI gives a sense of the size of the differences found in the study.

• e.g., research study - 50% of patients treated with Drug A are cured,
  compared with 45% of patients treated with Drug B.
         • ARR I s thus 5%.
         • Statistical analysis P<0.001, statistically significant.
         • But if 95% CI of ARR is 0% to 10%, indicates result is not clinically
           significant (includes “0%” - “no difference”).
      “Therapy”: Statistical Significance 
                   Clinical Significance

• Are the results clinically important?
   – Duration of pharyngitis: 8.1 days to 7.4 days

   – Weight: 279 lbs to 266 lbs after 3 months

   – Survival increased from 4.5 mos to 5.2 mos with 100% mortality at
      12 months

   – Claudication: Increase in walking distance by 34 ft.
          “Diagnosis Review” - Worksheet
VALIDITY
•    Clearly focused question?
•    Appropriate reference standard?
•    Reference standard & test applied to all subjects? (verification bias)
•    Did results of standard influence interpreting test results? (review bias)
•    Disease status reported and varied? (spectrum bias)
•    Test method reported with sufficient detail to be replicated?

REUSLTS AND PRECISION
1.   What are results?
2.   Certainty & precision? (95% CI’s)

APPLICABILITY
1.   Can the results be applied to my patient?
2.   Are local resources (equipment, expertise, cost) sufficient to apply these
     results?
       “Diagnosis”: 2X2 table
    Diagnostic test characteristics
• Sensitivity
• Specificity
• Predictive Value
• Likelihood Ratios
               DIAGNOSTIC TEST

                DISEASE
                Present   Absent   TOTALS

T   Test        True     False  All positive
    positive  Positive Positive
E
    Test       False       True All
S   negative Negative Negative negative
             All with  All      Entire
T   TOTALS disease     without  population
                       disease
“Diagnosis”: What are the Results?

            DISEAS E

            (+)       (-)
   T (+)
                  a         b
   E
   S (-)
                  c         d
   T
“Diagnosis”: What are the Results?
              Pt has   Dz (-)
             disease
              Dz (+)
   Test(+)
                   a            b



   Test(-)
                   c            d
“Diagnosis”: What are the Results?
                 Pt has   Dz (-)
                disease
                 Dz (+)
      Test(+)
                      a            b



      Test(-)
                      c            d
“Diagnosis”: What are the Results?

               Dz (+)       Pt does
                            not have
                            disease
                             Dz (-)
     Test(+)
                        a          b


     Test(-)
                        c          d
“Diagnosis”: What are the Results?
           Pt has   Dz (-)
          disease
           Dz (+)
Test(+)
                a            b



Test(-)
                c            d
“Diagnosis”: What are the Results?
           Pt has   Dz (-)
          disease
           Dz (+)
Test(+)
                a            b
           TP


Test(-)
           FN   c            d
          “Diagnosis”: Sensitivity
Sensitivity is proportion of people with disease who
                   have a positive test

             Dz (+)       Dz (-)
Test(+)
                      a            b
               TP
Test(-)
                      c            d
             FN


Sensitivity = (a/a+c) =(TP/TP+FN)
“Diagnosis”: What are the Results?

          Dz (+)       Pt does
                       not have
                       disease
                        Dz (-)
Test(+)
                   a          b


Test(-)
                   c          d
“Diagnosis”: What are the Results?

          Dz (+)       Pt does
                       not have
                       disease
                        Dz (-)
Test(+)
                   a          b


Test(-)
                   c          d
“Diagnosis”: What are the Results?

          Dz (+)       Pt does
                       not have
                       disease
                        Dz (-)
Test(+)
                   a          b


Test(-)                  FP
                   c          d


                        TN
           “Diagnosis”: Specificity
Specificity is proportion of people without disease who have negative test

                         Dz (+)        Dz (-)
          Test(+)
                                  a             b
                                       FP

           Test(-)
                                  c             d
                                       TN

       Specificity = (d/b+d) =(TN/FP+TN)
“Diagnosis”: Tradeoffs of sensitivity &
      specificity labeling diabetes
Blood sugar      Sensitivity    Specificity

•   70           98.6%         8.8%
•   100          88.6%         69.8%
•   130          64.3%         96.9%
•   160          47.1%         99.8%
•   200          27.1%         100%
  “Diagnosis”: Choosing a test

• SnNout-
 A sensitive test, if negative, rules out a
 disease
• SpPin-
 A specific test, if positive, rules in a disease
   “Diagnosis”: Sensitivity & Specificity

• Useful for picking a test (test properties)
  – Screening- prefer sensitive test
  – Diagnosis – prefer specific test
• Less help in making diagnosis
        “Diagnosis”: What are the Results?

• In patients, what you know are their test results- you are trying to
  determine whether they actually have the disease.
• Positive Predictive Value :
   – Of all who tested positive for a disease, the proportion that actually has it

• Negative Predictive Value :
   – Of all who tested negative for a disease, the proportion that actually does
     not have it
“Diagnosis”: What are the Results?

         DISEAS E

         (+)       (-)
T (+)
               a         b
E
S (-)
               c         d
T
  “Diagnosis”: Positive Predictive Value

          Proportion of people with a positive test
          Dz (+) Dz (-)         who have a disease

Test(+)     TP       FP
              a       b           PPV =
                                    a/a+b=
Test(-)                                          TP/TP+FP
              c       d            = true positives
                                            over all positives
   “Diagnosis”: Negative Predictive Value
          Proportion of people with a negative test
                              who don’t have a disease
          Dz (+) Dz (-)
Test(+)             FP            NPV=
              a       b             d/d+c=
Test(-)      FN     TN                   TN/TN+FN
              c       d              = true negatives
                                         over all positives
          “Diagnosis”: What are the Results?
 PPV dependent on prevalence, even when using the
                     same test
• Example: Prevalence of a particular disease in a population
  is 50%. Sensitivity= 90% Specificity= 95%

          Dz (+) Dz (-)
Test(+) 90         5                PPV: = a/a+b
               a         b
Test(-)                               = 95%
          10   c
                   95    d
          100      100       200
              “Diagnosis”: PPV & prevalence
    PPV dependent on prevalence using same test
• Example: Prevalence of a particular disease in a population
  is 5%. Same test, same sensitivity & specificity
    – Sensitivity= 90% Specificity= 95%


          Dz (+) Dz (-)
Test(+)                                   PPV: = a/a+b
          9         10
                a         b
Test(-)                                    = 47%
          1     c
                    180   d
           10       190       200
                  “Diagnosis”: PPV & NPV
•       Useful for diagnosis
    –     Probability of disease after (+ ) or (–) test

•       Drawbacks:
    –     Sensitive to prevalence of disease
    –     Prevalence of disease in general population may not be
          the same as that of patients you see in clinic/ER.
    –     Not all test results can be categorized as “+” or “-”.

For these reasons, some consider PPV & NPV “Old
    School”.
    “Diagnosis”: What are the Results?
                Likelihood Ratios
• Likelihood Ratio is how much more likely is it
  that someone with this finding has the disease,
  compared to someone who doesn’t. It does NOT
  vary with prevalence.

Technically, the + LR is how much more likely
  someone is to get any positive test result if they
  have disease, compared to someone who
  doesn’t.
“Diagnosis”: Likelihood ratio


LR =    SENSITIVITY
        1 - SPECIFICITY
                     “Diagnosis”:
  What do all the numbers mean?
The Likelihood Ratio is a diagnostic weight;
  It tells you by how much a given diagnostic test result will
   raise or lower the probability of having the disorder.
Pretest Probability: the chance that the pt has disease, prior
  to ordering any tests. This is often an estimation based on
  clinical experience
Post-test Probability: the chance that the pt has disease,
  given the results of the test
    “Diagnosis”: What are the Results?

What do all the numbers mean?
A LR of 1.0 means the post-test probability is exactly the
   same as the pretest probability.

A LR >1.0 increases the probability of having the disorder.

A LR<1.0 decreases the probability of having the disorder.
    “Diagnosis”: What are the Results?

Likelihood ratios >10 or <0.1 generate large changes from pre- to
   post-test probability and are generally considered significant.
        Strong evidence to rule in/rule out a diagnosis.

Likelihood ratios of 5-10 and 0.1-0.2 generate moderate changes
   in probability.
        Moderate evidence to rule in/rule out a diagnosis.

Likelihood ratios of 2-5 and 0.2-0.5 generate small changes.
        Minimal evidence to rule in/rule out a diagnosis

Likelihood ratios 0.5-2 usually have little effect
Figure 1a:
Likelihood Ratio
Nomogram
            LRs = Diagnostic Weights
Likelihood Ratio                                Change in
                                                probability

Values greater than 1                10         +45%
INCREASE probability of              5          +30%
disease
                                     2          +15%
                                     1.0        0
Values between 0 and 1               0.5        -15%
DECREASE probability of              0.2        -30%
disease
                                     0.1        -45%

        From Steve McGee, Evidence Based Physical Diagnosis
Table 2a:
Likelihood Ratios of
Tests for the
Diagnosis of
Appendicitis
            Likelihood ratio
• DVT
  – Homan’s sign       +LR 1.5
  – Doppler                  + LR 39


• ANEMIA
  – Conjunctival rim pallor +LR 16.7
        Summary Diagnostic Test
                DISEASE
               Present       Absent        TOTALS
T                                                   P
    Positive   True     False      ALL
                                                    P
E            Positive Positive  Positives           V

S   Negative   False     True  ALL                  N
             Negative Negative Negatives            P
                                                    V
T
              All with    All  Entire
    TOTALS   disease   without population
                       disease
               Sensitivity   Specificity
                   Common pitfalls
• Results reported as relative risk
   – (ex. Migraines, CVA & OC)
• Results that came from recalculating the data after
  trial was done
   – (Post-hoc analysis) (ex. Hot study)
• Over-interpreting results
• Relying on just one study (ex. Mg for heart dz),
  a poor study, or wrong type of study (ex. HRT)
• Confusing statistical significance with clinical
  significance     (ex. Drugs for BPH)
• Not looking at CI’s
• Not considering who funded study
        Steps of EBM-5 A’s

•   Ask
•   Acquire
•   Appraise
•   Apply
•   Assess
   Core of EBP

“Supposing is good
  but finding out
    is better.”
        Mark Twain

				
DOCUMENT INFO