Yes go on No stop

Document Sample
Yes go on No stop Powered By Docstoc
					               Objectives
•   Introduction to RCTs
•   Level of evidences
•   Three Step Guide in Using an Article
    to Assess Therapy
                         Indications
        • RCTs are the ideal study design to answer
          questions related to the effects of health care
          interventions which are small to moderate

        • Any clinical maneuver offered to the study
          participants that may have an effect on their health
          status (that is, preventive strategies, screening
          programmes, diagnostic tests or educational
          models).

EBMRC                           Dr.Soltani                       EMRC
         Testing Interventions - Stages
        • Phase I – 1st stage using humans (case series20-80)
           u   Evaluation of basic safety, metabolism, doses, route
           u   Healthy volunteers used
           u   Unblinded, uncontrolled
        • Phase II (20->100)
           u   Small randomized, controlled, blinded
           u   Tests tolerability and different doses
                • E.g., optimal dosage without side effects
           u   Applied to patients with relevant illness
           u   Goal - Identify suitable formulation of drug and desired effect
               shrinkage of a measurable tumor, a favorable effect on a biomarker
           u   Can we go to next steps?


EBMRC                                     Dr.Soltani                                EMRC
                    Intervention Trials
        • Phase III
          u   Referred to as clinical trial
          u   Evaluation of efficacy of drug, involve definitive
              endpoint
          u   Usually randomized, blinded, controlled trial
          u   If successful, licensed and marketed
        • Phase IV
          u   Large studies after approval of drug
          u   Often observational, study long-term effects
               • effectiveness” of proven interventions in wide-scale use,
                 rate of serious side effects
          u   Evaluate drug in “real life”, additional uses

EBMRC                                 Dr.Soltani                             EMRC
        Randomised controlled trial
                 design

        • Advantages
        • Allows rigorous evaluation of a single variable in a
          precisely defined patient group

        • Prospective design

        • Potentially eradicates bias by comparing two
          otherwise identical groups

        • Allows for meta-analysis
EBMRC                          Dr.Soltani                        EMRC
             Randomised controlled trial
                      design
        • Disadvantages
        • Expensive and time consuming; hence, in practice:

        • Many randomised controlled trials are either never
          done, are performed on too few patients, or are
          undertaken for too short a period

        • Most are funded by large research bodies or drug
          companies, who ultimately dictate the research agenda



EBMRC                           Dr.Soltani                        EMRC
               Objectives
•   Introduction to RCTs
•   Level of evidences
•   Three Step Guide in Using an Article
    to Assess Therapy
              The hierarchy of evidence
        •   Systematic reviews and meta-analyses
        •   Randomised controlled trials
        •   Cohort studies
        •   Case-control studies
        •   Cross sectional surveys
        •   Case reports.
        •   Expert opinion (CME, Lectures, Rounds)



EBMRC                       Dr.Soltani               EMRC
               Objectives
•   Introduction to RCTs
•   Level of evidences
•   Three Step Guide in Using an Article
    to Assess Therapy
        Three Step Guide in
         Using an Article to
          Assess Therapy


EBMRC           Dr.Soltani     EMRC
        Critical appraisal is a mixture
        of the science of examining
        validity and the art of assessing
        utility.


EBMRC                 Dr.Soltani            EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
EBMRC   Dr.Soltani   EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
                    Simple randomization
        •   An almost infinite number of methods can be used to generate a simple
            randomisation sequence based on a random-number. For example, for
            equal allocation to two groups,predetermine the direction to read the
            table: up, down, left, right, or diagonal. Then select an arbitrary
            starting point—ie, first line, 7th number:
                        56 99 20 20 52 49 05 78 58 50 62 86 52 11 88
                        31 60 26 13 69 74 80 71 48 73 72 18 60 58 20
                                       55 59 06 67 02 . . .
        •   For equal allocation, an investigator could equate odd and even
            numbers to interventions A and B, respectively.
        •   Therefore, a series of random numbers 05, 78, 58, 50, 62, 86, 52, 11,
            88, 31, &c, represent allocation to intervention A,B, B, B, B, B, B, A,
            B, A, &c.
        •   Alternatively, 00–49 could equate to A and 50–99 to B, or numbers 00
            –09 to A and 10–19 to B,ignoring all numbers greater than 19.
        •   Any of a myriad of options suffice, provided the assignment
            probabilities and the investigator adhere to the predetermined scheme.

EBMRC                                    Dr.Soltani                                   EMRC
        Benefits of Random Allocation
                      (Randomization)
        1.Reduces bias in those selected for
          treatment
          u   guarantees treatment assignment will not be
              based on patients’ prognosis

        2.Prevents confounding
          u   known and unknown potential confounders are
              evenly distributed


EBMRC                          Dr.Soltani                   EMRC
        Benefits of Random Allocation
               (Randomization)




EBMRC               Dr.Soltani          EMRC
        Benefits of Random Allocation
               (Randomization)




EBMRC               Dr.Soltani          EMRC
        Benefits of Random Allocation
               (Randomization)




EBMRC               Dr.Soltani          EMRC
          Was assignment to exposure &
         comparison groups randomised?
        • The method of randomisation should be described
          (e.g. using a computer algorithm).
        • Non-randomized concurrent controls
           u   Alternation
           u   Odd/Even hospital no. or date of birth
           u   First letter of surname
        • Difficult to argue that one group is different from
          another but allocation is predictable, so bias can
          arise from selection of patients: Keirse (1988)
           u   so randomization must be unpredictable
EBMRC                               Dr.Soltani                  EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
        Is allocation concealed?




EBMRC             Dr.Soltani       EMRC
                  Ensuring Allocation
                     Concealment
        BEST – most valid technique
        § Central computer 
        randomization

        DOUBTFUL
        § Envelopes, etc


        NOT RANDOMIZED
        § Date of birth, alternate days, etc
EBMRC                             Dr.Soltani   EMRC
            Do Not Confuse Allocation
            Concealment with Blinding

        • Allocation concealment seeks to
          prevent selection bias, protects
          assignment sequence before and until
          allocation, and can always be
          successfully implemented




EBMRC                    Dr.Soltani              EMRC
EBMRC   Dr.Soltani   EMRC
               Importance of allocation
                    concealment
        • Trials that used inadequate or unclear allocation
          concealment, yielded up to 40% larger estimates
          of effect.
        • Moreover, the worst concealed trials yielded
          greater heterogeneity in results—ie, the results
          fluctuated extensively above and below the
          estimates from better studies.
        • Indeed, having a randomized (unpredictable)
          sequence should make little difference without
          adequate allocation concealment.
EBMRC                          Dr.Soltani                     EMRC
           Do Not Confuse Allocation
        Concealment with Blinding (Cont’d)


        • Blinding seeks to prevent information
          bias, protects sequence after
          allocation, and cannot always be
          successfully implemented




EBMRC                    Dr.Soltani               EMRC
              Importance of allocation
                   concealment

        • Moreover, the results of such a trial can be
          more damaging than similar results from an
          explicitly observational research study.

        • Biases are usually assumed and
          acknowledged in observational studies.



EBMRC                       Dr.Soltani                   EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
                         Placebo effect
         Trial in patients with chronic severe itching


                                        No treatment



                         Trimeprazine
                            tartrate
        Cyproheptadine
            HCL




         Treatment vs no treatment for itching


EBMRC                                   Dr.Soltani       EMRC
                             Placebo effect
             Trial in patients with chronic severe itching


                                            No treatment



                             Trimeprazine
                                tartrate
            Cyproheptadine                                 Placebo
                HCL




             Treatment vs no treatment vs placebo for itching

        Placebo effect - attributable to the expectation that
EBMRC
        the treatment will have an effect
                                  Dr.Soltani                         EMRC
                 Acupuncture in back pain
                                                               • Randomised studies
                                                               • Compared with sham
                                                                 acupuncture
                                                               • Validity of acupuncture
                                                                 checked independently
                                                               • 12 trials in total, 9 with
                                                                 outcomes
                                                               • 4 were blind (250 patients)
                                                               • 5 were not blind (204
                                                                 patients)
Ernst & White. Arch Intern Med 1998 158:2235-41
  EBMRC                                           Dr.Soltani                               EMRC
                           Blinding
        • Single blind: (subject or investigator): one of
          the three categories of individuals (normally
          participant rather than investigator) remains
          unaware
        • Evaluate surgical interventions.
        • In addition, two surgical procedures could be
          compared under single-blind conditions with
          the use of identical wound dressings to keep
          investigators blind to the type of procedure
          during the assessment of the outcomes

EBMRC                         Dr.Soltani                    EMRC
        Double-Blinded           Single-Blinded



EBMRC               Dr.Soltani                    EMRC
                            Blinding
        • Double-blind trial: (subject and observer):
          participants, investigators, and assessors usually
          all remain unaware of the intervention
          assignments throughout the trial.
        • Triple blind: (subject, observer, and data reviewer)
          usually means a double-blind trial that also
          maintains a blind data analysis.




EBMRC                          Dr.Soltani                        EMRC
EBMRC   Dr.Soltani   EMRC
EBMRC   Dr.Soltani   EMRC
    Blinding and Reporting
• Usually reduces differential assessment of
  outcomes (information bias)
•       Authors should explicitly state who was blinded –
        and how.
•       Many investigators and readers consider a
        randomized trial as high quality simply because it
        is “double-blind,” as if double-blinding is the sine
        qua non of an RCT.
        u   Trials not “double-blinded” should not
            automatically be deemed inferior trials.
EBMRC                         Dr.Soltani                  EMRC
        Descriptions of blinding




EBMRC             Dr.Soltani       EMRC
        Descriptions of blinding




EBMRC             Dr.Soltani       EMRC
        Descriptions of blinding




EBMRC             Dr.Soltani       EMRC
        Descriptions of blinding




EBMRC             Dr.Soltani       EMRC
        Blinding in randomised trials: hiding
                    who got what
        • Double blinding prevents bias but is less important,
          than adequate allocation concealment.
        • open studies are more likely to favour experimental
          interventions over the controls and that studies that are
          not double-blinded can exaggerate effect estimates by
          17%

        • Furthermore, in some trials, blinding cannot be
          successfully implemented, whereas allocation
          concealment can always be successfully implemented.
EBMRC                           Dr.Soltani                      EMRC
                   Placebo
        Blinding frequently necessitates
         the use of placebos ( The best
           strategy to achieve blinding
          during data collection is with
               the use of placebos)


EBMRC                Dr.Soltani            EMRC
                         Placebo
        • Placebos are more difficult to develop and
          implement successfully in non-drug trials.
          For example, it is difficult to develop and
          implement placebo physiotherapy,
          acupuncture, or electrical stimulation
        • An active placebo is a placebo with
          properties that mimic the symptoms or side-
          effects eg, dry mouth, sweating—that might
          otherwise reveal the identity of the
          (pharmacologically) active test treatment.
EBMRC                       Dr.Soltani                  EMRC
               Algorithm :Randomization
        •   Yes
        •   Comparable?
        •   Yes: Ok
        •   No
        •   Significant Difference?
        •   Statistical: Ok
        •   Clinical:?
        •   Adjustment?
        •   Yes: Ok
        •   No:
        •   Discussion
        •   admit: Ok
        •   No admit: Not Ok
EBMRC                                 Dr.Soltani   EMRC
                   Algorithm :Blinding
    •   Yes
    •   No
    •   Outcome
    •   Objective
    •   Ok
    •   Subjective
    •   Not Ok
    •   Both
    •   Significance
    •   ONLY Objective
    •   Classification For Subjective Outcome
    •   Accurate: Yes Not Ok
    •   Not Accurate : Ok
    •   ONLY Subjective
    •   Not Ok
    •   Both
    •   Ok

EBMRC                                Dr.Soltani   EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
            Selection of Participants
        u   Terminology
            •   Target population
                – People to which findings will be generalized
                – Population from which sample is selected
                – Ex: Diabetic patients in city-country
            •   Study Sample
                – Subset of population available to study
                – Subjects in the study
                – Ex: diabetic patients in DM clinic
            •   Selecting subjects
                – Establish inclusion/exclusion criteria
                – Sample size
EBMRC                             Dr.Soltani                     EMRC
          What were the key selection (inclusion & exclusion)
        criteria? Were they well defined? Were they replicable?


        • List important selection criteria; e.g. age
          group, gender, risk profile, medical history.
          Usually in Methods section.
        • Logic:
        • There should be sufficient information in
          the paper (or referenced) to allow the reader
          to theoretically select a similar population


EBMRC                           Dr.Soltani                        EMRC
EBMRC   Dr.Soltani   EMRC
EBMRC   Dr.Soltani   EMRC
               Baseline comparisons

 • Logic:
 • Such information allows readers to see the possibilities of
   generalisation to other populations.
 • Furthermore, it allows physicians to infer the results to
   particular patients.
 • Compare baseline characteristics between treatment
   groups to discover possible confounders: randomisation
   will produce very similar baseline statistics if the sample
   size is large


EBMRC                      Dr.Soltani                      EMRC
        Comparison of baseline data




                Does P>0.05
                  indicate
              comparability of
            treatment groups?



                                   Chan et al. Lancet 1997
EBMRC                 Dr.Soltani                     EMRC
EBMRC   Dr.Soltani   EMRC
        Significance tests for baseline differences




                                          T E
                                        IA
                                    O PR
                             P R
                     AP
                 I N

                                           Chan et al. Lancet 1997
EBMRC                  Dr.Soltani                            EMRC
                      Baseline data
           Effect of azathioprine on the survival of patients
           with primary biliary cirrhosis

                                                Azathioprine     Placebo
        Mean age                                    54.7            54.9
        Serum bilirubin (mmol/L)                    37.2            30.9
        Stage I disease %                            14              12
        Stage II disease %                           44              43
        Stage III disease %                          15              15
        Stage IV disease %                           27              30
                                                      Christensen et al. Gastro 1985
EBMRC                              Dr.Soltani                                     EMRC
                      Baseline data
           Effect of azathioprine on the survival of patients
           with primary biliary cirrhosis

                                                Azathioprine     Placebo
        Mean age                                    54.7            54.9
        Serum bilirubin (mmol/L)                    37.2            30.9
        Stage I disease %                            14              12
        Stage II disease %                           44              43
        Stage III disease %                          15              15
        Stage IV disease %                           27              30
                                                      Christensen et al. Gastro 1985
EBMRC                              Dr.Soltani                                     EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
         How complete was the follow up?
         How many dropouts were there?
        • Conventionally, a 20% drop out rate is
          regarded as acceptable, but this depends on
          the study question.
        • Some regard should be paid to why
          participants dropped out, as well as how
          many.
        • It should be noted that the drop out rate may
          be expected to be higher in studies conducted
          over a long period of time.
        • A higher drop out rate will normally lead to
          downgrading, rather than rejection of a study.
EBMRC                        Dr.Soltani                    EMRC
             Bias: a one-sided inclination
                      Of the mind
                         Over-estimation of
                          treatment effect

        •   Not random                   40%
        •   Not double-blind             17%
        •   Duplicate information        20%
        •   Small trials                 30%
        •   Poor reporting quality       25%

EBMRC                       Dr.Soltani         EMRC
Assess Validity and Applicability to
        my practice setting

1.Is the study a randomized control trial (RCT)?
               Yes (go on) No (stop)
2.Were the patients properly selected for the trial and
    randomized with concealed assignment?
              Yes (go on) No (stop)
3.Were patients and study personnel “blind” to treatment?
              Yes (go on) No (pause)
4.Were the intervention and control groups similar at the
    start? (Check “Table 1” of most studies)
              Yes (go on) No (stop)
5.Was follow-up complete?
ii. Were patients analyzed in the groups to which they were
    randomized (“intention-to-treat” analysis)?
          How do we get and maintain
             comparable groups?
        • Achieved by random allocation
          (randomization)

        • Checked by analysis of baseline data

        • Maintained by 100% follow-up
          u Follow all patients
          u No exclusions from analysis (intention to

            treat)
EBMRC                        Dr.Soltani                 EMRC
              Intention to treat analysis

        • Some pt.s may forget or refuse to take
          their medication or they’re too sick or
          suffer the outcome of interest before
          taking medication / surgery.




EBMRC                     Dr.Soltani                EMRC
   Intention to treat




             Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
EBMRC                  Dr.Soltani                            EMRC
   Intention to treat




             Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
EBMRC                Dr.Soltani                              EMRC
   Intention to treat
        High risk? 




                      Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
EBMRC                         Dr.Soltani                              EMRC
          Intention-to-Treat Principle
                 Maintaining the randomization

        Principle: 
           Once a patient is randomized, s/he should be 
           analyzed in the group randomized to - even if they 
           discontinue, never receive treatment, or crossover.


          Exception: If patient is found on BLIND reassessment
           to be ineligible based on pre-randomization criteria.




EBMRC                            Dr.Soltani                        EMRC
                 Intention to treat analysis
        • Analysis which includes all randomized patients
           in the groups to which they were randomly
           assigned
             regardless of :
         1- their compliance with the study entry criteria
         2- the treatment they actually received
         3- subsequent withdrawal from treatment
         4- deviation from protocol



EBMRC                         Dr.Soltani                     EMRC
                   Intention to treat analysis
                                  benefits
        • Gives an unbiased estimate of the true effect (the
          only non-bias way to analyze study results)
        • Conservative estimate – tends to underestimate
        • Analysis that is supported by the randomization
              and maintains randomization
        ( known & unknown prognostic factors will be equally
           distributed in 2 groups & observed effect is just due to the
           treatment assigned)




EBMRC                              Dr.Soltani                             EMRC
                 Intention to treat analysis
                            drawbacks

        • Unfortunately, applying the intention-to-treat
          principle doesn’t solve the problem When
          treatment is effective but non-adherence is
          substantial
        • Intention-to-treat principle underestimates the
          magnitude of the treatment effect that will occur
          in adherent patients.




EBMRC                          Dr.Soltani                     EMRC
    Exercise (ITT,PPA)




           Hollis S and Campbell F     BMJ 1999 319 p670
EBMRC                     Dr.Soltani                       EMRC
    Answer




         Hollis S and Campbell F     BMJ 1999 319 p670

EBMRC                   Dr.Soltani                       EMRC
        Subgroup analysis: Sceptical unless:

  • the subgroups make biological and clinical
    sense?
  • the differences are both clinically &
    statistically significant?
  • was a-priori hypothesis (before this data)?
  • other evidence supports these sub-groups?
  • few (OK) or many (nix) sub-group analyses?


EBMRC                  Dr.Soltani              EMRC
                          Summary

        1.Is the study a randomized control trial (RCT)?
                        Yes (go on) No (stop)
        2.Were the patients properly selected for the trial and
            randomized with concealed assignment?
                       Yes (go on) No (stop)
        3.Were patients and study personnel “blind” to
            treatment?
                       Yes (go on) No (pause)
        4.Were the intervention and control groups similar at the
            start? (Check “Table 1” of most studies)
                       Yes (go on) No (stop)
        5.Was follow-up complete?
EBMRC                          Dr.Soltani                     EMRC
        Jadad Criteria for Scoring RCTs
                   (1997 Cont Clin Trials 17:1-12)

        • 1. Randomization
             • Appropriate (= 1 point) if each patient had equal chance of
               receiving intervention and investigators could not predict
             • Add 1 point if mechanism described and appropriate
             • Deduct 1 point if mechanism described and inappropriate
        • 2. Double blinding
             • Appropriate (= 1 point) if stated that neither the patient
               nor investigators could identify intervention, or if “active
               placebo”, “identical placebo” or “dummies” mentioned
             • Add 1 point if method described and appropriate
             • Deduct 1 point if mechanism described and inappropriate
        • 3. Withdrawals and dropouts
             • Appropriate (= 1 point) if number and reasons for loss-to-
               FU in each group described.
EBMRC                               Dr.Soltani                                EMRC
                      Expert opinion

            Was the study described randomization? (0-1)
            Was the study described double blinding? (0-1)
            Was there a description of withdrawal and drop-out?(0-1)


    Appropriate randomization                 Inappropriate randomization
    (concealed) blinding? (+1)                Blinding? (-1)


                             Scoring range 0-5
                             Poor quality < 3

EBMRC                            Dr.Soltani                             EMRC
   Other Issues – Quality Scoring
    • Quality is difficult to measure
    • No consensus on method of scale development –
      not even for RCT’s
    • Few reliability/validity studies of scoring systems
        u   inter-rater reliability of quality assessment often poor
    • Relies on quality of the reporting itself
        u   sometimes study is blinded or randomized, but if not
            explicitly stated then it suffers in quality assessment
    • Difficult to detect bias from publications
    • More recent studies score higher – partly because
      they conform to recent standardized reporting
      protocols (e.g., RCT’s – CONSORT)
EBMRC                              Dr.Soltani                          EMRC
     Study 1: MRC/BHF Heart Protection Study of
        Antioxidant vitamin supplementation, …

• Citation: Lancet 2002; 360: 23-33.
• ?: Does Antiox sup (C, E, Beta Carotene) reduce death,
  vascular events or Ca in people with high 5 year risk.
• Design: RCT
• Subjects: 20,536 (40-80 y.o.), 75% male, w high risk
• Setting: 69 UK Hospitals, 5 yr f/u
• Control: Placebo
• Allocation concealed: Y
• Blinded: Pt, Dr, Collectors, Analyzers
• Analysis: ITT
 EBMRC                    Dr.Soltani                  EMRC
    Study 1: MRC/BHF Heart Protection Study of
       Antioxidant vitamin supplementation, …
• Findings: No benefit (in any outcome). All NNH
  and NNT non-significant.
• Limits: One dose, high risk, duration
• Strengths: RCT, # with 99.6% f/u (83 %
  compliance)
• Funding: UK MRC, Brit Heart Found, Merck,
  Roche (*Reassured us analysis was independent of
  funding source)
• Comments: agree other RCT (put to rest a few
  observational and Cambridge heart antioxidant
  study)
EBMRC                   Dr.Soltani                   EMRC
        Thank you

EBMRC      Dr.Soltani   EMRC
          How to Analyze
        Therapy in the Medical
              Literature
                          Akbar soltani. MD, MS
               Tehran University of Medical Sciences (TUMS)
            Endocrine and Metabolism Research Center (EMRC)
            Evidence-Based Medicine research Center (EBMRC)
                             Shariati Hospital
                          www.soltaniebm.com
                                www.ebm.ir
EBMRC                          Dr.Soltani                     EMRC
        Three Step Guide in
         Using an Article to
          Assess Therapy
               (Part-2)


EBMRC           Dr.Soltani     EMRC
EBMRC   Dr.Soltani   EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
          Key components of the statistical analysis plan for the
                primary endpoint or endpoints include:


    •   Specifying how the outcome will be measured. Common measures are:
    •   ■ Binary (whether or not an event has occurred) — for example, whether or
        not the subject has experienced a complete or partial response from cancer
        treatment at 12 months. Typical measures of the event are proportions (risk),
        rates or odds, and measures of treatment effect include odds ratios and
        differences in the proportions (or rates) between the intervention and control
        groups.

    •   ■ Count (the frequency of an event in a set time period) — for example, the
        number of episodes of epilepsy experienced by patients in a 30-day period. A
        typical unit of measurement would be the rate (count per unit time), and
        measures of treatment effect include incidence density ratios (similar to odds
        ratios) or differences between the rates in the groups being compared.




EBMRC                                  Dr.Soltani                                  EMRC
             Key components of the statistical analysis plan for the
                   primary endpoint or endpoints include:


        •   Time to event (how long it takes to observe the outcome of interest) —
            for example, the survival time of patients with advanced breast cancer.
            Endpoints of this type usually contain censored data (ie, the event of
            interest has not been observed by the end of the follow-up period), and
            analyses would involve comparing “averaged” relative risks or hazard/
            risk ratios (pooled across the time period of the study) between the
            groups.

        •   ■ Measurement on a continuous scale. Examples include blood
            pressure and temperature measurements, and analyses involve
            comparing the difference between the means of the intervention and
            control groups.

        •   ■ Other measurements include ordinal scales (eg, quality- of-life
            ratings, 5-point trauma scales) and non-ordered scales (eg, patient
            preferences between oral, intravenous or combination treatment
            delivery). Outcomes measured on these scales require specialised
            statistical methods.
EBMRC                                     Dr.Soltani                                  EMRC
                 Communicating evidence...

        After 3 years, 4 cardiac rehabilitation programmes were
        assessed with RCTs. The results were all significant and
        the costs comparable:
    • Programme A reduced the rate of deaths by 20%
    • Programme B produced an absolute reduction in deaths by
      3%
    • Programme C increased the rate of patient survival from
      84% to 87%
    • Programme D meant that 33 people need to enter a
      rehabilitation programme to prevent one death
EBMRC                          Dr.Soltani   Fahey et al BMJ 1995 311:1056-60   EMRC
                 HOPE STUDY(3)

        A cholesterol-lowering drug reduced coronary
        heart disease. Those treated had 34% fewer 
        heart attacks than the non-treated group.

        Would you wish to be treated?




EBMRC                      Dr.Soltani                  EMRC
         FRAMINGHAM STUDY(2)

        A cholesterol-lowering drug reduced coronary
        heart disease. There were 1.5% fewer 
        heart attacks.

        Would you wish to be treated?




EBMRC                      Dr.Soltani                  EMRC
        DENMARK CAD STUDY(2)

        Studies of a cholesterol-lowering drug 
        showed that if 67 people took it for 5 years
        it would prevent one heart attack. 
        (Two others would have heart attacks anyway)

        Would you wish to be treated?




EBMRC                     Dr.Soltani                   EMRC
                     Measuring Risk:
                          Relative Risk




        Relative Risk (RR) =    rate in exposed       = 0.04 = 0.67
                               rate in nonexposed        0.06

                                                SHEP. JAMA. 1991;265:3255-3264
EBMRC                              Dr.Soltani                                    EMRC
              Communicating risk:
                                                40%


          RRR=?
          ARR=?

                                    20%   20%



             10%        10%
      5%



placebo
  EBMRC                Dr.Soltani                     EMRC
                          Measuring Risk:
                           Relative Risk  4% 6%




        Relative Risk (RR) =    rate in exposed       = 0.04 = 0.67
                               rate in nonexposed        0.06

                                                SHEP. JAMA. 1991;265:3255-3264
EBMRC                              Dr.Soltani                                    EMRC
        Measuring Risk: ARR?



              2%

                                6%
              4%



EBMRC              Dr.Soltani        EMRC
                       Measuring Risk:
                   Absolute Risk Reduction
        Absolute Risk Reduction (ARR) is the absolute
          difference in event rates between the experimental and
          control patients.

        Calculated by:
        ARR = CER - EER = 0.06 - 0.04 = 0.02

        In its decimal form the ARR is not easy to use!
          Converted to a percentage - there is an absolute risk
           reduction of 2%

EBMRC                               Dr.Soltani                     EMRC
        Measuring Risk: RRR?



              2%

                                6%
              4%



EBMRC              Dr.Soltani        EMRC
                        Measuring Risk:
                    Relative Risk Reduction
   Relative Risk Reduction (RRR) is the proportional reduction in event rates
        between the experimental and control patients.

   Two ways to calculate:
   RRR = (1 - RR) = (1 - 0.67) = 0.33
                                OR                        CER = control event rate
   RRR = CER - EER = 0.06 - 0.04 = 0.33                   EER = experimental
             CER               0.06                       event rate

        Therefore, treatment reduced the stroke rate by 33%
        OR a RRR of 33% means that the new treatment reduced the risk of death by
        33% relative to that occurring among control patients




EBMRC                                   Dr.Soltani                             EMRC
         Relative Risk Reduction:

   • Usually reported in studies.
   • Ratio of the improvement of outcome over
     outcome without intervention (Rx):
   • {Control Event Rate (CER) — Experimental
     Event Rate (EER)} / CER
   • i.e. {CER-EER}/CER
   • often independent of prevalence!
   • often similar at different ages!

EBMRC                Dr.Soltani             EMRC
        Absolute Risk—> The risk our
              patient is facing!
   • How likely is our patient to die (or have the
     outcome of interest) without intervention? =
     Control Event Rate (CER)
   • consider this individual patient’s risk factors
     to estimate Patient Expected Event Rate =
     PEER.
   • Absolute Risk usually increases with age.
   • Improvement measured as Absolute Risk
     Reduction (ARR)
EBMRC                   Dr.Soltani                 EMRC
                Usefulness of the ARR:
                   Number Needed to Treat
        Number Needed to Treat (NNT) is the number of
          patients a clinician needs to treat in order to prevent
          one additional adverse outcome.
        NNT is for dichotomous outcomes.


        Calculated by:
        NNT = 1/ARR = 1/0.02 = 50

          Therefore, you would have to treat 50 hypertensive patients to prevent
          one stroke.



EBMRC                                  Dr.Soltani                                  EMRC
                  Number Needed to Treat
        CER=0.06=100----------6
        EER=0.04=100----------4
        SO:     100-----------2
                X------------1
                50-----------1
        Calculated by:
        NNT = 1/ARR = 1/0.02 = 50

          Therefore, you would have to treat 50 hypertensive
          patients to prevent one stroke.
EBMRC                             Dr.Soltani                   EMRC
                Communicating risk
        • For a patient fitting the inclusion criteria for
          the X trial, your message would then be:
        • “If 100 people like you are given no treatment
          for five years 94 will live and six will die.
          Whether you are one of the 94 or one the six, I
          do not know.
        • Then, if 100 people like you take a certain drug
          every day for five years 96 will live and four
          will die. Again, I do not know whether you are
          one of the 96 or one of the four.

EBMRC                         Dr.Soltani                     EMRC
        Risk of coronary death
          (based on Helsinki Heart Study)




EBMRC                 Dr.Soltani            EMRC
                Presentations of Risks
        • Relative Risk: 0.029/0.044 =0.66

        • Relative Risk Reduction: 1 - 0.66 = 0.34 (34%)

        • Absolute Risk Reduction: 0.044 - 0.029 = 0.015(1.5%)

        • Number Needed to Treat: 1/0.015 = 67



EBMRC                            Dr.Soltani                      EMRC
                   Risk Reduction:
                   Expressed as RRR

        A cholesterol-lowering drug reduced coronary
        heart disease. Those treated had 34% fewer 
        heart attacks than the non-treated group.

        Would you wish to be treated?




EBMRC                      Dr.Soltani                  EMRC
                   Risk Reduction:
                   Expressed as ARR

        A cholesterol-lowering drug reduced coronary
        heart disease. There were 1.5% fewer 
        heart attacks.

        Would you wish to be treated?




EBMRC                      Dr.Soltani                  EMRC
                  Risk Reduction:
                  Expressed as NNT

        Studies of a cholesterol-lowering drug 
        showed that if 67 people took it for 5 years
        it would prevent one heart attack. 
        (Two others would have heart attacks anyway)

        Would you wish to be treated?




EBMRC                     Dr.Soltani                   EMRC
        Why not just use RRR? PTH trial
CER = 6 % Age =70 + 7
EER = 3 %
ARR = CER – EER = 3%
NNT = 1/ARR = 1/ 0.03 = 33
CER = 1/1000 Age = 55 + 5
EER = 1/2000
ARR=1/1000-1/2000=1/2000
NNT = 1/ARR= 2000
EBMRC                   Dr.Soltani        EMRC
           Why not just use RRR?




    RRR remains the same despite differences in
    absolute rate of events.
EBMRC                     Dr.Soltani              EMRC
              Why not just use RRR?




        ARRs reflect underlying susceptibility of patients
        and provides more complete information.
EBMRC                         Dr.Soltani                     EMRC
              Why not just use RRR?




        NNTs provide a useful measure of the clinical
        effort that must be expended to avoid bad events.
EBMRC                         Dr.Soltani                    EMRC
EBMRC   Dr.Soltani   EMRC
        Effects of baseline risk and
             RRR on the NNT




EBMRC               Dr.Soltani         EMRC
         Underlying risk:
        Benefits vs Harms


           Imagine a powerful but risky
           treatment with an RRR of 50% for
           a bad outcome BUT causes a
           serious side effect in 1%
           Two bad side effects for every bad
           outcome prevented.

                  Is it worth it?
EBMRC           Dr.Soltani                      EMRC
         Randomised trial of cholesterol lowering in 4,444 patients with CHD:
                the Scandinavian Simvastatin Survival Study (4S)
                             Lancet 1994: 344; 1383-1389



        • 4,444 patients recruited as a sample
        • inclusion criteria
           u CHD and cholesterol 5.5 - 8 mmol/l

        • exclusion criteria
           u planned cardiac surgery, HF, child bearing potential

        • simvastatin Vs. placebo
        • double blind
        • Outcomes
           u mortality, major coronary events, admissions for

             acute CHD, incidence of revascularisation
             procedures
EBMRC                               Dr.Soltani                              EMRC
            4S Study: self evaluation
   •    Median follow up 5.4 y
   •    analyse by intention to treat
   •    Significant reduction in all cause
        mortality
        u 11.5% placebo Vs. 8.2% simvastatin
        u ARR =

        u RRR =

        u NNT =      patients with CHD and cholesterol
          5.5 - 8 need to treated with simvastatin (20
          mg) for 5.4 years to save one life

EBMRC                       Dr.Soltani                   EMRC
                  4S Study Cont’d
   •    Median follow up 5.4 y
   •    analyse by intention to treat
   •    Significant reduction in all cause
        mortality
        u 11.5% placebo Vs. 8.2% simvastatin
        u ARR = 11.5 - 8.2 = 3.3%

        u RRR = (11.5 - 8.2)/11.5 = 29%

        u NNT = 1/ARR = 30


   • 30 patients with CHD and cholesterol 5.5
     - 8 need to treated with simvastatin (20
EBMRC
     mg) for 5.4 years to save one life
                           Dr.Soltani           EMRC
                  Insulin treatment in MI
• acute MI
• history of diabetes mellitus

• At what level of insulin would you start intensive (i.v.) insulin
  treatment?

•   4,5 mmol/l (80 mg/dl)
•   5,6 (100)
•   7,0 (126)
•   11,1 (200)
•   15 (270)
•   20 (360)

EBMRC                         Dr.Soltani                          EMRC
                 Insulin treatment in MI (DIGAMI)
                           BMJ 1997; 314: 1512



        • Population
          u   Acute MI, DM


        • Intervention (RCT, ITTA)
          u intensive insulin Tx (i.v. infusion x 24h, than
            min. 4 x d. for 3 months, vs
          u standard treatment




        • Outcome measure
EBMRC                          Dr.Soltani                     EMRC
            Insulin treatment in MI (DIGAMI)
                        BMJ 1997; 314: 1512
              death
                                         ARR = Rc – Ra
            Yes   No       Total
                                         u 0,07 (0,01–0,14)
intensive   58    248      306
standard    82    232       314          NNT = 1/ARR
                                         u 14 (8–174)
• Ra = 58/306 = 0,19
• Rc = 82/314 = 0,26                     RR = Ra/Rc
                                         u 0,73 (0,54–0,97)


                                         RRR = 1–RR
                                         u 27% (3–46)




EBMRC                       Dr.Soltani                        EMRC
                   Relative and absolute
        baseline
                     Ra    Rc                RR     RRR    ARR    NNT
          risk

  small               4%   3%                0,75   0,25   0,01   100

  large              20%   15%               0,75   0,25   0,05    20




EBMRC                           Dr.Soltani                              EMRC
                   Relative and absolute
        baseline
                     Ra    Rc                RR     RRR    ARR    NNT
          risk

  small               4%   3%                0,75   0,25   0,01   100

  large              20%   15%               0,75   0,25   0,05    20




EBMRC                           Dr.Soltani                              EMRC
             Relative and absolute (what’s
                    the difference?)
• spironolactone 25 mg/d
• death within 2 years

        Ra     Rc    RR      RRR          ARR   NNT # of pills   Cost


   35% 46% 0,76              0,24 0,11            9    6570 1000$


4 Drug lowering risk of death by 1/4
4 1000$ to delay one death
4 What after 2 years


EBMRC                              Dr.Soltani                           EMRC
        relative and absolute (what’s
•
                     the difference?)
     clopidogrel (75 mg/d)
• cardio-vascular death, MI, CVA within 9 months after acute
  coronary syndrome.

    Ra      Rc       RR       RRR        ARR       NNT # of pills    cost

                                                          13500     40000$
9,3% 11,4% 0,82               0,18 0,02             50


•    drug lowering the risk of MI by 1/3
•    40 000 $ to prevent one MI
•    what after 9 months?
•    who is to decide (physician, patient, insurance, society)?

EBMRC                                 Dr.Soltani                            EMRC
            Number Needed to Treat (NNT)
              (very trendy but tricky):
 • only defined for specific prevalence-Patient’s
   Expected Event Rate=PEER!
 • only defined for a specific intervention!
 • only defined for a specific outcome!
        u    eg. Pravastatin™ 40 mg nocte x10 years, in a 65
            year old male, ex-smoker with high BP and
            Diabetes, to reduce MI or Death.
 • NNT is the inverse of Absolute Risk
   Reduction: i.e. NNT = 1/ARR
EBMRC                          Dr.Soltani                      EMRC
        Number Needed to Treat (NNT)
           for previous example:
 • 12 fewer MI or death in 10 years per 100
   persons treated: ARR=12/100
 • NNT = 1/(12/100)=100/12= 8.3
 • But the same Relative Risk Reduction (RRR)
   of 40% with a low prevalence:
 • 0.4 fewer MI/death per 100 treated,
   ARR=0.4/100.
 • NNT = 1/(0.4/100) = 100/0.4 = 250!
EBMRC                Dr.Soltani             EMRC
             Why NNT is not enough
        • NNT for continuous outcome is difficult to
          calculate
        • Time can be added to calculations cautiously
        • We can not compare NNTs from different studies
          easily
        • Example: 2 RCT (Risedronate) 2 NNT! Can we
          say Risedronate is better than Risedronate!




EBMRC                        Dr.Soltani                    EMRC
        Number Needed to Treat (NNT)
           for previous example:
 • 12 fewer MI or death in 10 years per 100
   persons treated: ARR=12/100
 • NNT = 1/(12/100)=100/12= 8.3
 • But the same Relative Risk Reduction (RRR)
   of 40% with a low prevalence:
 • 0.4 fewer MI/death per 100 treated,
   ARR=0.4/100.
 • NNT = 1/(0.4/100) = 100/0.4 = 250!

EBMRC                Dr.Soltani             EMRC
        Why Odds Ratios? > compare
         results of different studies.
 • consider 2x2 table:
 • RRR is (a-b/a) — but you can only go in rows
   within same study!
 • Odds ratio is (a/c)/(b/d) = ad / bc — the
   individual ratios are in columns, and therefore
   are independent of the prevalence which is
   different in different studies.
 • must use odds ratios to combine RCT’s

EBMRC                 Dr.Soltani                EMRC
        Odds Ratio (OR) to NNT — is the
        improvement worth the trouble?
        • 1>OR>0, lower the OR = better the
          treatment (Rx) >> lower NNT.
        • for any OR, NNT is lowest when
          PEER=0.5
        • estimate the PEER (patient’s risk)
        • apply the OR to get patient's NNT.



EBMRC                     Dr.Soltani           EMRC
        Convert PEER & OR to NNT:




EBMRC             Dr.Soltani        EMRC
        Formula used in the table:




EBMRC                     Dr.Soltani   EMRC
           Table induced nausea!
  • lower OR >> lower NNT
  • Patient needs to be at risk (non-trivial
    PEER) in order for risk reduction to be
    worth the effort.
  • for any OR, NNT lowest when PEER=0.5
  • more effective treatment > lower NNT
  • BUT are your patient’s values satisfied by
    the intervention and its sequelae?


EBMRC                  Dr.Soltani                EMRC
        Choice of Effect Size Measures
 • RR and RRR most relevant for assessing causality
 • Relative risk and RRR for biological effect
 • Basal risk is very important for unbiased estimate
   of risk in your patients
 • ARR and NNT are most relevant for clinical and
   public health decisions
 • RR, RRR often favored because they look more
   impressive
 • Papers should report absolute measures, relative
   measures and NNT (Sackett)

EBMRC                    Dr.Soltani                     EMRC
                  Hypothermia after cardiac arrest
                             NEJM 2002; 346: 549



        • Population
           u   Patients resuscitated after cardiac arrest


        • Intervention (randomization, blinding, ITTA)
           u Cooling to 32–34°C for 24 h
           u Maintenance of normal temperature




        • Outcome measure
EBMRC
           u   Death at 6 months  Dr.Soltani                EMRC
          Hypothermia after cardiac arrest
                         NEJM 2002; 346: 549
            death
                                           ARR = Rc – Ra
          Yes       no      total
                                           u 0,14
Cooling   56    81           137
Normal    76    62           138           NNT = 1/ARR
                                           u 8
• Ra = 56/137 = 0,41
• Rc = 75/138 = 0,55                       RR = Ra/Rc
                                           u 0,74


                                           RRR = 1–RR
                                           u 26%



EBMRC                         Dr.Soltani                   EMRC
                         Hypothermia after cardiac arrest
                                     NEJM 2002; 346: 549

                             • Survival analysis
               100
                             • Relative Hazard (Hazard Ratio)
                             • HR ~ 0,65        RR = 0,74
                75
survival [%]




                                                             hypothermia

                50
                                                             normotermia

                25

                                                                   time [days

                     0          50                     100   150       200
EBMRC                                     Dr.Soltani                         EMRC
                                            • Survival analysis
Survival without recurrence [%]




                                  100                                           After 12 years
                                                                                RR ~ 0,98
                                                       HR ~ 0,68
                                   75                                           After 4 years
                                                                                RR ~ 0,16

                                   50

                                                                 treated
                                   25

                                                Non-treated                           Time [years]


                                        0   4   5                          10   12     15
EBMRC                                               Dr.Soltani                                  EMRC
        WHI: Coronary Heart Disease




               years   1                2   3
              4        5            6


EBMRC                  Dr.Soltani               EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
        What measures of precision of effects were
               reported (CIs, p-values)?

        • Either confidence intervals or p values
          for the estimates of effect should be
          reported.




EBMRC                      Dr.Soltani                EMRC
    A small P value implies that the alternate
    hypothesis is a better explanation for the
    data and chance is an unlikely explanation
    for the result.

  10 of 10             0.001
  9 of 10              0.02
  8 of 10              0.11
  7 of 10              0.34

                                    P VALUE

EBMRC                  Dr.Soltani                EMRC
            What we will be taking about
                      now?
• Statistical significance

• Clinical significance
        u ARR, NNT, RR, RRR
        u Confidence interval



EBMRC                   Dr.Soltani         EMRC
                Spironolactone in CHF (RALES)
                           NEJM 1999; 341: 709



        • Population
          u   Class IV NYHA, LVEF <35%
        • Intervention
          u   spironolacton 25 mg/d
        • Outcome measure
          u   Death, at 2 years

                       (7–16)
        • death rate 35% vs 46%; p <0,001
EBMRC
        • ARR = 11%               Dr.Soltani     EMRC
            Spironolactone in CHF (RALES)
                       NEJM 1999; 341: 709
   All people with CHF (population)



                        -20%         -10%    0   10%   20%




                        -20%         -10%    0   10%   20%

   Patients from RCT

EBMRC                       Dr.Soltani                       EMRC
            Spironolactone in CHF (RALES)
                          NEJM 1999; 341: 709

• ARR = 11% (7–16)
• Range within which the true population value is (with certain
  probability)


                           -20%         -10%    0    10%      20%
• If we were to repeat the same study (PICO) 100 times, the result
  would fall within this range 95 times




                           -20%         -10%    0    10%      20%

EBMRC                          Dr.Soltani                            EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
        P Value or CI?




EBMRC        Dr.Soltani   EMRC
EBMRC   Dr.Soltani   EMRC
           Was the precision of the effect
               estimates sufficient?
        • If 95% confidence intervals are wide and
          include the no effect point (e.g. RR=1,
          RD=0) or p-values are >> 0.05, then the
          precision of the estimates is likely to be
          poor & insufficient




EBMRC                      Dr.Soltani                  EMRC
         If no statistically significant effects
        detected, was there sufficient power?

   • If an effect estimate is not significantly
     different from no effect and the confidence
     interval is wide, the study is probably not large
     enough to detect a real difference between
     treatment and comparison groups (i.e. a low
     power study).
   • A non significant effect associated with a tight
     CI suggests there is no effect and that the
     study has adequate power.

EBMRC                    Dr.Soltani                  EMRC
        How large should the study
                   be?




EBMRC              Dr.Soltani        EMRC
         What sample size is needed?

  For E.H.D.* the usual mortality rate is 0%
  What sample size is needed to detect a reduction
    in mortality?
  • 100
  • 1,000
  • 100,000
  • 1,000,000

  * Excessive happiness disorder
EBMRC                  Dr.Soltani               EMRC
                            Sample Size: Café Rule 1
                           The 50:50 Rule (proportions)

         50 events are needed in the control group:
         (For an 80% chance of finding a 50% reduction)




Glasziou P, Doll H. Was the study big enough? Two cafe rules. Evid Based Med. 2006;11(3):69-70.
 EBMRC                                        Dr.Soltani                                          EMRC
               What sample size is needed?

        • There is usually a 12% mortality rate
          u   You think your treatment will lower
              mortality by 50%
        • What sample size is needed?




EBMRC                         Dr.Soltani            EMRC
                What sample size is needed?

        • There is usually a 12% mortality rate
          u   You think your treatment will lower mortality by
              50%
        • What sample size is needed?

        • 12% means
          u   12/100 or 24/200 or 48/400
          u   and 50 per 417
        • Control + Treatment Groups = 834 in total

EBMRC                            Dr.Soltani                      EMRC
        Confidence Intervals for Small Numerators


        • Example: A new drug is given to 60
          people. It seems to work, and has no
          serious adverse effects.
        • The authors conclude it is "safe and
          effective." The upper limit for the 95% CI
          for any serious adverse effect is, or 5%.



EBMRC                       Dr.Soltani                 EMRC
   Confidence Intervals for Small Numerators


More than 10% of FDA approved have serious
side effects.
                     Rule of three
Probability of event not occurring = 1-risk
Probability of event not occurring in n patient = (1-risk )^ n
(1- maxrisk ) ^ n = 0.05                  1- maxrisk =(0.05)^(1/n)
For n>30           1-maxrisk = 1-3/n          maxrisk = 3/n ; n > 3
95% confident of adverse event = 0/n – 3/n
EBMRC                        Dr.Soltani                              EMRC
        Another example from textbooks!
        • PREGNANCY IMPLICATIONS — Women who
          are pregnant when vaccinated or who become
          pregnant within 28 days of vaccination should be
          counseled on the theoretical risks to the fetus. The
          risk of rubella-associated malformations in these
          women is so small as to be negligible. MMR is the
          vaccine of choice if recipients are likely to be
          susceptible to measles or mumps as well as to
          rubella.
        • Up-to-date 14.2


EBMRC                          Dr.Soltani                        EMRC
        Another example from textbooks!
        •   TI - Persistent fetal rubella vaccine virus infection following inadvertent
            vaccination during early pregnancy.
        •   AU - Hofmann J; Kortung M; Pustowoit B; Faber R; Piskazeck U; Liebert UG
        •   SO - J Med Virol 2000 May;61(1):155-8.

        •   Inadvertent immunisation of seronegative women with RA27/3 rubella virus
            live-attenuated vaccine several weeks before and after conception is described.
            Whereas in 5 cases the vaccine virus was not transmitted vertically, in 1 case
            vaccination led to the development of persistent fetal infection with prolonged
            virus shedding for more than 8 months. Sequence analysis carried out on
            isolates from amniotic fluid, from cord blood leukocytes as well as from
            infantile urine confirmed an infection by the vaccine strain. At birth, the
            newborn infant exhibited none of the symptoms compatible with the
            congenital rubella syndrome and signs indicative for development of late onset
            disease are not apparent. This observation constitutes the first unequivocal
            documented case of rubella vaccine virus related to persistent fetal infection.




EBMRC                                       Dr.Soltani                                        EMRC
        Another example from textbooks!
        • Based on one observational study in 94
          women, no one report adverse reactions.
        • Rule of 3: 3/94=upper limit of adverse
          effects that may not have been seen.




EBMRC                      Dr.Soltani               EMRC
    Confidence Intervals for Small Numerators




EBMRC                 Dr.Soltani                EMRC
        Approximate maximum event rate
             for small numerators

             Events       Estimate of max

               0………………………..3/n
               1………………………..4/n
               2………………………..5/n
              3………………………...7/n
              4…………………………9/n

EBMRC                 Dr.Soltani            EMRC
        Approximate maximum event rate
             for small numerators




EBMRC                Dr.Soltani          EMRC
 95% confidence limits on extreme results
        denominator   0%could be as high as     100%could be as low as
         10             26%                     74%
         20             14%                     86%
         30             10%                     90%
         40             7%                      93%
         50             6%                      94%
         60             5%                      95%
         70             4%                      96%
         80             4%                      96%
         90             3%                      97%
         100            3%                      97%
         150            2%                      98%
         300            1%                      99%




EBMRC                              Dr.Soltani                            EMRC
                “Rule of Three”


        The number of exposures required to ensure a
         95% probability of observing at least one
         episode of a rare side effect




EBMRC                    Dr.Soltani                EMRC
        Penicillin anaphylaxis                1/10000   300000




  Aplastic anemia(chloram.)                   1/50000   150000




“To be 95% sure we will observe at least one case of rare
Side effect, we need to treat approximately three times the
number of individuals in the denominator”



EBMRC                            Dr.Soltani                EMRC
        Reverse application of the Rule of Three


        RCT        N=3000         Serious side effect =zero




              We can be 95% cofident that if anaphylaxis
              occurs,its frequency,on average is no more than 1
              per 1000 uses


EBMRC                            Dr.Soltani                       EMRC
“In a RCT of 300 patients receiving treatment,there were no serious
side effect.In a nonconcurrent cohort study in which 3000 patients
received the treatment,there were 10 cases of a serious side effect.the
authors concluded that the two studies were incompatible,and the
data from the RCT should be accepted, demonstating that the
treatment was free of serious side effects.”


                   Do you agree?


 EBMRC                          Dr.Soltani                         EMRC
                     Adverse Outcome




        RR = [A/(A+B)] / [C/(C+D)]     RCT or Cohort

        OR = AD/BC      Case control

EBMRC              Dr.Soltani                          EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
        How can I apply the results to
               patient care?

 • Were the study patients similar to my patient?
 • Were all clinically important outcomes
   considered?
 • Are the likely treatment benefits worth the
   potential harm and costs?




EBMRC                  Dr.Soltani                   EMRC
              Biologic Issues:
 • Are There Pathophysiologic Differences in the
   Illness Under Study That May Lead to a
   Diminished Treatment Response?
    u Malaria

    u HTN

    u Osteoporosis




EBMRC                   Dr.Soltani                 EMRC
                    Biologic Issues:
   • Are There Patient Differences That May Diminish
     the Treatment Response? Between-population
     differences in response to treatment may arise from
        1.   differences in drug metabolism, (increased activity of
             hepatic N-acetyltransferase, among Asian descent.)

        2.   differences in immune response,(Haemophilus influenzae
             vaccine, for example, has a lower efficacy rate in
             indigenous Alaskan populations )

        3.   differences in environmental factors that affect drug
             toxicity, (the incidence of thyroid dysfunction from
             amiodarone differs in low-iodine vs high-iodine
             environments.)
EBMRC                               Dr.Soltani                        EMRC
                Socioeconomic Issues

        • Are There Important Differences in
          Patient Adherence That May Diminish
          the Treatment Response?
        • For example, both types of problems may
          affect the safety of outpatient
          administration of anticoagulant agents.



EBMRC                     Dr.Soltani                EMRC
                 Socioeconomic Issues

        • Are There Important Differences in
          Provider Adherence That Might Diminish
          the Safety and Efficacy of the Treatment?
        • In this section, the term provider adherence
          or compliance refers to a host of diagnostic
          tests, monitoring equipment, interventional
          requirements, and other technical
          specifications that clinicians must use or
          satisfy to administer a treatment safely and
          effectively. effectiveness.
EBMRC                       Dr.Soltani                   EMRC
                   Socioeconomic Issues

        • For instance:
          u   carotid endarterectomy by expert center
          u   Warfarin or streptokinase in developing
              countries




EBMRC                          Dr.Soltani               EMRC
                    Epidemiologic Issues

        • Do Patients in My Practice Have Comorbid
          Conditions That Significantly Alter the
          Potential Benefits and Risks of the Treatment?
          u   In malaria-endemic areas, clinicians may expect an
              increase in false-positive pneumonias.
          u   An example comes from the management of patients
              with acute myocardial infarction . In the Philippine
              study, noncardiac causes—mostly pneumonia with
              sepsis—were responsible for 11 of the 30 deaths.
              Streptokinase will not reduce mortality in such patients.


EBMRC                              Dr.Soltani                             EMRC
        Are The Likely Benefits Worth The
            Potential Risks And Costs?
        • What Is the Patient’s Risk of Adverse Target
          Events?
          u   First, they can use their intuition, which may sometimes
              be accurate—at least in terms of the extent to which
              risk is increased or decreased relative to a typical
              patient.
          u   Second, if the randomized trials report risks in patient
              subgroups, clinicians can choose the risk that best
              applies to the patient in their practice
          u   Systematic reviews that pool data from multiple trials
              can provide more precise estimates.
          u   Clinicians are most likely to find information about
              risks in easily identifiable subgroups of patients from
              studies directly targeted at prognosis
EBMRC                              Dr.Soltani                            EMRC
        Are The Likely Benefits Worth The
            Potential Risks And Costs?




EBMRC                 Dr.Soltani            EMRC
         Underlying risk:
        Benefits vs Harms


           Imagine a powerful but risky
           treatment with an RRR of 50% for
           a bad outcome BUT causes a
           serious side effect in 1%
           Two bad side effects for every bad
           outcome prevented.

                  Is it worth it?
EBMRC           Dr.Soltani                      EMRC
           Examples
        •Brief
         Critical
         Appraisal
EBMRC         Dr.Soltani   EMRC
     Study 1: MRC/BHF Heart Protection Study of
        Antioxidant vitamin supplementation, …

• Citation: Lancet 2002; 360: 23-33.
• ?: Does Antiox sup (C, E, Beta Carotene) reduce death,
  vascular events or Ca in people with high 5 year risk.
• Design: RCT
• Subjects: 20,536 (40-80 y.o.), 75% male, w high risk
• Setting: 69 UK Hospitals, 5 yr f/u
• Control: Placebo
• Allocation concealed: Y
• Blinded: Pt, Dr, Collectors, Analyzers
• Analysis: ITT
 EBMRC                    Dr.Soltani                  EMRC
    Study 1: MRC/BHF Heart Protection Study of
       Antioxidant vitamin supplementation, …
• Findings: No benefit (in any outcome). All NNH
  and NNT non-significant.
• Limits: One dose, high risk, duration
• Strengths: RCT, # with 99.6% f/u (83 %
  compliance)
• Funding: UK MRC, Brit Heart Found, Merck,
  Roche (*Reassured us analysis was independent of
  funding source)
• Comments: agree other RCT (put to rest a few
  observational and Cambridge heart antioxidant
  study)
EBMRC                   Dr.Soltani                   EMRC
         Study 2: Should we advise parents to
            administer OTC cough med,…
• Citation: Arch Dis Child 2002; 86: 170-5.
• ?: In children with URTI, are OTC cough meds effective
  in relieving coughs.
• Design: Review (used Cochrane SR)
• Subjects: 438 children in 6 studies
• Selection: 328 studies exclusion induced cough or
  chronic & not OTC, RTC or placebo.
• Control: Placebo
• Allocation concealed: NR (poor)
• Blinded: variable (pt or clinician NR for 3)
• Analysis: NR
 EBMRC                   Dr.Soltani                  EMRC
         Study 2: Should we advise parents to
            administer OTC cough med,…
• Findings: No benefit (1 statistical but not clinical).
• Limits: Few low quality studies, wide spectrum of
  disease, multiple drugs (anti-hist, anti-tus,
  mucolytics)
• Strengths: SR (missed ITT).
• Funding: NHS (studies NR)
• Comments: agree other reviews (2) and for now,
  we can not recommend OTC’s for cough with
  URTI

 EBMRC                   Dr.Soltani                  EMRC
    Three Step Guide in Using an Article to
               Assess Therapy

        1. Are the results of the study valid?
        2. What are the results? What measures of
           precision of effects were reported (CIs, p-
           values)?

        3. How can I apply these results to patient care?




EBMRC                         Dr.Soltani                    EMRC
        Thank you

EBMRC      Dr.Soltani   EMRC
        Summary 1: Set your goals.

  • define your 4 (or 3) part question.
  • do you want a true systematic review?
  • does this narrow review address my
    question?
  • PRE-DEFINED search, inclusion, exclusion!




EBMRC                Dr.Soltani             EMRC
             Summary 2: Be Sceptical!

  •     look for bias, conflict of interest.
  •     critical appraisal of primary studies?
  •     consistent results? if not, why not?
  •     does our patient fit the groups studied?
  •     does it matter to our patient?




EBMRC                      Dr.Soltani              EMRC
        Summary 3: Risks that matter.
  • Absolute risk > estimate the Patient
    Expected Event Rate (PEER)
  • obtain Relative Risk Reduction (RRR) or
    Odds Ratio (OR) from a Meta-analysis
  • plug into a table to estimate Number Needed
    to Treat (NNT)




EBMRC                 Dr.Soltani              EMRC
        Summary 4: Sceptical & common
                    sense!
  • beware of post-hoc sub-group analysis,
    especially if multiple.
  • step back and consider if the systematic
    review really related to our patient’s
    situation (PEER), culture and expectations?
  • do not loose sight of common sense!



EBMRC                 Dr.Soltani                  EMRC

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:7/22/2013
language:English
pages:198
jny jhtw jny jhtw
About