Docstoc

Summarising the Data - Division of Clinical Neurosciences

Document Sample
Summarising the Data - Division of Clinical Neurosciences Powered By Docstoc
					      Meta-analysis:
 summarising data for two
arm trials and other simple
     outcome studies
         Steff Lewis
         statistician
        When can/should you do
           a meta-analysis?
• When more than one study has estimated an
  effect
• When there are no differences in the study
  characteristics that are likely to substantially
  affect outcome
• When the outcome has been measured in
  similar ways
• When the data are available (take care with
  interpretation when only some data are
  available)
            Types of data
• Dichotomous/ binary data

• Counts of infrequent events
• Short ordinal scales

• Long ordinal scales
• Continuous data

• Censored data
         What to collect
• Need the total number of patients in
  each treatment group
                   Plus:
• Binary data
  – The number of patients who had the
    relevant outcome in each treatment group
• Continuous data
  – The mean and standard deviation of the
    effect for each treatment group
•   Then enter data into RevMan / MIX
    (easy to use and free)
http://www.mix-for-meta-analysis.info/
http://www.cc-ims.net/RevMan/




Or R (harder to use and free)
Or Stata (harder to use and costs)
Etc etc....
Summary statistic for each
        study
• Calculate a single summary
  statistic to represent the effect
  found in each study
• For binary data
  – Risk ratio with rarer event as
    outcome
• For continuous data
  – Difference between means
Meta-analysis
          Averaging studies

• Starting with the summary statistic for
  each study, how should we combine
  these?
• A simple average gives each study
  equal weight
• This seems intuitively wrong
• Some studies are more likely to give an
  answer closer to the „true‟ effect than
  others
           Weighting studies

• More weight to the studies which give
  us more information
   – More participants
   – More events
   – Lower variance
• Weight is closely related to the width of
  the study confidence interval: wider
  confidence interval = less weight
    Displaying results graphically

• RevMan (the Cochrane Collaboration‟s
  free meta-analysis software) and MIX
  produce forest plots (as do R and Stata
  and some other packages)
Review :          Corticosteroids for acute traumatic brain injury
Comparison:       01 Any steroid administered in any dose against no steroid
Outcome:          01 Death at end of follow up period

Study                                      Steroid                      Control                   RR (fixed)                             RR (fixed)
or sub-category                              n/N                         n/N                       95% CI                                 95% CI

Alexander 1972                           16/55                          22/55                                                      0.73 [0.43, 1.2
Brackman 1983                            44/81                          47/80                                                      0.92 [0.70, 1.2
CRASH 2004                             1052/4985                       893/4979                                                    1.18 [1.09, 1.2
Chacon 1987                               1/5                            0/5                                                       3.00 [0.15, 59.
Cooper 1979                              26/49                          13/27                                                      1.10 [0.69, 1.7
Dearden 1986                             33/68                          21/62                                                      1.43 [0.94, 2.1
Faupel 1976                              16/67                          16/28                                                      0.42 [0.24, 0.7
Gaab 1994                                19/133                         21/136                                                     0.93 [0.52, 1.6
Giannotta 1984                           34/72                           7/16                                                      1.08 [0.59, 1.9
Grumme 1995                              38/175                         49/195                                                     0.86 [0.60, 1.2
Hemesniemi 1979                          35/81                          36/83                                                      1.00 [0.70, 1.4
Pitts 1980                              114/201                         38/74                                                      1.10 [0.86, 1.4
Ransohoff 1972                            9/17                          13/18                                                      0.73 [0.43, 1.2
Saul 1981                                 8/50                           9/50                                                      0.89 [0.37, 2.1
Stubbs 1989                              13/98                           5/54                                                      1.43 [0.54, 3.8
Zagara 1987                               4/12                           4/12                                                      1.00 [0.32, 3.1
Zarete 1995                               0/30                           0/30                                                         Not estimabl

Total (95% CI)                                 6179                        5904                                                    1.12 [1.05, 1.2
Total events: 1462 (Steroid), 1194 (Control)
Test for heterogeneity: Chi² = 26.46, df = 15 (P = 0.03), I² = 43.3%
Test for overall effect: Z = 3.27 (P = 0.001)

                                                                                  0.1   0.2    0.5       1       2       5    10
                                                                                        Steroid better       Steroid w orse
Heterogeneity
What is heterogeneity?

•   Heterogeneity is variation between the
    studies‟ results
         Causes of heterogeneity
Differences between studies with respect
  to:
• Patients: diagnosis, in- and exclusion
  criteria, etc.
• Interventions: type, dose, duration,
  etc.
• Outcomes: type, scale, cut-off points,
  duration of follow-up, etc.
• Quality and methodology:
  randomised or not, allocation
  concealment, blinding, etc.
     How to deal with heterogeneity

1. Do not pool at all

2. Ignore heterogeneity: use fixed effect
  model

3. Allow for heterogeneity: use random
  effects model

4. Explore heterogeneity: meta-regression
  (tricky)
How to assess heterogeneity from a
forest plot
Statistical measures of heterogeneity


• The Chi2 test measures the
  amount of variation in a set of
  trials, and tells us if it is more than
  would be expected by chance
                                                                      Trials from
                Estimates with 95% confidence intervals
Study
                                                                    Cochrane logo:
                                                                  Corticosteroids for
Liggins 1972                                                         preterm birth
Block 1977                                                         (neonatal death)
Morrison 1978
Taeusch 1979
Papageorgiou 1979
Schutte 1979
                                                                  Heterogeneity test
Collaborative Group 1981                                          Q = 11.2 (6 d.f.)
Pooled                                   0.61   ( 0.46 , 0.81 )
                                                                  p = 0.08
                    0.05 0.25 1   4
                         Odds ratio
         Corticosteroids better       Corticosteroids worse
                                                          Corticosteroids for
                Estimates with 95% confidence intervals
Study
                                                             preterm birth
                                                           (neonatal death)
Liggins 1972
Block 1977
Morrison 1978
Taeusch 1979
Papageorgiou 1979
Schutte 1979
                                                          Heterogeneity test
Collaborative Group 1981                                  Q = 11.2 (6 d.f.)
                                                          p = 0.08

                                                          Heterogeneity test
                                                          Q = 44.7 (27 d.f.)
                                                          p = 0.02

                    0.05 0.25 1   4   0.05 0.25 1   4
                        Odds ratio        Odds ratio
     I squared quantifies
        heterogeneity
                       Q  df
             I  100 
               2

                         Q
     where Q = heterogeneity c2 statistic


I2 can be interpreted as the proportion of
total variability explained by
heterogeneity, rather than chance
• Roughly, I2 values of 25%, 50%,
  and 75% could be interpreted as
  indicating low, moderate, and high
  heterogeneity
• For more info see: Higgins JPT et
  al. Measuring inconsistency in
  meta-analyses. BMJ
  2003;327:557-60.
Fixed and random effects
              Fixed effect

Philosophy behind fixed effect model:
• there is one real value for the treatment
  effect
• all trials estimate this one value

Problems with ignoring heterogeneity:
• confidence intervals too narrow
            Random effects

Philosophy behind random effects
  model:
• there are many possible real values for
  the treatment effect (depending on dose,
  duration, etc etc).
• each trial estimates its own real value
Example
 Could we just add the data from all
        the trials together?

• One approach to combining trials would
  be to add all the treatment groups
  together, add all the control groups
  together, and compare the totals
• This is wrong for several reasons, and it
  can give the wrong answer
If we add up the columns we get 34.3%       From a meta-analysis, we get
 vs 32.5% , a RR of 1.06, a higher chance   RR=0.96 , a lower chance of
 of death in the steroids group             death in the steroids group
  Problems with simple addition of
              studies

• breaks the power of randomisation
• imbalances within trials introduce bias
            *




The Pitts trial contributes 17% (201/1194) of all the data to the
experimental column, but 8% (74/925) to the control column.
Therefore it contributes more information to the average chance
of death in the experimental column than it does to the control
column.
There is a high chance of death in this trial, so the chance of
death for the expt column is higher than the control column.
Interpretation
Interpretation - “Evidence of
absence” vs “Absence of evidence”
• If the confidence interval crosses the
line of no effect, this does not mean that
there is no difference between the
treatments
• It means we have found no
statistically significant difference in the
effects of the two interventions
              In the example below, as more data is included,
              the overall odds ratio remains the same but the
              confidence interval decreases.
              It is not true that there is „no difference‟ shown
              in the first rows of the plot – there just isn‟t
              enough data to show a statistically significant
              result.
Review :          Steff
Comparison:       01 Absence of evidence and Evidence of absence
Outcome:          01 Increasing the amount of data...

Study                                 Treatment                    Control                  OR (fixed)                                OR (fixed)
or sub-category                          n/N                        n/N                      95% CI                                    95% CI

1 study                                10/100                  15/100                                                         0.63   [0.27,   1.48]
2 studies                              20/200                  30/200                                                         0.63   [0.34,   1.15]
3 studies                              30/300                  45/300                                                         0.63   [0.38,   1.03]
4 studies                              40/400                  60/400                                                         0.63   [0.41,   0.96]
5 studies                              50/500                  75/500                                                         0.63   [0.43,   0.92]

                                                                             0.1   0.2    0.5      1      2       5      10
                                                                               Favours treatment       Favours control
Interpretation - Weighing up benefit
and harm

•   When interpreting results, don‟t just
    emphasise the positive results.
•   A treatment might cure acne instantly,
    but kill one person in 10,000 (very
    important as acne is not life
    threatening).
Interpretation - Quality

•   Rubbish studies = unbelievable results
•   If all the trials in a meta-analysis were
    of very low quality, then you should
    be less certain of your conclusions.
•   Instead of “Treatment X cures
    depression”, try “There is some
    evidence that Treatment X cures
    depression, but the data should be
    interpreted with caution.”
Summary
• Choose an appropriate effect measure
•   Collect data from trials and do a meta-
    analysis if appropriate
•   Interpret the results carefully
    –   Evidence of absence vs absence of
        evidence
    –   Benefit and harm
    –   Quality
    –   Heterogeneity
Sources of statistics help and advice

Cochrane Handbook for Systematic
Reviews of Interventions
 http://www.cochrane.org/resources/handbook/index.htm


The Cochrane distance learning material
 http://www.cochrane-net.org/openlearning/


The Cochrane RevMan user guide.
 http://www.cc-ims.net/RevMan/documentation.htm

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:4/13/2011
language:English
pages:36