Empirical evidence of bias in treatment effect estimates in by gdf57j



                                       Empirical evidence of bias in treatment effect estimates in
                                       controlled trials with different interventions and outcomes:
                                       meta-epidemiological study
                                       Lesley Wood,1 Matthias Egger,2 Lise Lotte Gluud,3 Kenneth F Schulz,4 Peter Juni,2 Douglas G Altman,5
                                       Christian Gluud,3 Richard M Martin,1 Anthony J G Wood,1 Jonathan A C Sterne1

  Department of Social Medicine,       ABSTRACT                                                            We combined data on a large number of controlled
University of Bristol, Bristol         Objective To examine whether the association of                  trials from three meta-epidemiological studies (collec-
                                       inadequate or unclear allocation concealment and lack of         tions of meta-analyses) to examine whether the effects
  Institute of Social and Preventive
Medicine (ISPM), University of         blinding with biased estimates of intervention effects           of reported allocation concealment and blinding on
Bern, Switzerland                      varies with the nature of the intervention or outcome.           estimates of intervention effects differ depending on
  Copenhagen Trial Unit, Centre for    Design Combined analysis of data from three meta-                the nature of the outcome (objective or subjective, all
Clinical Intervention Research,                                                                         cause mortality or other outcome) and according to
Rigshospitalet, Copenhagen
                                       epidemiological studies based on collections of meta-
University Hospital, Copenhagen,       analyses.                                                        whether the meta-analysis examined a drug or non-
Denmark                                Data sources 146 meta-analyses including 1346 trials             drug intervention.
  Family Health International and
                                       examining a wide range of interventions and outcomes.
Department of Obstetrics and
Gynaecology, School of Medicine,       Main outcome measures Ratios of odds ratios quantifying          METHODS
University of North Carolina at        the degree of bias associated with inadequate or unclear         We analysed data from three published meta-epide-
Chapel Hill, NC, USA                                                                                    miological studies that included information on
5                                      allocation concealment, and lack of blinding, for trials
  Centre for Statistics in Medicine,                                                                    allocation concealment, blinding, and outcome
Oxford, UK                             with different types of intervention and outcome. A ratio of
                                       odds ratios <1 implies that inadequately concealed or non-       events.1 3 4 Schulz et al used 33 meta-analyses from the
Correspondence to: J A C Sterne
jonathan.sterne@bristol.ac.uk          blinded trials exaggerate intervention effect estimates.
                                                                                                        Pregnancy and Childbirth Group of the Cochrane
                                                                                                        Collaboration.1 Each meta-analysis included at least
BMJ 2008;336:601-5                     Results In trials with subjective outcomes effect estimates
                                                                                                        five trials with a combined total of at least 25 outcome
                                       were exaggerated when there was inadequate or unclear
                                                                                                        events. Kjaergard et al used 14 meta-analyses from 11
                                       allocation concealment (ratio of odds ratios 0.69 (95% CI
                                                                                                        systematic reviews, all of which included at least one
                                       0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In
                                                                                                        trial of at least 1000 participants.3 Egger et al analysed
                                       contrast, there was little evidence of bias in trials with
                                                                                                        122 meta-analyses from the Cochrane Database of
                                       objective outcomes: ratios of odds ratios 0.91 (0.80 to
                                                                                                        Systematic Reviews that contained at least five rando-
                                       1.03) for inadequate or unclear allocation concealment
                                                                                                        mised trials.4 See bmj.com for details. We removed
                                       and 1.01 (0.92 to 1.10) for lack of blinding. There was little
                                                                                                        duplicate meta-analyses until there was no overlap
                                       evidence for a difference between trials of drug and non-
                                                                                                        between the remaining meta-analyses, except for a
                                       drug interventions. Except for trials with all cause mortality
                                                                                                        small number of trials that had more than one
                                       as the outcome, the magnitude of bias varied between
                                                                                                        intervention arm or presented more than one outcome.
                                       Conclusions The average bias associated with defects in          Assessment of trial quality
                                       the conduct of randomised trials varies with the type of         In the study by Schulz et al one researcher, who was
                                       outcome. Systematic reviewers should routinely assess            blinded to the trial outcome, assessed the methodo-
                                       the risk of bias in the results of trials, and should report     logical quality of the included trials using a detailed
                                       meta-analyses restricted to trials at low risk of bias either    classification scheme.1 In the study by Kjaergard et al
                                       as the primary analysis or in conjunction with less              assessments were done by two observers who were not
                                       restrictive analyses.                                            blinded to study results.3 Inter-rater reliability of
                                                                                                        quality assessments was assessed in 30 randomly
                                       INTRODUCTION                                                     selected randomised controlled trials and found to be
                                       Several studies have found that randomised controlled            high (intraclass correlation coefficient 0.96). The study
                                       trials with inadequate allocation concealment or lack of         of Egger et al was based on quality assessments by
                                       blinding (see box 1) tend to exaggerate estimates of             authors of the Cochrane reviews, which were generally
                                       intervention effects, compared with adequately con-              done in duplicate by two observers.4 Definitions of
                                       cealed or adequately blinded trials.1-4 However, other           adequate allocation concealment and blinding were
                                       studies have failed to confirm these associations.5 6 A          similar in all three studies (see bmj.com).
This article is an abridged version
of a paper that was published on       possible reason for these inconsistencies is that the
bmj.com on 3 March 2008. Cite          magnitude of bias varies between meta-analyses                   Interventions and outcomes
this article as: BMJ 3 March 2008,
doi: 10.1136/                          according to other trial characteristics such as the             For each meta-analysis in the final dataset, we classified
bmj.39465.451748.AD                    nature of the intervention or outcome.                           the type of intervention and the type of outcome. We

BMJ | 15 MARCH 2008 | VOLUME 336                                                                                                                              601

                                        coded interventions using the classification of Moja et
                                                                                                                  Box 1: Definitions of trial characteristics studied in this
                                        al7 and subsequently dichotomised them as drug or
                                        non-drug interventions.
                                                                                                                  Allocation concealment—Procedures that secure strict
                                           We classified outcomes in two ways: firstly, as
                                                                                                                  implementation of the schedule of random assignments
                                        objectively or subjectively assessed, and, secondly, as                   by preventing foreknowledge of forthcoming allocations
                                        all cause mortality or other outcomes. The definition of                  by study participants or by those recruiting them to the
                                        objective and subjective outcomes was based on the                        trial
                                        extent to which outcome assessment could be influ-
                                                                                                                    It is always feasible to conceal allocation
                                        enced by investigators’ judgment. Objectively assessed
                                                                                                                    Failure to conceal allocation may lead to biased
                                        outcomes included all cause mortality, measures based
                                                                                                                    selection of participants into intervention groups
                                        on a recognised laboratory procedure, and other
                                                                                                                  Blinding—Procedures that prevent study participants,
                                        objective measures (details on bmj.com).
                                                                                                                  caregivers, or outcome assessors from knowing which
                                                                                                                  intervention was received
                                        Statistical methods
                                                                                                                    Blinding of participants and caregivers may not be
                                        We measured intervention effects as odds ratios.
                                                                                                                    feasible (such as in a trial of surgery versus radiotherapy
                                        Outcome events were recoded where necessary so                              for prostate cancer), but it may still be possible to blind
                                        that an odds ratio below 1 indicated a beneficial effect of                 assessment of outcomes
                                        the experimental intervention. We calculated the                            Blinding may reduce the risk that knowledge of the
                                        combined effect estimates separately in trials with                         intervention received, rather than the intervention itself,
                                        and without the characteristic of interest (inadequate or                   affects outcomes or outcome measurements
                                        unclear allocation concealment or lack of blinding).
                                        We used logistic regression models to estimate ratios of
                                        odds ratios comparing intervention effects in trials with
                                        and without the characteristic of interest, and derived                  blinding, we estimated the effect of each characteristic
                                        95% confidence intervals using robust standard errors                    on intervention effects in the same logistic regression
                                        to allow for heterogeneity between meta-analyses.8                       model.
                                           We also calculated ratios of odds ratios separately in
                                        each meta-analysis and combined these using random                       RESULTS
                                        effects meta-analyses, in order to estimate variability                  Of 159 meta-analyses contained in the three studies
                                        between meta-analyses in the effect of trial                             that contributed data, we analysed a total of 146, which
                                        characteristics.8 We included interaction terms in                       included 1346 trials. The commonest interventions
                                        logistic regression models to assess whether effects of                  were drug treatments (89 (61%) meta-analyses and 919
                                        trial quality varied with the type of intervention or type               (68%) trials) and prevention or screening interventions
                                        of outcome. To assess whether there was confounding                      (22 (15%) meta-analyses, 161 (12%) trials). The
                                        between the effects of allocation concealment and                        commonest types of outcome were physician assessed
                                                                                                                 outcomes (42 (29%) meta-analyses, 323 (24%) trials)
                                                                                                                 and all cause mortality (27 (18%) meta-analyses, 295
                                                                                                                 (22%) trials). Outcomes were classified as objectively
                             No of            Ratio of         Ratio of odds ratios P value Variability in
(No of meta-analyses)       trials*          odds ratios             (95% CI)       of test of bias† (P value)   assessed in 78 (53%) meta-analyses and 718 (53%) trials
                                                                                   interaction                   (details on bmj.com).
Overall (102)              532 v 272                           0.83 (0.74 to 0.93)      –       0.11 (<0.001)
                                                                                                                 Allocation concealment and estimates of intervention
All cause mortality (23)   119 v 90                            1.01 (0.90 to 1.15)               0.02 (0.24)     effects
Other outcomes (79)        415 v 182                           0.76 (0.66 to 0.87)              0.14 (<0.001)
                                                                                                                 We included 102 meta-analyses in our analysis of
                                                                                                                 associations between allocation concealment and
Objective outcomes (62) 310 v 174                              0.91 (0.80 to 1.03)              0.11 (<0.001)
                                                                                      0.009                      estimates of intervention effects (fig 1). Of the 804
Subjective outcomes (40) 222 v 98                              0.69 (0.59 to 0.82)              0.07 (0.011)
                                                                                                                 trials in these meta-analyses, 272 (34%) had adequate
Drug intervention (65)     411 v 205                           0.87 (0.76 to 1.00)              0.09 (<0.001)
                                                                                                                 allocation concealment. Overall, intervention effect
                                                                                       0.27                      estimates were exaggerated by 17% in the trials with
Other intervention (37)    121 v 67                            0.77 (0.64 to 0.93)              0.16 (<0.001)
                                                                                                                 inadequate or unclear allocation concealment com-
                                       0.5   0.75 1    1.5 2
                                      Inadequately Inadequately
                                                                                                                 pared with those with adequate allocation concealment
                                                                                                                 (ratio of odds ratios 0.83 (95% CI 0.74 to 0.93)). The size
                                      beneficial      beneficial                                                 of the bias varied between meta-analyses (heterogene-
* Inadequately or unclearly concealed v adequately concealed                                                     ity variance 0.11, P<0.001).
† Between-meta-analysis heterogeneity variance
                                                                                                                    Bias associated with inadequate or unclear allocation
Fig 1 | Ratios of odds ratios comparing estimates of intervention effects in 532 trials with
                                                                                                                 concealment seemed restricted to trials assessing out-
inadequate or unclear allocation concealment versus 272 trials with adequate concealment.                        comes other than all cause mortality (ratio of odds
Columns on the right show P values from tests of interaction, and estimates of the variability                   ratios 0.76 (95% CI 0.66 to 0.87)): there was no
between meta-analyses in the size of bias                                                                        evidence of such bias in trials assessing all cause

602                                                                                                                                            BMJ | 15 MARCH 2008 | VOLUME 336

Comparison                   No of            Ratio of          Ratio of odds ratios P value Variability in       in bias (see fig 2) was lower when the outcome was all
(No of meta-analyses)       trials*          odds ratios              (95% CI)       of test of bias† (P value)   cause mortality than for other types of outcome.
Overall (76)               314 v 432                            0.93 (0.83 to 1.04)      –       0.11 (<0.001)
                                                                                                                  Combined analysis of the effects of allocation
                                                                                                                  concealment and blinding
All cause mortality (18)   79 v 121                             1.04 (0.95 to 1.14)               0.01 (0.27)
Other outcomes (58)        235 v 311                            0.83 (0.70 to 0.98)
                                                                                                 0.18 (<0.001)
                                                                                                                  To investigate whether allocation concealment and
                                                                                                                  blinding were independent of each other in their effects
Objective outcomes (44) 210 v 227                               1.01 (0.92 to 1.10)              0.08 (<0.001)
                                                                                                                  on estimates of intervention effect, we included the
                                                                                        0.01                      effects of each characteristic in the same logistic
Subjective outcomes (32) 104 v 205                              0.75 (0.61 to 0.82)              0.14 (0.001)
                                                                                                                  regression model. For all cause mortality and other
Drug intervention (57)     250 v 372                            0.92 (0.81 to 1.05)              0.10 (<0.001)    objectively assessed outcomes, there was little change
                                                                                        0.66                      in the estimated effects of allocation concealment and
Other intervention (19)     64 v 60                             1.00 (0.71 to 1.39)              0.22 (0.003)
                                       0.5   0.75 1     1.5 2
                                                                                                                  blinding. For subjectively assessed outcomes, the effect
                                      Non-blinded     Non-blinded
                                                                                                                  of each characteristic was modestly attenuated (ratio of
                                                                                                                  odds ratios for inadequate or unclear allocation
                                                                                                                  concealment 0.75 (0.63 to 0.88), for lack of blinding
* Non-blinded v blinded
† Between-meta-analysis heterogeneity variance
                                                                                                                  0.77 (0.65 to 0.91)). Similar attenuation was seen for
                                                                                                                  outcomes other than all cause mortality (ratio of odds
Fig 2 | Ratios of odds ratios comparing intervention effect estimates in 314 non-blinded trials                   ratios for inadequate or unclear allocation concealment
versus 432 blinded trials. Columns on the right show P values from tests of interaction, and                      0.80 (0.73 to 0.87), for lack of blinding 0.85 (0.76 to
estimates of the variability between meta-analyses in the size of bias                                            0.95)).

                                        mortality (ratio of odds ratios 1.01 (0.90 to 1.14),                      We analysed a dataset that included many meta-
                                        interaction P=0.002). The effect of allocation conceal-                   analyses and trials and represented a wide variety of
                                        ment also varied (interaction P=0.009) according to                       clinical disciplines, interventions, and outcomes. We
                                        whether trials had subjectively assessed outcomes                         found that bias in intervention effect estimates from
                                        (ratio of odds ratios 0.69 (0.59 to 0.82)) or objectively                 clinical trials, resulting from inadequate allocation
                                        assessed outcomes (0.91 (0.80 to 1.03)). The variability                  concealment and lack of blinding, varied according to
                                        between meta-analyses in bias (see fig 1) was lower                       the type of outcome assessed. There was little evidence
                                        when the outcome was all cause mortality than for                         of bias in trials assessing all cause mortality or other
                                        other types of outcome. There was little evidence                         objectively assessed outcomes. In contrast, inadequate
                                        (interaction P=0.27) that bias associated with inade-                     allocation concealment and lack of blinding were
                                        quate allocation concealment differed in trials that                      associated with over-optimistic estimates of inter-
                                        assessed drug interventions compared with trials                          vention effects for subjectively assessed outcomes.
                                        assessing non-drug interventions.                                         The size of bias varied between meta-analyses, but this
                                                                                                                  variability was less in trials assessing all cause mortality,
                                        Blinding and estimates of intervention effects                            where there was little evidence of bias. It thus seems
                                        Figure 2 shows the associations between blinding and                      that in situations when bias is likely the size of such bias
                                        estimates of intervention effects, based on 76 meta-                      is unpredictable.
                                        analyses containing 746 trials, of which 432 (58%)
                                        were blinded. Overall, estimates of intervention                          Strengths and weaknesses of study
                                        effects were exaggerated by 7% in non-blinded                             The reported quality of a trial may not reflect the way
                                        compared with blinded trials (ratio of odds ratios 0.93                   that it was conducted in practice: well conducted trials
                                        (0.83 to 1.04)), although the size of the bias varied                     may be reported badly.9 10 Even clearer patterns than
                                        between meta-analyses (heterogeneity variance 0.11,                       those identified here might have emerged had more
                                        P<0.001).                                                                 accurate information been available.
                                           The bias associated with lack of blinding was greater                    We assembled a large dataset by combining data
                                        (interaction P=0.011) in trials assessing outcomes other                  from three meta-epidemiological studies, but the way
                                        than all cause mortality (ratio of odds ratios 0.83 (0.70                 in which trial characteristics were assessed varied
                                        to 0.98)) than in those assessing all cause mortality (1.04               between the original studies.
                                        (0.95 to 1.14)). Similarly, bias associated with lack of                    We did not account for bias due to selective reporting
                                        blinding seemed restricted to trials with subjectively                    of outcomes and bias due to exclusion of patients
                                        assessed outcomes (ratio of odds ratios 0.75 (0.61 to                     because of lack of adherence to the randomised
                                        0.93)), with no evidence of such bias in trials with                      intervention or missing outcome data. Hence our
                                        objective outcomes (1.01 (0.92 to 1.10)) (interaction                     assessment of the effects of inadequate allocation
                                        P=0.010). There was little evidence (interaction                          concealment or lack of blinding could be confounded
                                        P=0.66) that the size of the bias associated with lack of                 by them, as well as by characteristics of study
                                        blinding differed between trials of drug and non-drug                     populations and interventions that influence estimates
                                        interventions. The variability between meta-analyses                      of intervention effects.

BMJ | 15 MARCH 2008 | VOLUME 336                                                                                                                                           603

                                                                              strategies for reducing bias, beyond blinding, as its
            Box 2: Recommendations for practice                               effect remains when adjusted for blinding.
               Since bias associated with lack of adequate allocation
                concealment or lack of blinding is less for trials with       Implications
                objectively assessed outcomes than trials with
                                                                              Although many medical journals have adopted the
                subjectively assessed outcomes, efforts to minimise
                                                                              CONSORT (consolidated standards of reporting
                bias are particularly important when objective
                measurement of outcomes is not feasible
                                                                              trials) recommendations, many of the trials included
                                                                              in this study were published before the CONSORT
               Authors of systematic reviews, and those critically
                                                                              statement. Improved transparency about the way that
                appraising trials, should routinely assess the risk of bias
                in results associated with the way each trial was done.
                                                                              trials were conducted should allow a refined under-
                Such assessments should be outcome specific. The              standing of the potential of different aspects of the
                Cochrane Collaboration has recently formulated                conduct of trials to bias results. Our results suggest that
                detailed guidance on how to do this (see www.cochrane.        efforts to minimise selection and performance bias are
                org/resources/handbook)                                       particularly important when objective measurement of
               Systematic reviewers should present meta-analyses             outcomes is not feasible. It is always possible to conceal
                restricted to trials at low risk of bias for each outcome,    the allocation of patients to intervention groups, but
                either as the primary analysis or in conjunction with less    blinding of participants and physicians is not always
                restrictive analyses                                          feasible. In such situations it is desirable to specify at
                                                                              least one objectively assessed outcome, even if the
                                                                              outcome of most interest is subjective.
           Results in context with other literature                              Systematic reviewers should routinely assess the risk
           Some meta-epidemiological studies, including but not               of bias in results associated with the way each trial was
           restricted to those combined here, have provided                   done. Such critical appraisal should allow for our finding
           evidence that inadequate or unclear allocation con-                that lack of blinding or of adequate allocation conceal-
           cealment and lack of double blinding are associated                ment are particularly important for subjectively assessed
           with over-optimistic estimates of the effect of inter-             outcomes. Some critical appraisal checklists, such as the
           ventions assessed in clinical trials.1-4 However, the              GATE appraisal,12 ask whether the outcome measure-
           evidence is not consistent. Balk et al found no                    ments are either blinded or objective. The Cochrane
           association between adequacy of allocation conceal-                Collaboration is publishing detailed guidance on
           ment or double blinding and effect estimates.5 This                assessing risk of bias in trial results, available at www.
           may be because the largest group of meta-analyses                  cochrane.org/resources/handbook.13
           included in that study were specifically selected                     There is currently no simple or widely accepted
           because the outcome was all cause mortality. Als-                  method to integrate assessments of risk of bias into
           Nielsen et al also found that trials with inadequate or
           unclear generation of the allocation sequence tended to
           exaggerate effects.6 11 Their estimated effect of blinding
           was similar to that reported in our study.                          WHAT IS ALREADY KNOWN ON THIS TOPIC
                                                                               Flaws in the conduct of randomised controlled trials, such as
           Possible mechanisms                                                 inadequate allocation concealment or lack of blinding, may
           Our finding that bias resulting from inadequate                     bias their results
           allocation concealment varied with the type of out-                 Several studies have found that such flaws lead to
           come variable was unexpected. The purpose of                        exaggerated estimates of the effects of medical
           allocation concealment is to avoid selection bias.                  interventions, but other studies have failed to confirm these
           Such selection bias would be expected to be greatest                associations
           when it is easy to assess patients’ prognosis at the time
           they are recruited to a trial, and affect the results of trials     WHAT THIS STUDY ADDS
           with objectively assessed as well as subjectively                   The average bias associated with lack of adequate allocation
           assessed outcomes. Conversely, lack of blinding                     concealment or lack of blinding was less for trials with
           would be expected to introduce bias if knowledge of                 objectively assessed outcomes than trials with subjectively
           intervention groups affected the care received or the               assessed outcomes, and less for trials with all cause
           assessment of outcomes. This could happen indepen-                  mortality as the outcome than for trials with other outcomes
           dently of whether selection bias due to inadequate                  Except when the outcome was all cause mortality, the size of
           concealment of allocation was present.                              bias varied between meta-analyses. Therefore, in situations
              There was a strong association between inadequate                when bias is likely the size of such bias is unpredictable
           or unclear allocation concealment and lack of blinding              Since the bias associated with inadequate or unclear
           (see bmj.com). The bias we observed with inadequate                 allocation concealment varied with outcome, it may result in
           or unclear allocation concealment may result at least in            part from an association with subsequent flaws in the
           part from its association with subsequent flaws in the              conduct of the trial rather than from selection biases. The
           conduct of the trial, in particular with biased outcome             effect of allocation concealment remained even after
                                                                               adjustment for blinding, suggesting it is a marker for other
           assessment, rather than from selection bias. Adequate
                                                                               bias-reducing strategies, beyond blinding
           allocation concealment may also be a marker for other

604                                                                                                         BMJ | 15 MARCH 2008 | VOLUME 336

                                   meta-analyses. The variability in bias between meta-                                  effect in meta-analyses of randomized controlled trials. JAMA
                                   analyses that we identified implies that more emphasis
                                                                                                                    6    Als-Nielsen B, Chen W, Gluud LL, Siersma V, Hilden J, Gluud C. Are trial
                                   in meta-analyses should be given to trials at low risk of                             size and reported methodological quality associated with treatment
                                   bias.14 We believe that systematic reviewers should                                   effects? Observational study of 523 randomised trials. 12th
                                                                                                                         International Cochrane Colloquium, Ottawa, Canada. 2004.
                                   present meta-analyses restricted to trials at low risk of
                                                                                                                    7    Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A.
                                   bias for each outcome, either as the primary analysis or                              Assessment of methodological quality of primary studies by
                                   in conjunction with less restrictive analyses.                                        systematic reviews: results of the metaquality cross sectional study.
                                                                                                                         BMJ 2005;330:1053.
                                      Box 2 summarises our recommendations for                                      8    Sterne JAC, Jüni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical
                                   practice.                                                                             methods for assessing the influence of study characteristics on
                                                                                                                         treatment effects in ‘meta-epidemiological’ research. Stat Med
                                   We thank all authors and funders of the meta-epidemiological studies that             2002;21:1513-24.
                                   contributed data to the present study.                                           9    Huwiler-Muntener K, Juni P, Junker C, Egger M. Quality of reporting of
                                                                                                                         randomized trials as a measure of methodologic quality. JAMA
                                   Contributors: See bmj.com.
                                   Funding: LW was funded by a PhD studentship from the UK Medical
                                                                                                                    10   Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, et al. Bad
                                   Research Council Health Services Research Collaboration.
                                                                                                                         reporting does not mean bad methods for randomised trials:
                                   Competing interests: None declared.                                                   observational study of randomised controlled trials performed by the
                                                                                                                         Radiation Therapy Oncology Group. BMJ 2004;328:22-4.
                                   1     Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of          11   Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C.
                                         bias. Dimensions of methodological quality associated with                      Multivariable modelling for meta-epidemiological assessment of the
                                         estimates of treatment effects in controlled trials. JAMA                       association between trial quality and treatment effects estimated in
                                         1995;273:408-12.                                                                randomized clinical trials. Stat Med 2007;26:2745-58.
                                   2     Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does          12   Jackson R, Ameratunga S, Broad J, Connor J, Lethaby A, Robb G, et al.
                                         quality of reports of randomised trials affect estimates of intervention        The GATE frame: critical appraisal with pictures. Evid Based Med
                                         efficacy reported in meta-analyses? Lancet 1998;352:609-13.                     2006;11:35-8.
                                   3     Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality          13   Higgins JPT, Altman DG, eds. Assessing risk of bias in included
                                         and discrepancies between large and small randomized trials in                  studies. In: Higgins JPT, Green S, eds. Cochrane handbook for
                                         meta-analyses. Ann Intern Med 2001;135:982-9.                                   systematic reviews of interventions. Version 5.0.0. Cochrane
                                   4     Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are          Collaboration, 2008. www.cochrane-handbook.org/.
                                         comprehensive literature searches and the assessment of trial quality      14   Welton NJ, Ades AE, Carlin JB, Altman DG, Sterne JAC. Models for
                                         in systematic reviews? Empirical study. Health Technol Assess                   potentially biased evidence in meta-analysis using empirically based
                                         2003;7:1-76.                                                                    priors. J R Statist Soc A (in press).
                                   5     Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C,
                                         et al. Correlation of quality measures with estimates of treatment         Accepted: 19 December 2007

                                       Congenital coronary artery disease
                                       My 13 year old son Tom was known to have congenital                             At postmortem examination, the origin of the main stem
                                       valvular heart disease affecting all of his valves. The                      of Tom’s left coronary artery was found to be very narrow.
                                       severity of the valve disease was classified by the                          The pathologist had great difficulty in finding the ostium,
                                       consultant paediatric cardiologist he saw regularly as                       which was “slit-like.” Otherwise his coronary arteries were
                                       relatively mild.                                                             normal. The left ventricular muscle was hypertrophied,
                                          At his last review, 15 months before he died, we gave a                   without myocardial disarray, and showed features of
                                       cardiologist a history of central chest pain on extremes of                  chronic ischaemia. The cause of death was given as “I (a)
                                       exertion or cold. There were four or five particularly                       Cardiac arrhythmia. Due to or as a consequence of: I (b)
                                       memorable episodes. One was climbing a mountain, when                        Left ventricular hypertrophy and ischaemia. Due to or as a
                                       the pain subsided promptly with rest. Another was in a                       consequence of: I (c) Congenital abnormality of the left
                                       cold swimming pool, and the pain subsided within minutes                     coronary artery (slit-like ostium).” In part II the pathologist
                                       of warming up. My son recognised “his pain.” We also                         also referred to multiple dysplastic cardiac valves.
                                       gave a history of poor exercise tolerance. I relayed a                          Tom was a marvellous lad. He was kind, generous,
                                       history of what I thought might be angina. On further                        happy, bright, always thinking of others rather
                                       questioning Tom may have agreed that the pain was                            than himself, and popular at school in spite of his
                                       stabbing. We were told that the pain did not sound like                       weak physique. He was much loved and the centre
                                       cardiac pain and that Tom could exercise normally. No                        of our family.
                                       investigation was performed of his chest pain.                                  I confess that, when I described his history of chest
                                          Earlier this year my son was driven to school by his                      pain at his last review, I was not thinking about congenital
                                       18 year old brother, Tim. It was a lovely day, the windows                   coronary artery disease but instead his known aortic
                                       were down, and they were singing along to music together.                    valve disease. Only his age was against the pain being
                                       After walking through the back gate of the school, Tom                       cardiac. Please remember congenital coronary artery
                                       saw a friend and ran to catch him up. Just after disappearing                disease.
                                       round the corner of a building Tom collapsed. Shortly
                                       afterwards his elder brother arrived. Tom was not
                                       responding and was without a pulse. His elder brother                        Andrew James Burton general practitioner
                                       summoned help and gave cardiopulmonary resuscitation.                        jamesburton@dsl.pipex.com
                                       When the paramedics arrived Tom apparently had a pulse
                                       again, but shortly afterwards he went into ventricular                       Tom’s cardiologist saw this report, accepted it as a
                                       fibrillation, and all attempts at resuscitation, including                   reasonable account, and made no objection to it being
                                       defibrillation, failed.                                                      submitted for publication.

BMJ | 15 MARCH 2008 | VOLUME 336                                                                                                                                                              605

To top