Are the results valid by fuw70346

VIEWS: 21 PAGES: 26

									        Are the results valid?
• Was the validity of the included studies
  appraised?
             Are the results valid?
• Was the validity of the included studies appraised?
• When conducting a meta-analysis the overall results
  are weighted by the size of the contributing studies.
  However further weighting may be given to the
  most rigorously conducted studies. This can be
  achieved by making a judgement on the validity of
  the studies. This may be qualitative (a comment
  about key characteristics of the study) or
  quantitative where each study is scored say out of
  10, and the results of the study are then weighted
  according to the score achieved.
Give an example of the Oxford
       scoring system.
        Are the results valid?
• Were the assessments of the included
  studies reproducible?
   Were the assessments of the
  included studies reproducible?
• If the quality of the studies have been
  assessed then would the reader be able to
  understand the scoring system used and
  apply it to the papers?
• Some meta-analyses may appraise the
  studies individually and give a score out of
  ten, but not make transparent why a paper
  scored badly - was it because there were a
  lot of drop outs from the study,or because
  the randomisation had been poorly done etc.
        Are the results valid?
• Were the results consistent from study to
  study?
              Combinability
• First of all one should ask whether the
  results of separate trials can be
  meaningfully combined?
  – What criteria were used to decide that studies
    were similar enough to be combined??
  – Issues include study designs, types of
    patients and outcomes E.g. Can we combine
    RCTs with observational studies, or studies
    using cotrimoxazole with those using
    trimethoprim?
 Were the results consistent from
         study to study?
• Each study is considered to be from a
  different population, the rate varies from
  study to study and differences can be
  because of experimental error, chance or
  differences in the populations. There are
  tests for homogeneity (or heterogeneity)
  which help decide the degree of caution in
  interpreting or pooling the results.
     Were the results consistent from
             study to study?
• In general differences in treatments are likely to
  differ in magnitude rather than direction and this
  can be displayed graphically to support or refute
  homogeneity between studies. Results are
  homogeneous if they reflect the same “true” effect.
• References
   – Tests for homogeneity of effect in epidemiologic investigation.
     American Journal of Epidemiology 1977; 106: 125-9
   – Meta-analysis in clinical research. Annals of Internal Medicine
     1987; 107: 224-33
 Add in some diagrams showing studies with
heterogeneity and some that are homogeneous
        II. What are the results?
1. What are the overall results of the study?
  – Look at the Relative Risk (RR) of the main
    outcome in the two groups. In a meta analysis
    this is usually presented as a diamond with
    the 95% confidence intervals grouped about a
    line indicating the null hypothesis.
  – A number of outcomes may be considered at
    once so more than one diamond may be
    presented.
Meta-analysis
                PETO GRAPH
                  Outcome A
        Study

        A
        B
        C
        D
        E


        Total

                Placebo   0   Drug
         II. What are the results?
1. What are the overall results of the study?
  – Look at the Relative Risk (RR) of the main
    outcome in the two groups.
  – What about sub-group analyses? I.e. what are the
    results of the individual studies. Do some studies
    stand out as being different. Have the log odds
    ratios been calculated or has the meta analysis
    managed to pool original data in which case other
    sub group analyses may be possible - one of the
    advantages of pooling the original data.
       II. What are the results?
1. What are the overall results of the
  study?
  – Look at the Relative Risk (RR) of the main
    outcome in the two groups.
  – What about sub-group analyses?
  – Can you calculate the Number Needed to
    Treat (NNT) from the results presented? This
    necessitates an estimate of the absolute risk in
    the treated and control groups rather than
    simply having the odds ratio presented.
      II. What are the results?
2. How precise are the results?
      II. What are the results?
• How precise are the results?
• A meta analysis should always present the
  95% confidence intervals for each result.
  III. Will the results help me in
       caring for my patients?
1. Can the results be applied to my patient
  care?
Will the results help me in caring
         for my patients?
1. Can the results be applied to my patient
  care? Clinical significance.
• Refer back to the clinical problem
• Are the studies generalisable to our patient?
• Age, ethnicity, community or hospital
  patients etc?
            Generalisability
• Major problem with RCTs - i.e. do the
  results of this study apply to my patients?
• Need to know details of patients included in
  all the studies. This will help decision about
  the generalisability of the results and also
  may reveal reasons for heterogeneity
Will the results help me in caring
         for my patients?
2. Were all the clinically relevant outcomes
  considered?
Will the results help me in caring
         for my patients?
2. Were all the clinically relevant outcomes
  considered?
• What about other outcomes - particularly
  harm.
• What about quality of life issues?
Will the results help me in caring
         for my patients?
3. Are the benefits worth the harms and costs?
Will the results help me in caring
         for my patients?
3. Are the benefits worth the harms and costs?
• Cost differences in treatments.
• Greater benefits and less side effects?
      Critical Appraisal of a meta
        analysis - methodology
•   Selection bias
•   Generalisability
•   Combinability
•   Consistency
•   Statistics
•   Sensitivity analysis
                  Statistics
•   Pool the log odds ratios
•   Standardised average
•   Regression analysis on pooled data
•   Sub group analysis
         Sensitivity analysis
• Can be done using quality score of trials
  e.g. published vs unpublished, observational
  and RCTs, details of randomisation
• Publication bias - i.e. how many studies
  showing no difference would have to exist
  but not published to invalidate my results?

								
To top