Definitions of Evidence

Document Sample
Definitions of Evidence Powered By Docstoc
					      The Slight, the Fair and the Very
  Strong: the Meaning of Evidence in
Public Health from Bradford Hill to the
                Campbell Commission

                         R.E.G. Upshur
  Department of Family and Community
  Medicine, Public Health Sciences and
   Joint Centre for Bioethics, University
                              of Toronto
   Historical prelude: Fisher vs Hill
   Definitions of Evidence
   Characteristics of Evidence
   Models of Evidence
   Precaution and the Moral Epistemology of
    Public Health
   Conclusions
Lord Kelvin
   I often say that when you can measure what you are
    speaking about and express it in numbers, you
    know something about it; but when you cannot
    measure it, when you cannot express it in numbers,
    your knowledge is of a meagre and unsatisfactory
    kind; it may be the beginning of knowledge, but you
    have scarcely in your thoughts advanced to the
    state of Science, whatever the matter may be.
   Against that positivism which stops before the
    phenomena, saying “there are only facts” I
    should say: no, it is precisely facts that do not
    exist, only interpretations…
Gottlob Frege
   It cannot be required
    that everything should
    be proved, since that is
    impossible; but one can
    see to it that all
    propositions that are
    being used without
    being proved, are
    expressly stated as
    such, so that it is clearly
    known on what the
    whole structure rests.
Ronald Aylmer Fisher
Fisher: Cigarettes, Cancer and
Statistics, 1958
   Before one interferes with the peace of mind
    and habits of others, it seems to me that the
    scientific evidence-the exact weight of the
    evidence free from emotion-should be rather
    carefully examined.
   Requirements are: replication, randomization
    and control
Austin Bradford Hill
Hill: The Case for Action
   …in “real life” we shall have to consider what
    flows from that decision. On scientific
    grounds we should do no such thing. The
    evidence is there to be judged on its merits
    and the judgment should be utterly
    independent of what hangs upon it or who
    hangs because of it. But in another and more
    practical sense we may surely ask what is
    involved in our decision.
Definitions of Evidence
   The Oxford English Dictionary gives several
    senses of the term evidence:
    The quality or condition of being evident
    Manifestation
    That which makes evident, mark, trace
    Ground for belief; that which tends to prove
    or disprove any conclusion
Medical Definitions
     Evidence is a property of data that makes us alter our
      beliefs about how the world around us is working.
      Another way to say this is that evidence is the basis
      upon which we derive inferences. (Goodman and
      Royall 1988)
     In its simplest form, the available scientific evidence
      consists of the published report of a single piece of
      original research. (Miettinen 1998)
Legal definitions
   Any species of proof or probative matter legally
    presented at a trial of an issue, by the act of the
    parties and through the medium of witnesses,
    records, documents, exhibits, concrete objects for
    the purpose of inducing beliefs in the minds of the
    court or the jury as to their contention
   All means by which any alleged matter of fact, the
    truth of which is submitted to investigation,is
    established or disproved.
Evidence Based Public Health
   Evidence based public health can be defined
    as a public health endeavour in which there is
    an informed, explicit, and judicious use of
    evidence that has been derived from any of a
    variety of science and social science
    research and evaluation methods.
Canadian Health Services
Research Foundation
   Evidence is information that comes closest to the
    facts of a matter. The form it takes depends on
    context. The findings of high-quality,
    methodologically appropriate research are the most
    accurate evidence. Because research is often
    incomplete and sometimes contradictory or
    unavailable, other kinds of information are
    necessary supplements to or stand-ins for research.
    The evidence base for a decision is the multiple
    forms of evidence combined to balance rigour with
    expedience—while privileging the former over the
Characteristics of Evidence
Upshur JECP 2000

   Provisional, Defeasible,
   Emergent
   Incomplete
   Constrained: Ethical, Economic, Computational
   Collective
   Asymmetric
   Historical
   Market Influenced
Provisional, Defeasible,
   For example Peptic Ulcer Disease
   3 distinct pathogentic and therapeutic models
    in last three decades
   1970’s: stress, V&P, White Diet
   1980’s: Hypersecretion, H2 Blockers
   1990’s: Infection and Eradication
Warren and Marshall
Ethical Constraints
   Ethical: Some things we will not know because it is
    unethical to perform certain studies (Is this an
    absolute or relative restraint?
   Fisher: It is not the fault of Hill or Doll that they
    cannot produce evidence in which a thousand
    children of teen age have been laid under a ban that
    they shall never smoke and a thousand more
    chosen at random from the same age group have
    been under a compulsion to smoke at least thirty
    cigarettes a day. If that type of experiment could be
    done, there would be no difficulty.
Economic Constraints
   Economic: Had we but money enough and
    time, this lack of evidence would be no crime
   Evidence of certain kinds is expensive (see
    cohort studies)
Computational Constraints
   Computational: There are limits in terms of what can be known
    given the resources required to perform the calculations.
   C. David Naylor: Another difficulty arises from the Malthusian
    growth of uncertainty when multiple technologies are combined
    into clinical strategies. Take two technologies and they can be
    used in two different sequences; take five and the number of
    possible sequences is one hundred and twenty. Furthermore the
    elements in a clinical strategy are usually tested in separate
    studies leaving few data on the chains of conditional probabilities
    that link sequences of tests, treatments and outcomes.
Computational Constraints
 Hierarchical, serial clinical trials would permit
  identification of the optimum combination of these
  agent classes for Alzheimer disease through 127
  trials, enrolling 63,500 patients, requiring 286 years;
  for ischemic stroke through 31 trials, enrolling
  186,000 patients, requiring 155 years.
Saver JL. Kalafut M. Combination therapies and the
  theoretical limits of evidence-based medicine.
  Neuroepidemiology. 20(2):57-64, 2001 May.
   Evidence is collective
   Volume of information to great for any one
    person to master
   Evidence, then exists as a collective property
   There are differences in evidence access
    both within and between disciplines. This
    relates to both funding and access to
    intellectual resources. Some issues are not
    sufficiently prevalent to warrant full scale
    attention (rare diseases); some lack appeal
    from the point of view of funders (vitamins,
    public health interventions eg restaurant
   Many health care professions do not have an
    established history of commitment to
    research. Indeed, it is only in the last 30
    years that the RCT and in the past 10 years
    that the systematic review assumed
    prominence in medical conceptions of
    evidence. Other disciplines have neither the
    intellectual tradition, nor commitment to the
    production of researchers.
Markets: Alolaprilodine
   Market forces create redundancies
   However, evidence claims are frequently
    made (e.g. receptor blockade)
   Evidential indiscernables
Time Constraints
   There is a necessary trade off between the
    need to search for information and make
The conflicting hierarchy
   First used by Periodic Task Force for
    Preventive Care
   Now a proliferation of such hierarchies
   None are based on research, all are based
    on expert judgement or consensus
Conflicting Hierarchies
Upshur CMAJ 2003

                                                                                             Grade (A) of Recommendation
                                              Highest Level of Evidence for a treatment or

The Canadian Task Force on Preventive         1: Evidence from at least 1 properly           A=Good evidence to support the
Health Care 1                                 randomized controlled trial (RCT).             recommendation that the condition be
                                                                                             specifically considered in a PHE.

Osteoporosis guidelines2                      1+: systematic review or meta-analysis of      A= need supportive level 1 or 1+ evidence
                                              randomized controlled trials                   plus consensus
                                              1: one randomized trial with adequate power

Centre for Evidence Based Medicine Oxford3    1a: Systematic review with homogeneity of      A= consistent level 1 studies
                                              randomized controlled trials
                                              1b: Individual RCT with narrow confidence
                                              1c: all or none

Scottish Intercollegiate Guidelines Network   1++ High quality meta-analyses, systematic     A= At least one meta-analysis, systematic
grading system 4                              reviews of randomised controlled trials        review, or RCT rated as 1++ and directly
                                              (RCTs), or RCTs with a very low risk of bias   applicable to the target population; or a
                                              1+ Well conducted meta-analyses,               systematic review of RCTs or a body of
                                              systematic reviews of RCTs, or RCTs with a     evidence consisting principally of studies
                                              low risk of bias                               rated as 1+ directly applicable to the target
                                              1- Meta-analyses, systematic reviews or        population and demonstrating overall
                                              RCTs, or RCTs with a high risk of bias         consistency
Evidence Based Public Health
   Descriptive: to identify the qualities and distributions of variables;

   Taxonomic: to compare and classify variables into related groups or

   Analytic: to examine associations between variables—these may be
    hypothesised causal or therapeutic relations;

   Interpretive: to identify and explain meanings, usually from a particular

   Explanatory: to make observations intelligible and understandable; and

   Evaluative: to determine quality and worth—often assessing the
    relevance, effectiveness, and consequences of activities.
                 Q1                Q2
              Concrete/         Historical/
              Historical          Social
Particular                                     General
                 Q4                  Q3
              Personal/          Impersonal
             Mathematical       Mathematical

Krever Commission
In The Commission of Inquiry on the Blood
  System in Canada, Mr. Justice Krever said:
Where there is reasonable evidence of an
  impending threat to public health, it is
  inappropriate to require proof of causation
  beyond a reasonable doubt before taking
  steps to avert the threat.
Campbell Commission
The Commission therefore recommends:
That the precautionary principle, which states that action to reduce
  risk need not await scientific certainty, be expressly adopted as a
   guiding principle throughout Ontario’s health, public health and
   worker safety systems by way of policy statement, by explicit
   reference in all relevant operational standards and directions,
   and by way of inclusion, through preamble, statement of
   principle, or otherwise, in the Occupational Health and Safety
   Act, the Health Protection and Promotion Act, and all relevant
   health statutes and regulations.
That in any future infectious disease crisis, the precautionary
   principle guide the development, implementation and monitoring
   of procedures, guidelines, processes and systems for the early
   detection and treatment of possible cases.
Where does this leave us?
   Is precaution appropriate to all public health
    interventions or only communicable
   Would thresholds vary between
    communicable diseases, chronic diseases,
    health promotion, environmental health?
   Which formulation of the precautionary