Bias by liuqingyan



     Lecture 7 : Sources of Error:
 Selection and Measurement Bias
Learning outcomes

 Identify major sources of bias in
 research studies
 Describe ways to overcome bias in
 research studies
 Seeking the true answer in
 Aim of any study is to estimate the
 „true value‟ of a health event/disease
 or association
 The best that can be achieved in the
 real world is to make an empirical
 estimate of the true value (theoretical
 value) by observing the experience of
 a sample of the population that is
 specific in time and place
Possible outcomes of the study

Results of the           Truth – Actual
Study              No effect          Effect

No effect           Correct       Error (type 2)

Effect           Error (type 1)      Correct

 True value ~ is known as theoretical value
 Need to identify and control sources of
 External validity - extent to which the results
  of a study can apply to people outside the
  study – the target population
  ie how generalisable are the results to others
 Internal validity - extent to which a study
  properly measures what it is meant to:
   ie whether the conclusions are 'true'
    for the people in the study.
                                  Internal Validity

  Target          Study       Exposed
                Population                   Outcomes   Results
Population      or Sample    Unexposed

                         External Validity
 Seeking the true answer in
 Error means a departure of the
 empirical (observed) estimate from its
 theoretical value „true value‟

Errors can occur:
 By Chance  Random Errors
 Systematically     Systematic Errors
Sources of Error

 Random Error
 Selection Bias
 Information Bias
 Confounding
Random Error (Random
Sampling Error)
 Difference between an empirical
 (observed) estimate and its true
 value that is due to chance variation
Example: Prevalence of arthritis
in women 45-69years
 Study 1: a survey - 1 in 20 sample of women
 aged 45 to 69 years resident in Lidcombe in
 May 1994
     Results: Arthritis Prevalence: 30 /100 population
 A survey using another 1 in 20 random
 sample of women aged 45 to 69 years
     Results: Arthritis Prevalence: 40/100 population
 Entire population was surveyed
     Results: Arthritis Prevalence: 38/100 population
Random sampling variability

 The random sample surveys differ
 from the true population value
 because of random sampling
Random sampling variability

 How confident can we be that our
 estimate is close to the true value?
 A confidence interval is a range
 within which the true value is
 expected to lie
 Large samples have smaller CIs as
 they are less prone to chance error
 „systematic distortion or deviation in the
  study results from the true value‟
 „systematic difference between what the
  study estimates and what it is intended to
 Systematic error (bias) is present when
  the theoretical value of an epidemiological
  measure cannot be obtained from a study
  even in the total absence of random error
  overestimation or underestimation of
  the effect due to a deficiency in the design
  or execution of the study
  the likelihood that the study results will
  be distorted from the true value
 Bias results from systematic flaws in
     study design
     data collection
     analysis of results
Example - Prevalence of arthritis

 If we had a poor study design in our
  Lidcombe study and failed to include
  residents from two large nursing
  Result: arthritis prevalence:25/100
 Our prevalence estimate has a
  systematic error or bias
Detecting Bias
 Usually impossible to detect bias by
 looking at a prevalence estimate or
 other measure of frequency or
 association because we don‟t usually
 know the „true value‟
 Look for sources of bias in the study
 methods used
Selection Bias
 Error due to the way the study participants
  are selected
 Occurs when:
     Subjects are not representative of the target
     If subjects select themselves
     If subjects in the study are different from those
      not in the study
     If the disease/factor under investigation makes
      people unavailable for study
 Study looking at drinking habits in
 young people.
   Researcher  chose to interview a random
    sample of all people 18-25 years (students
    and academics) at the Cumberland
    Campus, University of Sydney
   Problems with the study??
  Sources of Selection Bias
 Poor selection of subjects from the study
      non-random selection
            eg volunteers may be different from
            those who do not enrol; (important in
            cross-sectional surveys; controls in
            case control studies)
      ill defined populations eg hospital controls used
       in case control study (different criteria for
       selecting cases and controls leading to
       exclusion bias) e.g. association of reserpine and
       breast cancer
  Sources of Selection Bias
 Poor selection of subjects from the study
      failure to locate or unwillingness to participate
       (response bias – those who agree to
       participate may be in some way different from
       those who refuse to participate or are not
      loss due to health outcome (healthy worker
       effect, survival bias)
Sources of Selection Bias
Non-participation can be due to:
 Lack of interest or personal relevance

 Inconvenience

 Avoidance of discomfort

 Financial cost

 Anxiety about health

 Antipathy to research
Sources of Selection Bias
Healthy worker effect/bias:
 Usually associated with environmental /
  occupational studies
 Low rates of morbidity and mortality found in
  both exposed and unexposed in the workplace
 To be active at work – need to be reasonably
  excludes people who are unhealthy.
Sources of Selection Bias
 Non-random assignment of exposure -
  where a doctor allocates treatment or
  patients select which group they belong
 Omission of subjects from analysis
   loss to follow-up – those who are lost to
    follow up or withdraw from the study may
    be different from those who are not
    followed for the entire study
   inability to obtain adequate measurements
    eg missing data
Information Bias

 Systematic error in the measurement
 of information about exposure or
 Error due to the incorrect
 classification of exposure or disease
Types of Information Bias:
 Recall bias (or subject error)– those with
  a particular outcome or exposure may
  remember events more clearly or amplify
  their recollections (case-control studies)

  Eg mothers of children with birth defects

 Instrument error – problem with the
  instrument eg ear temperature instrument
  – not calibrated
    Types of Information Bias ctd:
 Follow-up or surveillance bias (Cohort
  studies) – group with known exposure or
  outcome may be followed more closely or
  longer than the comparison group
 Hawthorne effect - people act different if they
  know they are being watched
 Observer bias (Rosenthal effect) – observers
  may have preconceived expectations of what
  they should find in an examination
    Interviewer bias - an interviewer‟s
     knowledge may influence the structure of
     questions and the manner of presentation,
     which may influence response
    Intra-observer versus inter-observer bias
Types of Information Bias ctd:
 Misclassification error – errors are made
  in classifying disease or exposure status
 Non-differential (random) misclassification
    subjects are misclassified with respect to
     exposure or case or control in a random
    Subjects have an equal chance of being
    This weakens the association observed
     between outcome & exposure, if a real
     association exists
 Assume that the true incidence rates of
  measles in
     non-vaccinated children is 10/1000 person
     1/1000 person years in vaccinated children.
     Therefore the theoretical incidence rate ratio
      will be 10.
 If 10% of the non-vaccinated children are
  misclassified as vaccinated and 10% of
  the vaccinated children are misclassified
  as not being vaccinated, then the
  observed relative rate will be 4.8
 Differential (non random) misclassification
    subjects are misclassified with respect to
     exposure or case or control in a
     non random way

   This changes the direction of the association
    observed between outcome & exposure
   E.g. association between alcohol intake and
    memory loss. Cases have a greater chance
    of being misclassified
“Intention to Treat” analysis in

 Analyse groups as they were
 originally randomised
“Once randomised always analysed”
 NOT by comparing the control group
 with only those in the drug group
 who actually took the drug
Choice of Analysis with RCTs
1. Compare those who actually took the
   drug and those who actually took the
    This avoids misclassification error
   Potential selection bias
   this ignores the original randomisation
   experimental data  observational data
Choice of Analysis with RCTs
2. Compare those in the drug group to
   those in the placebo group (ie the
   original randomised groups)
      = “intention to treat” analysis
      of what happens in real life.
Our Aim in research is to:

 Reduce bias
 Identify those biases that can‟t be
 Assess their potential impact
 Use this information when
 interpreting results
 Controlling Bias

 Choice of the study population and
 Control Group

   Select
         subjects and controls with
   respect to area of residence,
   occupation, place of employment
  Controlling Bias
Methods of data collection
 Construct specific data collection
  instruments eg q‟aires, interviews,
  physical examination.
 Use objective closed ended questions
 No differences in implementation
  between groups
 Study personnel to administer
  standardised instruments according to
  explicit instructions
 Controlling Bias
 Study personnel have standardised
   Follow specific procedures identical for all
   Adopting uniform ways of probing and
    responding to questions
   Using standard techniques to deal with
    missing information
 Study personnel blinded if possible

 Use multiple sources of data where
     Controlling Bias
                  Selection                      Measurement
                    Bias                            Bias

    Population    Participation Study Follow      Outcome        Analysis
     Selection      / Group          up          Assessment
                      High                       Staff trained
                  participation   No drop outs    in standard    Intention
Representative         rate                       procedures      to treat
 No differences                                    Measuring      analysis
between groups    Randomised                     instruments
                      group                        calibrated
                   allocation                      No recall

To top