How Do I Become a Clinical Research Associate

Document Sample
How Do I Become a Clinical Research Associate Powered By Docstoc
					Clinical Research: How to Avoid
        Magical Thinking

             Reese H. Clark, MD
   Director of Clinical Research, Pediatrix
       Consulting Associate Professor
               Duke University
• Define the different levels of evidence use to
  change clinical care
• Review how failure to do good clinical research is
  associated with bad outcomes
• To discuss different study designs, their strengths
  and weaknesses.
• To provide guidance on how to perform good
  clinical studies that improve neonatal care
         What Changes Care?
• Evidence
  – Publications
  – Meta-analysis
  – Consensus opinion
• Personal opinion (in my experience)
• Institutional history (the way we do it)
• Personal history (the last case I had)
       Experience is the ability
   to make make the same mistake
repeatedly with increasing confidence
  The goal of research is to
discover, learn, understand and
 teach principles that improve
      the quality of life
           Levels of Evidence
• Level 1 – Results from randomized control trials
  with meaningful outcome measure
• Level 2 – Case Control type of studies with treated
  and untreated patients and minimal evidence of
  selection biases
• Level 3 – The patient acts as his or her own
  control or Case control with some selection bias
  i.e., historical controls
• Level 4 – Case series
• Level 5 – Expert opinion based on experience
Pediatrics 2004;114(3):874
• “The best available evidence, however, is not
  always sound or valid evidence. Sometimes, when
  faced with a collection of reports that do not
  constitute good evidence, attempts to choose the
  best evidence become pointless; in this case, a
  statement of no good evidence is preferable.”

• Ambalavanan N et al. Clin Perinatol 2003;
          Definition of Research

• Research means a systematic investigation,
  including research development, testing and
  evaluation, designed to develop or contribute to
  generalizable knowledge.
• Activities which meet this definition constitute
  research for purposes of this policy, whether or
  not they are conducted or supported under a
  program which is considered research for other
• For example, some demonstration and service
  programs may include research activities.
                 What is Research?

• Human Subject Regulations Decision Charts
• The Office for Human Research Protections (OHRP)
  provides the following graphic aids as a guide for
  institutional review boards (IRBs), investigators, and
  others who decide if an activity is research involving
  human subjects that must be reviewed by an IRB under the
  requirements of the U.S. Department of Health and Human
  Services (HHS) regulations at 45 CFR part 46. OHRP
  welcomes comment on these decision charts. The charts
  address decisions on the following:
   – whether an activity is research that must be reviewed by an IRB
   – whether the review may be performed by expedited procedures,
   – whether informed consent or its documentation may be waived.
Most Important Research Issue

Protect the people who are volunteering
to participate in a clinical research study.
Drug Misadventures
         The Willowbrook Study
            Vulnerable Population Rules
• The site where a highly controversial medical
  study was conducted there between 1963 and 1966
  by medical researcher Saul Krugman,
• Healthy children were intentionally inoculated,
  orally and by injection, with the virus that causes
  the disease, then monitored to gauge the effects of
  gamma globulin in combating it. A public outcry
  forced the study to be discontinued.
• Researchers defended the deliberate injection of
  these children by pointing out that the vast
  majority of them would acquire the infection
 “Brought forth by a healer"
                                     480 BC
 Since Hippocrates's
  time, the potential
   damaging effect
 of a healer's actions
has been recognized:
  First do no harm
“primum non nocere”

The road to hell is paved with
good intentions……
-St. Bernard of Clairvaux ~1150 AD
          Iatrogenesis in Neonatology
  • Lowered thermal                             • Chloramphenicol ‘‘gray
    environment increased                         baby’’ syndrome 1956–1960
    mortality 1900–1964                         • Novobiocin jaundice 1957–
  • Supplemental oxygen RLF                       1962
    (ROP) 1941–1954                             • Hexachlorophene brain
  • Initial thirsting and                         lesions 1952–1971
    starving neurological                       • Epsom salts enemas
    deficits 1945–1970                            magnesium intoxication
  • Synthetic vitamin K                           1964–1965
    kernicterus 1945–1961                       • Feeding gastrostomy
  • Sulfisoxazole kernicterus                     increased mortality 1963–
    1953–1956                                     1969
                                                • Benzyl alcohol ‘‘gasping’’
                                                  syndrome –1982

From: Robertson AF, Reflections on Errors in Neonatology (Part I,II,III) J Perinatology 2003
With the best intent we can do great harm.

• Eferol -- Increases NEC, liver injury and death

• Verapamil -- Causes profound bradycardia

• Post-natal steroids -- May increase brain injury

• Overventilation -- Increases the risk of PVL
Martone WJ et al. Illness with fatalities in premature infants:
association with an intravenous vitamin E preparation, E-
Ferol. Pediatrics. 1986;78:591-600.

• A role for vitamin E in the prevention of
  retrolental fibroplasia was first reported in 1949.
• A number of clinical studies between 1978 and
  1983 suggested that vitamin E supplementation (to
  normal or supranormal serum levels) might
  prevent or ameliorate the course of retrolental
  fibroplasia and other complications of prematurity,
  especially following oxygen exposure.
• Vitamin E’s therapeutic or preventive role had not
  yet been clarified before it was used.
                  April 13, 1984 / 33(14);198-9

• CDC has received reports from two hospitals of clusters of an unusual illness
  occurring among low-birth weight (less than 1,500 grams), premature infants in
  neonatal intensive-care units.
• Thirteen affected infants developed clinically significant ascites, in addition to
  some or all of the following abnormalities: hepatomegaly, splenomegaly,
  cholestatic jaundice, azotemia, and thrombocytopenia.
• All affected infants had received parenteral nutrition therapy, in addition to
  other supportive measures.
• An intravenous vitamin E preparation, containing 25 mg/ml vitamin E, 9%
  polysorbate 80 and 1% polysorbate 20 in 2-ml vials (E-Ferol Aqueous
  SolutionR, distributed by O'Neal, Jones & Feldman, St. Louis, Missouri), was
  introduced in each hospital for addition to parenteral nutrition solutions
  approximately 1 month before the onset of illness in the first infant in both
• All affected infants received E-Ferol; some affected infants received up to 1 ml
  or more daily. Both outbreaks ceased shortly after use of E-Ferol was
  Do You Remember E-Ferol? The Penalty for Selling
          Untested Drugs in Neonatology
                 Jerold F. Lucey Pediatrics 1992;89;159;

• In 1984, E-Ferol killed at least 38 newborns.
• Iatrogenic disasters are often caused primarily by well-
  intentioned physicians using logical therapies which turned
  out to have unexpected, lethal side effects.
• A poorly managed, avaricious company, O’Neil, Jones and
  Feldman, Inc. decided to get the jump on the market and sell
  an untested preparation of intravenous vitamin E.
• Physicians assumed E-Ferol had been tested and approved for
  use by the FDA. It hadn’t been tested.
• An astute clinician spotted the problem.
• On January 19, 1989, three defendants pleaded guilty and were
  sentenced to fines of $130 000 each and 6-month jail
  sentences. These penalties were for conspiracy, mail fraud, and
  Cosmetic Act Felony.
• Clinical practice is dynamic
• Study design and execution can take years
• Muticenter studies require coordination
  which can further delay the process
• Publication delay is terrible
• Evidence-based practice requires
  continuous reevaluation of practice
                              Change in Event Rate With Time
                      Term Neonates With Meconium Aspiration Syndrome

                                     Surfactant          Vasopressors          iNO          ECMO
Percent of Patients












                                                    Year of Discharge
Study Types
             Study Designs
• Case Series
  – Retrospective
  – Prospective
• Case-Control
• Randomized Control Trial
  – Crossover design
  – No Crossover
       Case Series -- Reporting Our

• Evaluates the occurrence of an outcome and the
  factors associated with that outcome
• Examples
   – The effect of iNO on oxygenation
   – What factors are associated with, not that cause, IVH?
• Advantage: Easy to do. Easy to get consent.
• Disadvantage: No concurrent controls
• Never proves efficacy or safety
Ways to Strengthen a Case Series
• Define study plan, outcomes measures, and
  statistical methods prospectively
• Avoid hunting for an effect
• Be careful to evaluate the study sample for
  selection bias (e.g., ECMO centers admit
  other institutions treatment failures)
• Do not over-interpret the results
            Case-Control Trials
• Similar to case series except the cases are
  compared to a defined group of controls
• Type of controls:
   – Historical
   – Same period, different location
   – Matched for factors that influence the outcome
• Gives a better sense of efficacy but selection bias
  and confounding variables remain a problem
  Randomized Controlled Trials
• Patients are randomly assigned to one of
  several defined groups. The management of
  each group is strictly defined.
• Crossover design allows patients to
  “crossover” into other treatment groups.
  While easier to get consent for, the results
  are difficult to interpret.
 Outcomes = Study Endpoint = What is
          really important?
• Physiology -- Heart rate, blood pressure, PaO2
• Health consequence -- Survival, chronic lung
  disease, seizures, stroke, learning disability
• Quality of life -- Joyful participation in life
• Health economics -- Did we produce the same
  outcome more efficiently?

“Not everything that can be
      counted counts,
  and not everything that
 counts can be counted.”

                Albert Einstein
• Primary -- The outcome we are most
  interested in studying. Sample size is
  determined by estimating the number of
  patients needed to evaluate this endpoint.
  Every aspect of study design is directed at
  getting a clean measure of this outcome.
• Secondary -- All other measures of
      Characteristics of a Good Study
      Endpoint or Outcome Measure
• Easy to measure and to define
   – Survival is easy to define but hard to study
   – Chronic lung disease is easy to study but hard to define
• Valuable
   – Healthy survival
   – Not a transient rise in PaO2
• Occurs at a frequency that is feasible to study
• Outcome change must be attributable to the
  intervention studied
  Surrogate Outcome Measures
• Definition -- A measure that predicts or is
  closely associated with another measure of
• Example -- Grade 3-4 IVH is often used as
  a surrogate measure (or proxy) of
  neurological outcome. If we decrease the
  rate of severe IVH, we predict that we will
  improve neurological outcome.
   Failure of Surrogate Markers
• Ment LR et al. Pediatrics 1994;93:543-550
   – Low-dose prophylactic indomethacin decreased IVH
     from 18 to 12% in neonates 0.6-1.25 kg
   – Also reduced the rate of grade 3-4 IVH from 5 to 1.4%
   – Survival was not significantly different but was better
     (92 vs. 87%) in the treated group
• Ment LR et al. Pediatrics 1996;98:714-718
   – Follow-up showed no difference in IQ or the
     occurrence of cerebral palsy
      Prophylactic Indomethacin
      Ment et al. Pediatrics 1996;98:714-718

          Indomethacin        Control




       Mortality    All IVH        IVH 3-4   CP
             Postnatal Steroids
• Meta-analysis shows that steroids reduce the risk of
  CLD in premature neonates (Bhuta et al. Arch Dis Child
• CLD is associated with poor neurodevelopmental
• It might be expected that steroids might improve
  neurodevelopmental outcome
• Instead early steroids increase neurodevelopmental
  problems (Yeh et al. Pediatrics 1998;101)
• Summarize the results of different research
  studies of related problems
• Systematic approach to the identification
  and abstracting the critical information held
  in each study
• Present a comprehensive best estimate
  meant to summarize what is known about
  the clinical problem
        LeLorier et al. NEJM 1997;337:536
              Positive   Negative

Positive      13         6          19

Negative      7          14         21

Total         20         20         40
   Evaluation of Meta-analysis
Sensitivity                 65%
Specificity                 70%
Positive Predictive Value   68%
Negative Predictive Value   67%
Kappa value                 0.35 (0.06-.64)
• Relative Risk -- The probability (risk) of being
  treated with ECMO if you get iNO compared to if
  you did not get iNO (%ECMO use in iNO treated/
  %ECMO in control not treated with iNO)
• Odds Ratios -- The rate that ECMO patients are
  treated with iNO compared to patients who do not
  get ECMO. (%iNO exposure in ECMO patients) /
  (%iNO use in non ECMO patients). Better
  applied to morbidity factors like ICH, or CLD

• Confidence intervals -- How certain are you that
  the observation falls within your measured result.
  Usually the number is 95% CI
• Standard Deviation -- a measure of average
  variance from the mean (Square root of
  {Sum(individual values - mean value)2/number of
• Standard Error of the Mean -- STD/Square root
  of the sample size.
Relative Risk of Outcome

Decreased death
Good confidence

    Effect but no

       No effect

 Increased death
Good confidence

                    .2   .25   .33   .5   1   2   3   4   5

                         Relative Risk of Outcome
 Efficacy or Equivalency or Non-inferiority?

• Efficacy trials are directed at proving that one therapy is
  better than another with 95% confidence
• Equivalence implies that the two therapies produce the
  same outcome
   – If one therapy reduces health care cost, then we may only want to
     show that the two approaches produce similar outcomes
• Non-inferiority are done to show that patients treated with
  X do no worse than those treated with Y. Like efficacy
  but only one tail-test are used. Required sample size is
  Relative Risk of Outcome

The new therapy is
better and no risker

       Effect but no

  Equivalent but no

  Good confidence

                       .2   .25   .33   .5   1   2   3   4   5

                            Relative Risk of Outcome
Relative Risk of Death and/or ECMO
(% Death or ECMO iNO/ % in Control)

   NINOS (n = 235)

    INOSG (n = 58)

    Boston (n = 90)

  Ohmeda (n = 155)

     Total (n = 538)

                       0.00         0.50       1.00         1.50

                              Relative Risk of Death/ECMO
  Relative Risk of Death

 NINOS (n = 235)

  INOSG (n = 58)

 Boston (n = 90)

Ohmeda (n = 155)

  Total (n = 538)

                    0   1      2   3   4   5    6    7   8

                            Relative Risk of Death
Rate of ECMO or Death
NINOS trial, NEJM 1996
       iNO      Control




        MA         RD        Sep         PPH      CD
          S(n         S(n        sis(        N(n       H(n
                =11       =25         n=5        =41      =53
                   6)        )           0)          )       )
Problems With the General Application of
              Any Model
• Standardized Rate -- Observed outcome rate divided
  by the predicted rate
• Inadequate sample size
• Selection biases
• Neonatal care changes and the model must be
• May be slow in identifying poor performers if we
  have to wait for adequate sample size
        Sample Size Calculations

• Dependent on:
  – The absolute event rate in the population
    being studied
  – The absolute difference between the two
  – How certain you want to be in the measured
Effect of Sample Size On Confidence Interval and
   Probability that the Proportions are Different
           90%     p=0.6                           p=0.16                    p<0.001

                 Gr             Gr             Gr            Gr              Gr               Gr
                    p   1(         p   2(         p   1(        p     2(        p     1(         p     2(
                          n=             n=             n=              n=              n=               n=
                             10             10             10              10              10               10
                                )              )              0   )           0   )           0   0)           0   0)

                                            Sample Size
Site Variation
                              Center Differences & Outcomes
                          of Extremely Low Birth Weight Infants
Perecentage Died or Had
 Developlmental Delay







                                        Vohr B et al. Pediatrics April 2004, pp 781
         Across Site Variability
• Most sites randomize within center
• Each site is a mini-trial
• Differences in care are variable within sites and
  may effect the efficacy of the drug being studied
  within that site
  –   steroids
  –   nosocomial sepsis
  –   nutrition
  –   saturation targets
Site Variability in Proportion of Neonates Alive & Off Oxygen
                Treatment Study (“All Treated”)

                                       Surfactant 1   Surfactant 2   All Patients At Site

    Percent Alive & Off Oxygen





                   Difference between Group 1 and Group2
               Alive and off oxygen PMA36 for Treatment Study

                                                        Surfactant 2 better
Group 2 - Group 1
Primary Outcome




                           Surfactant 1 better

           Testing a Test
Test        Disease     Disease
            Present     Absent
Positive      A            B         PPV

Negative       C           D         NPV
           Sensitivity Specificity
            A/(A+C) D/(B+D)
• Create a network of centers with a common
  goal to answer important questions
• Define answerable questions
• Limit data collection to confounding variables
• Define strict time lines and review progress on
  a quarterly basis
• Present findings at national meetings
• Interpret results carefully
“For I was assailed by so many
 doubts and errors that the only
 profit I appeared to have drawn
from trying to become educated,
       was progressively to
 have discovered my ignorance.”

     Descartes, Discourse on Method, 1637.

Description: How Do I Become a Clinical Research Associate document sample