A Statistical Reflection on Adapative Study Designs

Document Sample
A Statistical Reflection on Adapative Study Designs Powered By Docstoc
					A Statistical Reflection on
Adapative Study Designs
       Dean Follmann
            NIH
                      Adaptation
• Modify some aspect of the clinical trial, while it is
  being conducted.
   –   Drop arms
   –   Modify sample size
   –   Change inclusion criteria
   –   Extend follow-up
• Appeal
   – Want answers quickly
   – Don’t know everything
   – You get surprised, need to change
   Adaptation 1—Quick Answers
• For a long time we’ve allowed for early
  termination if the question is settled.
• e.g. Look at test statistic after
  – 25%, 50% 75% 100% of the total infections
• P<.05 happens more than 5% of the time if we
  have multiple looks. Can correct for this
  statistically.
    Adaptation 1- Quick Answers
• Frequent looks require a <5% increase in planned
  sample size if focus is on identifying effective
  vaccines w/ a traditional approach.
• Aggressive identification of
  harmful/useless/weak with good power requires
  a 30% increase in planned sample size.
  – So p(weak) = .12 goes to p(weak) = .04
• Frequent looks require more reports from
  SCHARP.
  Adaptation 2: Don’t know much
• Monitoring may reveal interesting trends.
  – Differential effect by subgroup
  – Waning immunity, VE
  – Potential harm
• May be tempting to tinker with trial design.
  – Trends are unreliable early on.
  – Feel more comfortable sticking with the design.
                      MUSTT
• Multi Center Unsustained Tachycardia Trial


                          EP testing + antiarrhytmics

      Patients with
      arrhythmias




                              No Antiarrhymics
The Overshoot: PAVE100
       End of Study Adaptation
• After the study is completed, infections will
  continue to accrue.
• PAVE100:
  – 200 infections in 3 years
  – 100 “wasted” infections in 4th year
• Suppose VE=50% p=.001
• FDA likes two p<.05 for licensure
                          Another Study?
             Main Trial                Another
             Follow-up                 Study




                           Maintain

Randomize
To Vaccine

                            Maintain




             Vaccine shots + FU                  Placebo shots + FU
                Another Study
• Maintain arms & extend trial for 2 years, get
  200 more infections.
• Is trial extension an “independent” study?
   – Future 200 infections independent of past
   – Future 200 infections estimate late VE
   – Future 200 from same pool of original
     uninfecteds.
• Some studies divide sites to form ``two” trials
      End of Study Adaptation
• Suppose VE = 50% p=.001.
• Immunity & VE are waning. What to do?
                            Boosterism
             Main Trial                Extension 1           Extension 2
             Follow-up                 Follow-up             Follow-up



                                                 Randomize
                           Vaccinate              To boost
                           Everyone
Randomize
To Vaccine
                          Randomize
                           To boost




             Vaccine shots + FU                      Placebo shots + FU
                 Boosterism
• Use all uninfecteds to address question of
  boosting.
• Years faster than a new trial
• Like two strata within a ``boosting” trial
  – Those who get 1st shots in 2010 boosted in 2012
  – Those who get 1st shots in 2012 boosted in 2014
         Bayesian Approaches
• Basic idea---use your beliefs about the vaccine
  to blend with data to get more efficient
  analysis.
• Non-Bayesian---don’t trust your beliefs, only
  use the data.
            Bayesian Example
• Want to estimate the vaccine & placebo infection
  rates.
• Believe the vaccine infection rate is about 1%
  placebo infection rate is about 2%. Belief is
  equivalent to 1000 patients.
• Data
  – 40/1000 vaccine rate   4%         VE=12%
  – 45/1000 placebo rate 4.5%
• Bayes rates
  – (40+10)/2000 vaccine rate 2.50%   VE=23%
  – (45+20)/2000 placebo rate 3.25%
                      View
• Bayesian approaches are a means to an end.
  Can be more efficient, smoother, etc if well
  calibrated.
• Not really suited for Phase III licensure quality
  inference.
• OK in phase II, appealing to have the prior
  belief equivalent to 10% or less of planned
  sample size
• Is HIV vaccine field well-calibrated?
              The long view
• Adaptation strategy best evaluated using a
  long view.




1984                  Now                      2036
         Adaptation Evaluation
• Most statistical metrics describe a single trial
  – Power = probability identifying one good vaccine
  – Duration = time to knowledge about one vaccine
• Better metric. How many years to identify a
  successful vaccine under different statistical
  designs.
          Adaptation Evaluation
• Assume always at least two vaccine candidates to
  test. Most vaccines useless.
       good     useless                         Current Vaccine
                                                Candidates

• Strategy 1 Sequential
   – Undertake 2 arm vaccine trials V, P
• Strategy 2 Concurrent
   – Undertake 3 arm vaccine trials V1, V2, P
• Use frequent monitoring for each.
      2 vaccines at a time
 1               3                       5   7
 2                   4       6



Now                                              2036




 1     2     3       4   5       6   7



           1 vaccine at a time
              Results Pending
• Concurrent evaluation: 3 arms V1 V2 P
  – Optimal to allocate P:V    1.41 : 1
  – Typically use alpha/2 to control type I error rate
• Sequential evaluation: 4 arms: for V1 V2 P1 P2
• Evaluation can be applied to other issues.
  – Should 2 arm-trials have e.g. 176 infections or 131
    total infections.
               Conclusions
• Frequent monitoring makes perfect sense.
• Wary about aptation based on interim
  between arm comparison. Concern about
  over-reaction to unreliable trends.
• Bayesian approaches may be useful in phase II
  studies. Not phase III.
                  Conclusions
• Need to be able to act quickly on trial results
  – Extend FU for “overshoot” infections.
    Like a separate trial addressing distal VE
  – Vaccinate placebos
    Address question of boosting
  – Something we can’t imagine.
• Taking the long view & being pessimistic
  prudent way to choose the “default” design.
The Overshoot: WWII
              Boosterism Benefit
                        Immune Response Quartiles
Group Outcome           Weak   Modest   Good    Best   Total
Vaccine Uninfected       70     85      90      95     340
        Infected         30     15       10      5      60
        Total           100    100      100     100    400
Placebo Uninfected       70    75       75       80    300
        Infected        ~30    ~25      ~25     ~20    100
        Total           ~100   ~100     ~100    ~100   400

  Observed   Inferred

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:5/24/2013
language:English
pages:27