# A Statistical Reflection on Adapative Study Designs

Document Sample

```					A Statistical Reflection on
Dean Follmann
NIH
• Modify some aspect of the clinical trial, while it is
being conducted.
–   Drop arms
–   Modify sample size
–   Change inclusion criteria
–   Extend follow-up
• Appeal
– Don’t know everything
– You get surprised, need to change
• For a long time we’ve allowed for early
termination if the question is settled.
• e.g. Look at test statistic after
– 25%, 50% 75% 100% of the total infections
• P<.05 happens more than 5% of the time if we
have multiple looks. Can correct for this
statistically.
• Frequent looks require a <5% increase in planned
sample size if focus is on identifying effective
• Aggressive identification of
harmful/useless/weak with good power requires
a 30% increase in planned sample size.
– So p(weak) = .12 goes to p(weak) = .04
• Frequent looks require more reports from
SCHARP.
• Monitoring may reveal interesting trends.
– Differential effect by subgroup
– Waning immunity, VE
– Potential harm
• May be tempting to tinker with trial design.
– Trends are unreliable early on.
– Feel more comfortable sticking with the design.
MUSTT
• Multi Center Unsustained Tachycardia Trial

EP testing + antiarrhytmics

Patients with
arrhythmias

No Antiarrhymics
The Overshoot: PAVE100
• After the study is completed, infections will
continue to accrue.
• PAVE100:
– 200 infections in 3 years
– 100 “wasted” infections in 4th year
• Suppose VE=50% p=.001
• FDA likes two p<.05 for licensure
Another Study?
Main Trial                Another
Follow-up                 Study

Maintain

Randomize
To Vaccine

Maintain

Vaccine shots + FU                  Placebo shots + FU
Another Study
• Maintain arms & extend trial for 2 years, get
200 more infections.
• Is trial extension an “independent” study?
– Future 200 infections independent of past
– Future 200 infections estimate late VE
– Future 200 from same pool of original
uninfecteds.
• Some studies divide sites to form ``two” trials
• Suppose VE = 50% p=.001.
• Immunity & VE are waning. What to do?
Boosterism
Main Trial                Extension 1           Extension 2
Follow-up                 Follow-up             Follow-up

Randomize
Vaccinate              To boost
Everyone
Randomize
To Vaccine
Randomize
To boost

Vaccine shots + FU                      Placebo shots + FU
Boosterism
• Use all uninfecteds to address question of
boosting.
• Years faster than a new trial
• Like two strata within a ``boosting” trial
– Those who get 1st shots in 2010 boosted in 2012
– Those who get 1st shots in 2012 boosted in 2014
Bayesian Approaches
to blend with data to get more efficient
analysis.
• Non-Bayesian---don’t trust your beliefs, only
use the data.
Bayesian Example
• Want to estimate the vaccine & placebo infection
rates.
• Believe the vaccine infection rate is about 1%
placebo infection rate is about 2%. Belief is
equivalent to 1000 patients.
• Data
– 40/1000 vaccine rate   4%         VE=12%
– 45/1000 placebo rate 4.5%
• Bayes rates
– (40+10)/2000 vaccine rate 2.50%   VE=23%
– (45+20)/2000 placebo rate 3.25%
View
• Bayesian approaches are a means to an end.
Can be more efficient, smoother, etc if well
calibrated.
• Not really suited for Phase III licensure quality
inference.
• OK in phase II, appealing to have the prior
belief equivalent to 10% or less of planned
sample size
• Is HIV vaccine field well-calibrated?
The long view
• Adaptation strategy best evaluated using a
long view.

1984                  Now                      2036
• Most statistical metrics describe a single trial
– Power = probability identifying one good vaccine
– Duration = time to knowledge about one vaccine
• Better metric. How many years to identify a
successful vaccine under different statistical
designs.
• Assume always at least two vaccine candidates to
test. Most vaccines useless.
good     useless                         Current Vaccine
Candidates

• Strategy 1 Sequential
– Undertake 2 arm vaccine trials V, P
• Strategy 2 Concurrent
– Undertake 3 arm vaccine trials V1, V2, P
• Use frequent monitoring for each.
2 vaccines at a time
1               3                       5   7
2                   4       6

Now                                              2036

1     2     3       4   5       6   7

1 vaccine at a time
Results Pending
• Concurrent evaluation: 3 arms V1 V2 P
– Optimal to allocate P:V    1.41 : 1
– Typically use alpha/2 to control type I error rate
• Sequential evaluation: 4 arms: for V1 V2 P1 P2
• Evaluation can be applied to other issues.
– Should 2 arm-trials have e.g. 176 infections or 131
total infections.
Conclusions
• Frequent monitoring makes perfect sense.
• Wary about aptation based on interim
over-reaction to unreliable trends.
• Bayesian approaches may be useful in phase II
studies. Not phase III.
Conclusions
• Need to be able to act quickly on trial results
– Extend FU for “overshoot” infections.
Like a separate trial addressing distal VE
– Vaccinate placebos
– Something we can’t imagine.
• Taking the long view & being pessimistic
prudent way to choose the “default” design.
The Overshoot: WWII
Boosterism Benefit
Immune Response Quartiles
Group Outcome           Weak   Modest   Good    Best   Total
Vaccine Uninfected       70     85      90      95     340
Infected         30     15       10      5      60
Total           100    100      100     100    400
Placebo Uninfected       70    75       75       80    300
Infected        ~30    ~25      ~25     ~20    100
Total           ~100   ~100     ~100    ~100   400

Observed   Inferred

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 0 posted: 5/24/2013 language: English pages: 27