# ADAPTIVE BAYESIAN DESIGNS FOR DOSE-RANGING DRUG TRIALS by wpr1947

VIEWS: 7 PAGES: 182

• pg 1
```									Bayesian Clinical Trials
Scott M. Berry
scott@berryconsultants.com

BERRY
CONSULTANTS
STATISTICAL INNOVATION

1
Bayesian Statistics
• Reverend Thomas Bayes
(1702-1761)
• Essay towards solving a
problem in the doctrine of
chances (1764)

This paper, on inverse probability, led to Bayes
theorem, which led to Bayesian Statistics
2
Bayes Theorem
theorem:
'(q| X)  (q)*f (X | q)
• Assess prior ; subjective, include
available evidence
• Construct model f for data
• Find posterior '

3
Simple Example
•    = 0.25 or =0.75, equally likely.
•   DATA: Flip coin twice, both heads.
•   ???

4
Bayes Theorem
Pr[  = 0.75 | DATA] =

Pr[DATA | p=0.75] Pr[=0.75]
-----------------------------------------------------------------------------------
Pr[DATA | =0.75] Pr[=0.75] + Pr[DATA | =0.25] Pr[=0.25]
(0.75)2 (0.5)
----------------------------------- = 0.90
(0.75)2 (0.5) + (0.25)2 (0.5)
Posterior Probabilities
Likelihood           Prior Probabilities

5
Rare Disease Example
Suppose 1 in 1000 people have a rare disease,
X, for which there is a diagnostic test which is
99% effective. A random subject takes the test,
which says “POSITIVE.” What is the
probability they have X?

(0.99) (0.001)
---------------------------------------   = 0.0902 !!!
(0.99) (0.001) + (0.01) (0.999)

6
Bayesian Statistics
• A subjective probability axiomatic
approach was developed with Bayes
theorem as the ―mathematical crank‖--
Savage, Lindley (1950’s)
• Very different than classical statistics: a
collection of tools
• Before 1980-1990?: A philosophical
niche, calculation very hard.
• Early 1990’s: Computers and methods
7
Bayesian Approach
• Probabilities of unknowns:
hypotheses, parameters, future data
• Hypothesis test: Probability of no
treatment effect given data
• Interval estimation: Probability that
parameter is in the interval
• Synthesis of evidence
• Tailored to decision making: Evaluate
decisions (or designs), weigh outcomes
by predictive probabilities
8
Frequentist vs. Bayesian—
Seven comparisons
1. Evidence used?
2. Probability, of what?
3. Condition on results?
4. Dependence on design?
5. Flexibility?
6. Predictive probability?
7. Decision making?

9
Consequence of Bayes rule:
The Likelihood Principle
The likelihood function
LX(q) = f( X | q)
contains all the information in an experiment

LX q  q 
 LX q  q dq
10
• Short version of LP: Take data at
face value
• But ―data‖ can be deceptive
• Caveats . . .
– How identified?
– Why are they showing me this?

11
Example
• Data: 13 A's and 4 B's
• Parameter =  = P(A wins)
• Likelihood   13 (1–)4
• Frequentist conclusion?
Depends on design

12
Frequentist hypothesis testing
• P-value = Probability of observing data as or
more extreme than results, assuming H0. P-V =
P(tail of dist. | H0)
• Four designs:
(1) Observe 17 results
(2) Stop trial once both 4 A's and 4 B's
(3) Interim analysis at 17, stop if 0 - 4 or
13 - 17 A's, else continue to n = 44
(4) Stop when "enough information"

13
Design (1): 17 results
Binomial distribution
with n = 17,  = 0.5;
P-value = 0.049

14
Design (2): Stop when
both 4 A’s and 4 B’s

Two-sided negative
binomial with r = 4,  = 0.5;
P-value = 0.021

15
Design (3): Interim analysis
at n=17, possible total is 44

Analyses at
n = 17 & 44;
stop @ 17
if 0-4 or 13-17;
P = 0.085             Both shaded regions = 0.049

P(both) = 0.013;
net = 2(0.049) – 0.013
= 0.085 16
Design (4): Scientist’s
stopping rule: Stop when

• Cannot calculate P-value
• Strictly speaking, frequentist
inferences are impossible

17
Bayesian Calculations
• Data: 13 A's and 4 B's
• Parameter =  = P(A wins)
• For ANY design with these results, the
likelihood function is
P(data | p)   13 (1–)4
• Posterior probabilities & Bayesian
conclusion same for any design

18
Likelihood function of 

13   4
p (1–p)

0   .1   .2   .3   .4   .5   .6   .7   .8       .9   1
p

19
Posterior Distribution
Prior: 1 0 <  < 1
Posterior  1 *  13 (1–)4

= 1 *  13 (1–)4 / ∫ 1 *  13 (1–)4 d 

= {13!4!/18!}  13 (1–)4

20
Posterior density of 
for uniform prior: Beta(14,5)

13   4
p (1–p)

0   .1   .2   .3   .4   .5   .6   .7   .8       .9   1
p
21
Pr[ > 0.5 ]

P(p > 0.5 | data)
= 0.985

0   .1   .2   .3   .4   .5   .6   .7   .8       .9   1
p            22
PREDICTIVE PROBABILITIES
• Distribution of future data?

• P(next is an A) = ?

• Critical component of experimental design

• In monitoring trials

23
Laplace’s rule of succession

P(A wins next pair | data)
= EP(A wins next pair | data, )
= E( | data)
= mean of Beta(14, 5)
= 14/19                  Laplace uses
Beta(1,1) prior
24
Updating w/next observation
Beta(15, 5)

Beta(14, 6)

prob 5/19                                     prob 14/19

0   .1     .2   .3   .4   .5   .6   .7   .8       .9    1
p

25
Suppose 17 more
observations
P(A wins x of 17 | data)
= EP(A wins x | data, )

 17 x                       
= E  x   1 -  
17- x
|data,  
                            

Beta-Binomial Distribution          26
Predictive distribution
Predictive      .15

distribution of
# of successes .10
in next
88% probability
17 tries:                                                      of statistical
.05                                           significance

.00
0   1   2   3   4   5   6   7   8   9 10 11 12 13 14 15 16 17

Has more variability than any binomial 
27
Best fitting binomial vs.
predictive probabilities
Binomial, p=14/19

96% probability
of statistical
significance
0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17

Predictive, p ~ beta(14,5)
88% probability
of statistical
significance

0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17
28
Possible Calculation
 17 x        17- x  14  5  13
  x  1 -                      1 -   d
4

                       19 

• Simulate a  from the beta(14,5)

• Simulate an x from binomial(17, )

• Distribution of x’s is beta-binomial--the
predictive distribution
29
Posterior and
Predictive…same?
• Clinical Trial, 100 subjects. HA:  > 0.25?
FDA will approve if # success ≥ 33 [post >
0.95, beta(1,1)]
• See 99 subjects, 32 successes

• Pr[  > 0.25 | data ] = 0.955
• Predictive prob trial success = 0.327

30
Predictive Probabilities
for Medical Device
• Bayesian calculations  FDA:
– Some patients have reached 2 years
– Some patients have only 1-yr follow-
up

31
Continuous data; Patients w/both
12 and 24 months

2
4
-
m
o
n
t
n

12 - m o n t h O s w e s t r y   32
Some patients with
only 12-month data

2
4
-
m
o
n
t
n

33
Missing 24 months
Kernel density estimates

2
4

m
o
n
t                                         
h

12 - m o n t h O s w e s t r y
34
Small bandwidth (0.2)
a

35
Larger bandwidth (0.3)
b

36
Still larger bandwidth (0.4)
c

37
Very large bandwidth (0.5)
(nearly bivariate normal)
d

38
Condition on 12-month value
a

39
Conditional distribution
of 24-month value (0.2)
a

24-month Oswestry     40
For largest bandwidth (0.5)
d

24-month Oswestry      41
Multiple imputation: simulate full
set of 24-month data

2
4

m
o
n
t
h

12 - m o n t h O s w e s t r y   42
• Simulate experimental patients
and controls in this way—
multiple imputation
• Make inferences with full data (for
example, equivalent improvement)
• Repeat simulations (≥10,000 times)
• Gives probability of future results– for
example, of ―equivalence‖

43
Monitoring example:
Baxter’s DCLHb
• Blood substitute; emergency trauma
• Randomized controlled trial (1996+)
– Treatment: DCLHb
– Control: saline
– N = 850 (= 425x2)
– Endpoint: death

44
• Waiver of informed consent
• Data Monitoring Committee
• First DMC meeting:
DCLHb        Saline
Alive         28           33
Total         49           41
• No formal interim analysis

45
Bayesian predictive
probability of future results
(no stopping)
• Probability of significant survival benefit
for DCLHb after 850 patients: 0.00045
(PP=0.0097)
• DMC paused trial: Covariates?
• DMC stopped the trial

46
•   Endpoint: tumor response
•   Balanced randomized, A & B
•   Sample size planned: 164
•   Interim results after n = 34:
– Control: 4/16 = 25% (pCR)
– Herceptin: 12/18 = 67% (pCR)
•   Not unexpected (prior?)
•   Predictive prob of stat sig: 95%
•   DMC stopped the trial
•   ASCO and JCO—reactions …
47
Mixtures:
• Data: 13 A's and 4 B's

• Likelihood  p13 (1–p)4

48
Mixture Prior
 ~ 0 I[p=p0] + (1-0) Beta(a,b)
  0 I0 p013(1-p0)4 + (1-0) Kpa+13-1(1-p)b+4-1

 ~ 0 I0 + (1-0) Beta(a+13,b+4)
0 p13(1-p)4
0 = ---------------------------------------------------
(a)(b)(a+b+17)
0 p 13(1-p)4 + (1- ) --------------------------
0
(a+13)(b+4)(a+b)
49
Mixture Posterior 0=.5

Pr(p=0.5) = 0.246

P(p > 0.5) = 0.742

50
Crooked-Penny Example
• Flip the coin 20 times.
• What is q for your coin?
• Everyone reports p for their coin.
^

A new estimate for q?
Are others relevant for you?

51

This is you

52
One-Sample Problem
[q]~ Beta(a,b
• [X] ~ Binomial(n,q)
• [q|X]~Beta(a+X,b+n-X)
• Mean = (a + X/a+b+n)

53
For uniform prior (a= b= 1)

Posterior:
q~ Beta(17, 5

Prior:
q~ Beta(1, 1

0   .1   .2   .3   .4   .5   .6   .7   .8   .9   1
q         0.77
54
For a= b= 10
Posterior:
q~ Beta(26, 14

Prior: q~
Beta(10, 10

0    .1   .2   .3   .4   .5   .6    .7    .8   .9   1
q         0.65
55
Remember the other coins . . .

This is you

56
• In your setting the other coins give you
helps!!!!
• The coins do not have to be the same
or close, you learn the appropriate
amount of borrowing.

57
HIERARCHICAL MODELING

Population:

Sample:
Inferential
problems
Sample from sample:
58
Selecting coins
Population of coins—population of q’s:

Select two coins and toss each coin
•Estimate q1, q2.
•Estimate distribution of q’s in population.
59
Generic example: Unit is lab or
drug variation or lot or study
Unit     s      n     s/n
1      20     20     1.00   n = #observations
2        4    10     0.40
3      11     16     0.69
4      10     19     0.53   s = #successes
5        5    14     0.36
6      36     46     0.78
7        9    10     0.90   s/n = success
8        7      9    0.78
9        4      6    0.67         proportion
Total   106    150     0.71

60
If q1 = q2 = . . . = q9 = q
(all 150 units exchangeable)
106            44
p         (1-p)

0   .1   .2    .3    .4     .5    .6   .7   .8   .9       1
p       61
Assuming equal q’s,
95% CI for q: (0.63, 0.77)

• But 7 of 9 estimates lie outside this
interval.
• Combined analysis unsatisfactory.

• Nine different analyses even worse:
nine individual CIs?

62
Suppose ni independent
observations on unit i
• Suppose each unit has its
own q, with q1, . . . , q9
having distribution G.
• Observe x's, not q's.
• Xi ~ binomial(ni, qi).
• Likelihood is product of
likelihoods of qi

63
Bayesian view: G unknown =
G has probability distribution
• Prior distribution reflects heterogeneity vs
homogeneity.
• Assume G is Beta(a,b),
a > 0, b > 0 with a and b unknown.
• Study heterogeneity:
– little if a+b is large
– lots if a+b is small

64
Beta(a,b) for a, b = 1, 2, 3, 4:

65
Suppose uniform prior for
a & b on integers 1, . . ., 10

66
Posterior probabilities
for a & b
a
= 0.68
a+b

67
Calculating posterior
distribution of G
• Direct in this example
• Can be more complicated, and
require:
– Gibbs sampling (BUGS)
– Other Markov chain Monte Carlo

68
Posterior mean of G
(also predictive density for q)

0   .1   .2   .3   .4   .5   .6   .7   .8   .9       1
p       69
Contrast with likelihood
assuming all p’s equal
All p's equal

Posterior
mean of G

0   .1   .2   .3   .4   .5   .6   .7   .8   .9       1
p
70
Bayesian questions:
• P(q > 1/2) = ????
• P(next unit in study i is success) = ?
– How to weigh results in unit i?
– How to weigh results in unit j?
• P(unit in 10th study is success) = ?
– How to weigh results in study i?

71
Bayes estimates
Unit    x      n   x/n     Bayes
1    20       20     1.00 0.90
2     4       10     0.40 0.53
3    11       16     0.69 0.69
4    10       19     0.53 0.57
5     5       14     0.36 0.48
6    36       46     0.78 0.77
7     9       10     0.90 0.80
8     7        9     0.78 0.73
9      4        6     0.67 0.68
Total 106     150      0.68 0.68
(0.71)

72
Bayes estimates are regressed
or shrunk toward overall mean
Bayes estimates

0   .1   .2   .3   .4    .5   .6   .7   .8   .9       1
p

73
Baseball Example
• 446 players in 2000 with > 100 at bats

Jose Vidro

74
How good was Jose Vidro?
(200 hits in 606 at bats, 0.330)

X ~ Binomial(606, qJV
(hits)

qJV ~Beta(a,b

75
• Empirical Bayes:
aEB =95.5bEB=258.9
mean = 0.269; var = 0.0362-.0272)

[q|X] ~ Beta(200+95.5, 406+258.9)
(approx)

Posterior mean = 0.308
Posterior st. dev. = 0.015

76
Science, Feb 6, 2004, pp 784-6

77
www.fda.gov/ ohrms/dockets/ac/02/slides/
3829s2_03_Bristol-Meyers-meta-analysis.ppt

Efficacy of Pravastatin + Aspirin:
Meta-Analyses

[For statistical analysis, S.M. Berry et al.,
Journal of the American Statistical Association, 2004]
78
Meta-Analysis of these
Pravastatin Secondary Prevention Trials
Number of
Trial          Subjects*           % on Aspirin               Primary Endpoint

LIPID              9014                  82.7                    CHD mortality

CARE               4159                  83.7              CHD death & non-fatal MI

Atherosclerotic progression
REGRESS              885                  54.4
(& events)
Atherosclerotic progression
PLAC I              408                  67.5
(& events)
Atherosclerotic progression
PLAC II              151                  42.7
(& events)

Totals            14,617                 80.4

*99.7% of pravastatin-treated subjects received 40mg dose
79
Trial Commonalities
– Similar entry criteria
– Patient populations with clinically evident CHD
– Same dose of pravastatin (40mg)
– Randomized comparison against placebo
– All trials with durations of  2 years
– Pre-specified endpoints
– Covariates recorded
– Common meta-analysis data management

80
Patient Group Comparisons

Randomized Groups

Pravastatin       Placebo

Randomized
Aspirin Users   Prava+ASA       Placebo+ASA     Comparison
Aspirin
Prava alone     Placebo alone
Non-Users
Observational
Comparison

81
Is Pravastatin+Aspirin More Effective
than Pravastatin Alone?

– Aspirin studies were conducted before statins were
widely used
– Placebo-controlled trial with aspirin
is not feasible
– Investigation of pravastatin database
to explore this question

82
Is the Combination More Effective
than Pravastatin Alone?

– Unadjusted event rates in LIPID and
CARE suggest pravastatin + aspirin is
more effective than pravastatin alone

83
Event Rates for Primary Endpoints in LIPID
and CARE
Pravastatin-treated Subjects Only

Trial:        LIPID              CARE
Primary Endpoint:     CHD Death          CHD Death or
Non-fatal MI

Aspirin Users              5.8%                  9.3%

Aspirin Non-Users          8.8%               14.8%

Observational      Observational
Comparison         Comparison

84
Accounting for Baseline Risk Factors

– Age
– Gender
– Previous MI
– Smoking status
– Baseline LDL-C, HDL-C, TG
– Baseline DBP & SBP

diabetes and obesity

85
Meta-Analysis Endpoints Considered

– Fatal or non-fatal MI
– Ischemic stroke
– Composite: CHD death, non-fatal MI,
CABG, PTCA or ischemic stroke

86
Meta-Analysis Models
•Model 1:
– Multivariate Cox proportional hazards model
– Patients combined across trials; trial effect is a fixed
covariate

H(t) = l0(t)exp(Zb + fS + gT)

Baseline Hazards        Covariates    Study
Treatment Effects
constant                              effects
87
Relative Risk Reduction
Cox Proportional Hazards – All Trials
Fatal or Non-Fatal MI             Relative Risk (95% CI)                     RRR
0.69
Prava+ASA vs ASA alone                                                   31%
0.74
Prava+ASA vs Prava alone                                                  26%
0.400      0.600            0.800          1.000

Ischemic Stroke
0.71
Prava+ASA vs ASA alone                                                   29%
0.69
Prava+ASA vs Prava alone                                                  31%

0.400      0.600            0.800          1.000

CHD Death, Non-Fatal MI, CABG, PTCA, or Ischemic Stroke
0.76
Prava+ASA vs ASA alone                                                   24%
0.87
Prava+ASA vs Prava alone                                                  13%
0.400      0.600            0.800          1.000
88
RRR = Relative Risk Reduction
Meta-Analysis Models
•Model 2: Same as Model 1 except
– Allows trial heterogeneity:
Bayesian hierarchical (random effects)
model of trial effect

H(t) = l0(t)exp(Zb + fS + gT)

Baseline Hazards     Covariates      Study
Treatment Effects
piecewise-constant                   effects                 89
Hierarchical
Model 2 – Hierarchical, Random Effects

Fatal or Non-Fatal MI
0.100
Placebo
ASA alone
Prava alone
0.075
Cumulative
Proportion                                            Prava+ASA
of Events 0.050

0.025

0.000
0      1    2          3   4   5
Year
90
Model 2 – Hierarchical, Random Effects

Ischemic Stroke Only
0.025
Placebo
Prava alone
ASA alone
0.020

0.015                                      Prava+ASA
Cumulative
Proportion
of Events 0.010

0.005

0.000
0     1     2          3   4   5
Year
91
Model 2 – Hierarchical, Random Effects
CHD Death, Non-Fatal MI, CABG,
PTCA, or Ischemic Stroke
0.25
Placebo
ASA alone
0.20                                   Prava alone

Prava+ASA
0.15
Cumulative
Proportion
of Events 0.10

0.05

0.00
0   1    2          3   4   5
Year
92
Combination is More Effective
than Either Agent Alone

– Pravastatin + aspirin provides benefit for all three
endpoints:
 24% - 34% RRR compared with aspirin
 13% - 31% RRR compared with pravastatin

This benefit was similar in Models 1 and 2

This benefit was consistent in both LIPID and CARE
trials

93
Model 2:
Fatal or Non-Fatal MI
Cumulative Proportion of Events                                 Hazard
0.100                                                       0.025
Placebo
ASA alone
Placebo
Prava alone   0.020
0.075                                                                                          ASA alone
Prava alone

Prava+ASA     0.015
0.050                                                                                          Prava+ASA

0.010

0.025
0.005

0.000                                                       0.000
0        1      2          3     4    5                     0   1   2          3   4        5
Year                                                Year
94
Meta-Analysis Models
•Model 3: Same as Model 2 except
– Treatment hazard ratios vary over time

H(t) =           l0(t)exp(Zb
T                + fS)

Baseline Hazards          Covariates      Study
piecewise-constant
Effects
Within treatment
Hierarchical

95
Model 3:
Fatal or Non-Fatal MI
Cumulative Proportion of Events                                 Hazard
0.100                                                       0.030
Placebo

ASA alone
0.025
Prava alone
0.075
Placebo
0.020
ASA alone
Prava+ASA
0.050                                                       0.015                              Prava alone
Prava+ASA
0.010
0.025
0.005

0.000                                                       0.000
0        1      2          3   4      5                     0   1   2          3   4      5
Year                                                Year
Year
5 Separate Analyses: One per 96
Probability of synergy
between pravastatin & aspirin

Endpoint         Model 2   Model 3
All events        0.983     0.985
Cardiac events    0.945     0.947
Any MI            0.911     0.923
Stroke            0.924     0.906
Death             0.997     0.997

97
Conclusion of Hazard Analysis over Time

– Benefit of pravastatin+aspirin over aspirin
was present in each year of the 5-year duration of the trials

Benefit of pravastatin+aspirin over pravastatin was present
in each year of the 5-year duration of the trials
Benefits estimated from Model 1 (and confidence
intervals) confirmed by more general models and fewer
assumptions

98
Hierarchical modeling
in design
• Using historical information
• Combining results from multiple
concurrent trials (or many
centers)

99
Hierarchical modeling
& dose-response
• Example: drug Z (rozuvastatin) vs drug
A (atorvastatin)
(Berry et al., 2002, American Heart
Journal)

100
Studies involving drugs A and Z*, with %change from baseline.
%Change
Study      n       Dose     Mean     SD      Y
1.        46      10       –27      10     0.73
45      20       –34      10     0.66
2.        45      10       –35.3     8     0.647
3.        14      Placebo –1.4      18     0.986
13       5       –16.7    17     0.833
16      20       –33.2    18     0.668
12      80       –41.4    18     0.586
4.       222      10       –35      14     0.65
5.       210      20       –45.0    10     0.55
215      40       –51.1    12     0.489
6.       132      10       –37      13     0.63
7.       133      Placebo    1      12     1.01
707      10       –36      13     0.64
8.        17      Placebo    0       8     1.00
18      10       –35       8     0.65
9.        41      10       –35      13     0.65
10.        73      10       –38      10     0.62
51      20       –46       8     0.54
61      40       –51      10     0.49
10      80       –54       9     0.46                101
Study    n   Dose      Mean    SD      Y
11.     54   10        –30     18     0.70
12.   1897   10        –37.6   NA     0.624
13.     12   Placebo     7.6    9     1.076
11    2.5      –25.0    9     0.75
13    5        –29.0    9     0.71
11   10        –41.0    9     0.59
10   20        –44.3    9     0.557
11   40        –49.7    9     0.503
11   80        –61.0    9     0.39
14.     40   10        –29     12     0.71
15.    164   80        –46     NA     0.54
16.     12   Placebo     5.1    8.1   1.051
15   10        –43.9    7.8   0.561
13   80        –56.9    8.3   0.431
14    1*       –35.9    7.7   0.641
15    2.5*     –40.6    9.9   0.594
16    5*       –44.1    8.3   0.559
17   10*       –51.7    8.7   0.483
17   20*       –55.5   12.8   0.445
18   40*       –63.2    8.7   0.368
17.     17   Placebo     0.8   10.6   1.008
15   40*       –61.9    7.2   0.381   102
31   80*       –62.9    7.8   0.371
Dose-response model
• Yij = exp{as + at + bt log(d)} + eij
s for study
t for drug
d for dose
i for observation (1, . . . , 43)
j for patient within study/dose
eij is N(0, 2)
• Priors don’t matter much, except . . .
103
Prior for as ~ N(0, t2)
t2 is important
t2 large means studies heterogeneous—
little borrowing
t2 small means studies homogeneous—
much borrowing
• Prior of t2 is IG(10, 10)
• Prior mean and sd are 0.10 & 0.017

104
Likelihood

 ni                                  2 
 exp- 2  Y - expa s + at + bt log(d) 
43
-n

 2 i=1                                
i

Calculations of posterior &
predictive distributions by
MCMC
105
Posterior means and SDs
Parameter    Mean     StDev
aP     –0.0016   0.027
aA     –0.073    0.055
aZ     –0.34     0.059
bA     –0.149    0.021
bZ     –0.146    0.019
       0.152    0.024
t       0.087    0.011

106
Posterior means and SDs
Par. Mean StDev Par.    Mean         StDev
a1      0.102   0.023   a10    –0.072      0.022
a2     –0.017   0.032   a11     0.052      0.027
a3      0.062   0.025   a12    –0.054      0.013
a4     –0.014   0.018   a13    –0.028      0.024
a5     –0.072   0.035   a14     0.063      0.031
a6     –0.043   0.022   a15     0.104      0.042
a7      0.015   0.013   a16    –0.070      0.031
a8      0.002   0.029   a17    –0.017      0.033
a9     –0.013   0.033

107
Model fit

108
Interval
estimates
for pop.
mean:
model (line)
vs standard
(box)

109
Study/dose-
specific
interval
estimates:
model (line) vs
standard (box)

110
Posterior
dist’n of
reduction
(95%                Drug A
intervals)

Drug Z

111
Posterior
dist’n of mean
diff,
A–Z

112
Really neat . . .

• Using predictive probabilities for
designing future studies

• Contour plots

113
90
%        Mean Z10 better than A10: 12.1%
Y        P(Z10 betters A10) = 99.4%
80
f
o 70
r

A 60
Z

Observed %Y           50
for future study
40
with nA=nZ=20                     Z10 = A10

dA=dZ=10                  40       50     60      70      80    90
A
%Y f o r Z

Height       20              40            60
80              100           120
140             160           180
200             220           240
260             280           300    114
320             340
90
%        Mean Z10 better than A10: 12.1%
Y        P(Z10 betters A10) = 99.99%
80
f
o 70
r

A 60
Z

Observed %Y            50
for future study
40
with nA=nZ=100                  Z10 = A10

dA=dZ=10                     40      50      60      70      80    90
A
%Y f o r Z

Height       30             60             90
120            150            180
210            240            270
300            330            360
390            420            450    115
480            510            540
90
%        Mean Z5 better than A10: 7.0%
Y 80     P(Z5 betters A10) = 92.5%

f
o   70
r
Z 60
Observed %Y
A

for future study       50

with nA=nZ=20          40
Z5 = A10
dA=10,
40      50     60      70      80    90
dZ=5                                              A
%Y f o r Z

Height      20             40            60
80             100           120
140            160           180
200            220           240
260            280           300    116
320            340
90
%        Mean Z5 better than A10: 7.0%
Y        P(Z5 betters A10) = 98.6%
80
f
o 70
r

A 60
Z
Observed %Y
for future study       50

with nA=nZ=100         40
Z5 = A10
dA=10,
40      50      60     70     80    90
dZ=5                                               A
%Y f o r Z

Height       30             60           90
120            150          180
210            240          270
300            330          360
390            420          450    117
480            510          540
STELLAR trial results (each n≈160)
Dose (log scale)
10mg        20mg        40mg            80mg
0
Mean percentage change in
LDL -C from baseline
-10

-20                   -20%
-24%
-28%
-30                                               -30%
Predicted                                               -35%
-37%
atorva -36%
-40
-39%
-43%
Predicted -46%                                        -41%            -48%          -46%
-50
rosuva -50%
*                                             -52%         -46%           -51%

*
-54%
-55%        -52%
-60                                              *-58%

Rosuvastatin n=473           Atorvastatin n=634
Simvastatin n=648            Pravastatin n=485

*p<0.002 rosuvastatin 10mg vs. atorvastatin 10mg, simvastatin 10, 20 & 40mg,
pravastatin 10, 20 & 40mg; rosuvastatin 20mg vs. atorvastatin 20mg and 40mg,
simvastatin 20, 40 & 80mg, pravastatin 20 & 40mg; rosuvastatin 40mg vs.                                        118
atorvastatin 40mg, simvastatin 40 & 80mg, pravastatin 40mg
Recall:

Posterior
dist’n of
reduction
(95%                Drug A
intervals)

Drug Z

119
the ―Best‖ Dose
Scott M. Berry
scott@berryconsultants.com

BERRY
CONSULTANTS
STATISTICAL INNOVATION

120
Standard Parallel Group Design
Equal sample sizes at each of k doses.

Doses
121
True dose-response curve
(unknown)
Response

Doses
122
Observe responses (with error) at
chosen doses
Response

Doses
123
Dose at which 95% max effect
Response

True ED95

Doses
124
Response

?
Doses
125
Solution:
Increase number of doses
Response

True ED95

Doses
126
But, enormous sample size, and . . .
wasted dose assignments—always!
Response

True ED95

Doses
127
Solutions
• Lots of doses (continuum?)
• Model dose response
• Define what you are looking for
• Stop when you find what you are
looking for…
• Yogi Berra-ism: If you don’t know where
you are going, how do you know when
you get there?
128
Dose Finding Trial
• Real example (all details hidden, but flavor is
the same)
• ―Delayed‖ Dichotomous Response (random
waiting time)
• Combine multiple efficacy + safety in the
dose finding decision
• Use utility approach for combining various
goals
• Multiple statistical goals
129
Decisions:
Continue
Study, Stop
Subject Enters               Phase III

Utility

Statistical
Allocator                        Model

Wait Event
Results
Measured

130
Statistical Model
• The statistical model captures all the
uncertainty in the process.
• Capture data, quantities of interest, and
forecast future data
• Be ―flexible,‖ (non-monotone?) but capture
prior information on model behavior.
• Invisible in the process

131
Empirical Data
Observe Yij for subject i, outcome j

Yij = 1 if event, 0 otherwise
j = 1 is type #1 efficacy response
j = 2 is type #2 efficacy response
j = 3 is minor safety event
j = 4 is major safety event

132
Efficacy Endpoints
• Let d be the dose
• Pj(d) probability of event j, dose d.
 Pj d                        d 
q j d  = log              = a j d  + b j  d + g 
 1 - Pj d                   
      j 


aj(d) ~ N(j, 2)                      G(1,1)

N(–2,1)                        N(1,1)
IG(2,2)
133
Safety Endpoint
• Let di be the dose for subject i
• Pj(d) probability of safety j, dose d.
 Pj d                    d 
q j d  = log              = a j + bj           
 1 - Pj d               d + g j 
        

N(-2,1)                       G(1,1)
N(1,1)

134
Utility Function

• Multiple Factors:
• Monetary Profile (value on market)
• FDA Success
• Safety Factors
• Utility is critical: Defines ED?

135
Utility Function

U(d)=U1(P1)U2(P3)*U3(P0,P2)*U4(P4)

Monetary

FDA Approval
Extra Safety

P0 is prob efficacy 2 success for d=0
136
Monetary Utility

P1 - 0.1
U1 P1  =            for P1  0.1
0.4

U2 P3  = 1- 2 P3  for P3  0.5
1.5

137
138
139
U1 P1 U 2 P3 

P1

P3
140
U3: FDA Success

U 3 P2 , P0  = Pr P2  P0 | 250/arm trial

U 4 P4  = 0 if any Major safety
―DSMB?‖

141
Statistical + Utility Output
•   E[U(d)]
•   E[qj(d)], V[qj(d)]
•   E[Pj(d)], V[Pj(d)]
•   Pr[dj max U]
•   Pr[P2(d) > P0]
•   Pr[ P2 >> P0 | 250/per arm) each d

>> means statistical significance will be achieved
142
Allocator
• Goals of Phase II study?
• Find best dose?
• Learn the minimum effective dose?
• Allocator and decisions need to reflect this (if
not through the utility function)
• Calculation can be an important issue!

143
Allocator
• Find best dose?        
V P d -P d
1    1
*           **

• Learn about best dose? V  d - P 
*
P    2            0

d* is the max utility dose, d** second best

    + w V P d - P 
V* = w1V P d - P d
1
*
1
**
2       2
*
0

Find the V* for each dose ==> allocation probs
144
Allocator
V*(d≠0) =
V P d 
1

nd + 1 
               
 w1 Pr d = d * + w2 Pr d = d **  +

V P2 d 
n +1 

 w2 Pr d = d * 
 
d

V P0 
V*(d=0) =        n0 + 1
w2

145
Allocator
V d 
*
rd =    k            for all d
 V * d 
d =0

• ―Drop‖ any rd<0.05

• Renormalize

146
Decisions
• Find best dose? If found, stop: Pr(d = d*) > C1
If found, stop: Pr(P2(d*) >> P0)>C2
• Shut down allocator wj if stop!!!!
• Stop trial when both wj = 0

• If Pr(P2(d*) >> P0) < 0.10 stop for
futility
147
More Decisions?

•   Ultimate: EU(dosing) > EU(stopping)?
•   Wait until significance?
•   Goal of this study?
•   Roll in to phase III: set up to do this
•   Utility and why? are critical and should
be done--easy to ignore and say it is too
hard.
148
Simulations

• Subject level simulation
• Simulate 2/day first 70 days, then 4/day
• Delayed observation
– exponential with mean 10 days
• Allocate + Decision every week
• First 140 subjects 20/arm

149
Scenario #1
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.25   0.10   0.05   0.06    0      0
0.5    0.13   0.08   0.07    0    0.063
1     0.17   0.12   0.08    0    0.323
2.5    0.20   0.15   0.09    0    0.457
5     0.23   0.18   0.10    0    0.532
MAX
10    0.30   0.25   0.11    0    0.656

Stopping Rules: C1 = 0.80, C2 = 0.90     150
18 20 18   15   19   17   18
1 2 2      5    5    4    5
1 0 2      0    0    4    3
0 1 0      3    3    2    2
2 0 2      5    1    3    2

151
Dose Probabilities
0   .25 .5   1   2.5   5    10
P(>>Pbo)       .18 .33 .27 .29 .67 .67
P(max)         .01 .04 .06 .04 .33 .52
P(2nd)         .03 .06 .10 .13 .35 .32
Alloc     .06 .01 .02 .04 .06 .35 .46

152
20 20 18   19   19   25   24
1 2 2      5    5    7    7
1 0 2      1    0    8    5
0 1 0      4    3    2    2
3 0 2      1    3    7    7

153
Dose Probabilities
0   .25 .5   1   2.5   5    10
P(>>Pbo)       .12 .38 .36 .38 .92 .91
P(max)         .00 .00 .02 .04 .41 .53
P(2nd)         .00 .03 .06 .07 .47 .37
Alloc     .00 .00 .02 .04 .09 .34 .51

154
21 20 19   20   21   29   31
2 2 3      5    5    7   11
1 0 2      1    0    9    6
0 1 0      4    3    2    3
2 0 1      0    4   11   17

155
Dose Probabilities
0   .25 .5   1   2.5   5    10
P(>>Pbo)       .13 .39 .38 .26 .97 .85
P(max)         .00 .02 .03 .01 .39 .55
P(2nd)         .00 .03 .10 .05 .46 .35
Alloc     .11 .00 .03 .10 .05 .46 .35

156
23 20 20   21   25   36   45
2 2 4      5    5    7   12
1 0 2      1    1   10   10
0 1 0      4    4    3    3
4 0 0      4    0   10   16

157
Dose Probabilities
0   .25 .5   1   2.5   5    10
P(>>Pbo)       .16 .41 .38 .48 .93 .93
P(max)         .00 .02 .03 .04 .26 .65
P(2nd)         .00 .05 .07 .10 .49 .29
Alloc     .00 .00 .08 .11 .18 .35 .28

158
26 20 20   25   26   44   52
2 2 4      5    6    7   13
1 0 2      1    2   13   10
0 1 0      4    4    3    4
1 0 0      6    5   12   15

159
Dose Probabilities
0   .25 .5   1   2.5   5     10
P(>>Pbo)        .16 .40 .31 .41 .98 .89
P(max)          .00 .02 .03 .06 .27 .63
P(2nd)          .00 .06 .06 .12 .48 .28
Alloc     .16 .00 .10 .04 .13 .26 .30

160
26 20 21   26   33   52   61
2 2 4      6    7    8   18
1 0 2      1    3   13   15
0 1 0      4    4    4    4
6 0 3      5    5   10   12

161
Dose Probabilities
0   .25 .5   1   2.5   5    10
P(>>Pbo)       .13 .36 .32 .65 .96 .96
P(max)         .00 .01 .01 .09 .08 .81
P(2nd)         .00 .05 .05 .23 .52 .15
Alloc

162
Trial Ends
• P(10-Dose max Util dose) = 0.907
• P(10-Dose >> Pbo 250/arm) = 0.949
• 280 subjects:
32, 20, 24, 31, 38, 62, 73 per arm

163
Operating Characteristics

Pbo    0.25   0.5      1    2.5      5     10
SS     39     21     25     37     63     89    110
Pmax    ---   0.00   0.00   0.00   0.00   0.04   0.96
SS     66     66     66     66     66     66     66
Pmax    ---   0.00   0.00   0.00   0.01   0.06   0.93
164
Operating Characteristics
P(Success)      0.936      0.810
P(Cap)        0.064      0.190
P(Futility)    0.000      0.000
Mean SS         384        459
SD SS          186        224
Mean TDose      1754       1263
Max TDose       4818       2370

165
Scenario #2
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.25   0.10   0.05   0.06    0      0
0.5    0.13   0.08   0.07    0    0.063
1     0.17   0.12   0.08    0    0.323
2.5    0.20   0.15   0.10    0    0.452
5     0.23   0.18   0.15    0    0.502
10    0.25   0.20   0.40    0    0.302

Stopping Rules: C1 = 0.80, C2 = 0.90     166
Operating Characteristics

Pbo    0.25   0.5     1     2.5     5     10
SS    71      27     41     81    137    172    164
Pmax    ---   0.00   0.00   0.03   0.22   0.60   0.16
SS    100    100    100    100    100    100    100
Pmax    ---   0.00   0.00   0.03   0.20   0.44   0.33
167
Operating Characteristics
P(Success)      0.314      0.266
P(Cap)        0.686      0.734
P(Futility)    0.000      0.000
Mean SS         694        702
SD SS          193        190
Mean TDose      2954       1937
Max TDose       4489      2455.25

168
Simulation #3
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.1    0.10   0.05   0.06    0      0
0.5    0.13   0.08   0.07    0    0.063
1     0.30   0.25   0.11    0    0.656
2.5    0.17   0.12   0.08    0    0.323
5     0.20   0.15   0.09    0    0.457
10    0.23   0.18   0.10    0    0.532

Stopping Rules: C1 = 0.80, C2 = 0.90     169
Operating Characteristics

Pbo    0.25   0.5      1    2.5      5     10
SS     53     23     28    119     52     76    102
Pmax    ---   0.00   0.00   0.92   0.00   0.01   0.07
SS     87     87     87     87     87     87     87
Pmax    ---   0.00   0.00   0.83   0.00   0.02   0.15
170
Operating Characteristics
P(Success)      0.906      0.596
P(Cap)        0.092      0.404
P(Futility)    0.002      0.000
Mean SS         453        606
SD SS          187        205
Mean TDose      1663       1662
Max TDose       3771      2384.25

171
Scenario #4
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.25   0.06   0.05   0.05    0      0
0.5    0.06   0.05   0.05    0      0
1     0.06   0.05   0.05    0      0
2.5    0.25   0.20   0.10    0    0.573
5     0.25   0.20   0.10    0    0.573
10    0.25   0.20   0.10    0    0.573

Stopping Rules: C1 = 0.80, C2 = 0.90     172
Operating Characteristics

Pbo   0.25   0.5     1     2.5     5     10
SS    53     21     22     23    150    160    163
Pmax   ---   0.00   0.00   0.00   0.27   0.32   0.40
SS    92     92     92     92     92     92     92
Pmax   ---   0.00   0.00   0.00   0.28   0.33   0.40
173
Operating Characteristics
P(Success)      0.514      0.408
P(Cap)        0.486      0.592
P(Futility)    0.000      0.000
Mean SS         591        647
SD SS          239        220
Mean TDose      2840       1780
Max TDose       4815      2448.25

174
Scenario #5
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.1    0.07   0.06   0.06    0      0
0.5    0.08   0.07   0.07    0      0
1     0.09   0.08   0.08    0      0
2.5    0.09   0.08   0.09    0      0
5     0.09   0.08   0.10    0      0
10    0.09   0.08   0.11    0      0

Stopping Rules: C1 = 0.80, C2 = 0.90    175
Operating Characteristics

Pbo   0.25   0.5     1     2.5     5      10
SS    92     91     75     66     76     83     90
Pmax   ---   0.45   0.04   0.07   0.10   0.13   0.21
SS    84     84     84     84     84     84     84
Pmax   ---   0.44   0.04   0.08   0.12   0.15   0.17
176
Operating Characteristics
P(Success)      0.004      0.006
P(Cap)        0.484      0.544
P(Futility)    0.512      0.450
Mean SS         574        589
SD SS          250        258
Mean TDose      1637       1615
Max TDose       3223.5    2523.75

177
Scenario #6
Dose   P1      P2    P3     P4    UTIL
0     0.06   0.05   0.05    0      0
0.1    0.06   0.05   0.05    0      0
0.5    0.06   0.05   0.05    0      0
1     0.06   0.05   0.05    0      0
2.5    0.06   0.05   0.05    0      0
5     0.06   0.05   0.05    0      0

Stopping Rules: C1 = 0.80, C2 = 0.90
178
Operating Characteristics

Pbo   0.25   0.5     1     2.5     5     10
SS    66     77     51     34     38     41     43
Pmax   ---   0.90   0.01   0.02   0.02   0.02   0.03
SS    56     56     56     56     56     56     56
Pmax   ---   0.86   0.01   0.02   0.03   0.03   0.05
179
Operating Characteristics
P(Success)      0.000      0.000
P(Cap)        0.122      0.190
P(Futility)    0.878      0.810
Mean SS         350        395
SD SS          215        241
Mean TDose       811       1086
Max TDose       2404      2428.75

180
Bells & Whistles
•   Interest in Quantiles
•   Minimum Effective Dose
•   ―Significance,‖ control type I error
•   Seamless phase II --> III
•   Partial Interim Information
•   ―Biomarkers‖ of endpoint
•   Continuous (& Poisson)
•   Continuum of doses (IV)--little additional
n!!!
181
Conclusions
• Approach, not answers or details!

• Shorter, smaller, stronger!

• Better for company, FDA, Science, PATIENTS

• Why study?--adaptive can help multiple needs.