Docstoc

Evaluating Adaptive Dose Finding Designs and Methods

Document Sample
Evaluating Adaptive Dose Finding Designs and Methods Powered By Docstoc
					Evaluating Adaptive Dose Finding
      Designs and Methods
                Amit Roy
          Bristol-Myers Squibb

   IIR Conference on Clinical Trial Design
               Princeton, NJ
           September 12-14, 2006
 Acknowledgments: Adaptive Dose Finding Studies WG
• Alex Dmitrienko, Eli Lilly        • Franz K¨ nig, Med. U. Vienna
                                             o
• Amit Roy, BMS                     • Greg Enas, Eli Lilly
• Beat Neuenschwander, Novartis     • Jos´ Pinheiro, Novartis a
                                         e
• Bj¨ rn Bornkamp, U. of Dortmund
    o                               • Michael Krams, Pfizer
• Brenda Gaydos, Eli Lilly          • Qing Liu, J & J
• Chyi-Hung Hsu, Pfizer              • Rick Sax, AstraZeneca a
• Frank Bretz, Novartis             • Tom Parke, Tessella
• Frank Shen, BMS
 a
     Leaders




                                                                  2
                                                       Outline

• Background and motivation

• Adaptive Dose Finding PhRMA initiative: goals

• Simulation study to evaluate designs and methods for adaptive
  dose finding

• Simulation results

• Discussion




                                                                  3
                                                   Background

• Pharmaceutical industry pipeline problem: decreasing number
  of drug approvals, despite advances in basic science
• FDA’s Critical Path Initiative: “Innovation vs. Stagnation”
  White Paper
• Pharmaceutical industry (PhRMA) response: working groups
  (WGs) to address key drivers of poor performance
• Adaptive Design (AD) WG formed to foster wider usage and
  regulatory acceptance of adaptive designs (Gallo,
  Chuang-Stein, Dragalin, Gaydos, Krams and Pinheiro, 2006)
• Adaptive Dose Finding (ADF) WG formed to stimulate
  innovation in dose finding studies

                                                                4
                                                      Motivation

• Poor understanding of dose response (DR) for both efficacy
  and safety is pervasive in drug development

• Sub-optimal dose selection identified by both FDA and
  industry as one of root causes of late phase attrition and
  post-marketing problems with approved drugs

• Current dose finding designs and methods focus on selection of
  registrational study dose out of fixed, generally small number
  of dose levels, via pairwise hypothesis testing =⇒ inefficient

• Optimize patient treatment within a study, by minimizing
  patients exposed to ineffective treatments


                                                                   5
                 What is an Adaptive Clinical Study Design?a

  • Adaptive Clinical Study Design: A design in which data from
    the ongoing study is used to modify the conduct of the study
       – Need to define objective of the adaption
       – Adaption could involve any design element, not just dose

  • Adaptive BY design: Adaption is a design feature, not a
    remedy for indadequate planning
       – Through upfront planning is required
       – Decision rules for adaption are prespecified
   a
    Adapted from: Adaptive Design in Exploratory Development, Brenda Gaydos,
Joint Statistical Meeting, 2006



                                                                               6
                                             Goals: ADF WG

• Investigate and develop designs and methods for efficiently
  learning about safety and efficacy dose-response =⇒
  benefit/risk profile
• More accurate and faster decision making on dose selection
  and improved labeling
• Evaluate statistical operational characteristics of alternative
  designs and methods to make recommendations on their use in
  practice
• Increase awareness about this class of designs, promoting their
  use, when advantageous
• Document and publish findings of the working group

                                                                    7
                       Definition and Scope: ADF Designs

• Flexible dose-ranging designs allowing dynamic allocation of
  patients and possibly variable number of dose levels based on
  accumulating information
• Intended to strike balance between need for additional DR
  information and increased costs and time-lines
• Emphasis on modeling/estimation (learning) as opposed to
  hypothesis testing (confirming)
• Investigate existing and new ADF methods via simulation
• Evaluate potential benefits over traditional dose-ranging
  designs over variety of scenarios to make recommendations on
  practical usefulness of ADF methods

                                                                  8
                        Dose Finding Methods – Fixed Doses
• ANOVA: Conventional method based on pairwise comparisons and
  multiplicity adjustment (Dunnett); common approach used in dose
  finding studies – Amit Roy and Frank Shen
• MCP-Mod combination of multiple comparison procedure (MCP) to
  identify presence of DR, and modeling to estimate target dose(s) and
                                                      e
  DR profile (Bretz, Pinheiro and Branson, 2005) – Jos´ Pinheiro and
  Frank Bretz
• MTT: novel method based on Multiple Trend Tests (Liu, 2006) –
  Qing Liu
• BMA: Bayesian Model Averaging (Hoeting, Madigan, Raftery and
  Volinsky, 1999)– Beat Neuenschwander and Amy Racine
• LOCFIT: Nonparametric local regression fitting – Bj¨ rn Bornkamp
                                                    o
  and Frank Bretz

                                                                         9
        Dose Finding Methods – Adaptive dose allocation

• GADA: Adaptive dose allocation based on Bayesian normal
  dynamic linear model (Krams, Lees and Berry, 2005);
  allocation of patients to dose adaptively changed according to
  model-based optimization criteria (e.g., variance of target dose
  estimate) – Tom Parke and Michael Krams

• D-opt: adaptive dose allocation based on D-optimality
  criterion used with sigmoid-Emax model; model parameters
  re-estimated at interim analysis and corresponding D-optimal
  allocation determined for next interval – Alex Dmitrienko and
  Chyi-Hung Hsu




                                                                     10
                     Simulation study: Design and assumptions
• Objective: proof-of-concept + dose finding for neuropathic pain
• Primary endpoint: change from baseline in pain score (VAS)
• Key questions:
      – is there evidence of a dose response
         ∗ Significance level (one-sided FWER): 0.05
         ∗ Clinically relevant change in VAS: 1.3
      – which dose(s) should be tested in large confirmatory trials
      – how well is the dose response (DR) curve estimated
• Study design scenarios:
      – Sample sizes: 150 and 250 patients
      – Number of doses: 5, 7, and 9 doses a
a
    5 doses (0, 2, 4, 6, 8), 7 doses (0, 2, 4, 6, 8), and 9 doses (0, 1, . . . , 8)

                                                                                      11
                                                                                                    Dose response profiles
                                                                                   0   2     4      6   8
Expected change from baseline in VAS at Week 6


                                                                Umbrella                   Emax                 Sigmoid Emax
                                                                                                                                       0.0


                                                                                                                                       -0.5


                                                                                                                                       -1.0


                                                                                                                                       -1.5

                                                                  Flat                     Linear                   Logistic
                                                 0.0


                                                 -0.5


                                                 -1.0


                                                 -1.5


                                                        0   2      4       6   8                            0   2      4       6   8
                                                                                           Dose

                                                                                                                                             12
                                      Measuring performance

• Probability of identifying dose response: P r(DR)

• Probability of identifying clinical relevance and selecting a
  dose for confirmatory phase: P r(dose)

• Dose selection: Distribution of selected doses (rounded to
  nearest integer, if continuous estimate possible)




                                                                  13
                            Measuring performance (contd.)

• Target dose interval – doses that produce effect within ±10%
  of target effect ∆
                     Target dose         Target interval
        Model     actual   rounded      actual      rounded
        Linear     6.30       6      (5.67, 6.93)    {6,7}
       Logistic    4.96       5      (4.65, 5.35)     {5}
      Umbrella     3.24       3      (2.76, 3.81)    {3,4}
        Emax       2.00       2      (1.44, 2.95)    {2,3}
      Sig-Emax     5.06       5      (4.68, 5.58)     {5}

• Probabilities of under-, over-, and correct interval estimation:
  P − = P (dtarg < dmin ), P + = P (dtarg > dmin ),
  P ◦ = 1 − (P − + P + )
                                                                     14
Selected Simulation Results




                              15
                                        Probability Identifying DR – Flat DR
                                             0   1    2      3       4    5   6

                      N = 250                              N = 250                            N = 250
                      5 doses                              7 doses                            9 doses

LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA

                      N = 150                              N = 150                            N = 150
                      5 doses                              7 doses                            9 doses

LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA


         0    1   2     3       4   5    6                                        0   1   2     3       4   5   6


                                                     Significance level


             Type I Error is controlled at 5% by all methods

                                                                                                                    16
                          Probability of identifying DR (N = 150)
                                     60   70      80      90   100

                   Emax                        Sig Emax
LOCFIT

  BMA
                                                                            5 doses
   MTT                                                                      7 doses
                                                                            9 doses
MCPMod

 GADA

  Dopt

ANOVA

                   logistic                    umbrella                    linear
LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA


         60   70     80       90   100                           60   70    80        90   100
                                               Pr(DR)


Pr(DR) generally ↑ as # doses ↓ (for fixed sample size)

                                                                                                 17
                                            Probability dose selection – Flat DR
                                                 0   1      2     3       4   5   6

                          N = 250                               N = 250                           N = 250
                          5 doses                               7 doses                           9 doses
     LOCFIT
       BMA
        MTT
     MCPMod
      GADA
       Dopt
     ANOVA
                          N = 150                               N = 150                           N = 150
                          5 doses                               7 doses                           9 doses
     LOCFIT
       BMA
        MTT
     MCPMod
      GADA
       Dopt
     ANOVA


              0   1   2     3       4   5    6                                        0   1   2     3       4   5   6
                                                         Pr(dose | flat DR)


False positive for clinically relevant effect is generally greater for ANOVA

                                                                                                                        18
                                   Probability dose selection (N = 150)
                                     60   70      80      90   100

                   Emax                        Sig Emax

LOCFIT

  BMA
                                                                            5 doses
   MTT                                                                      7 doses
                                                                            9 doses
MCPMod

 GADA

  Dopt

ANOVA

                   logistic                    umbrella                    linear

LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA


         60   70     80       90   100                           60   70    80        90   100
                                               Pr(dose)


Most methods perfomed poorly, GADA generally best

                                                                                                 19
  Probability dose in target interval – Logistic DR
                                  5 doses               7 doses                  9 doses
                                 0    20    40     60                                   0   20     40      60

                  N = 250               N = 250                        N = 250                   N = 250
                  No dose                Under                          Right                     Over

LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA

                  N = 150               N = 150                        N = 150                   N = 150
                  No dose                Under                          Right                     Over

LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA


         0   20     40      60                            0       20     40      60

                                                  Probability (%)


  None of the methods were entirely adequate

                                                                                                                20
 Probability dose in target interval – Umbrella DR
                                 5 doses                7 doses                  9 doses
                             0       20    40      60                                   0   20      40     60

              N = 250                  N = 250                         N = 250                   N = 250
              No dose                   Under                           Right                     Over
LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA

              N = 150                  N = 150                         N = 150                   N = 150
              No dose                   Under                           Right                     Over
LOCFIT

  BMA

   MTT

MCPMod

 GADA

  Dopt

ANOVA


         0   20   40    60                                0       20      40      60

                                                 Probability (%)



                                                                                                                21
                Distribution of selected dose – Logistic DR (N = 150)
                                  2     4   6   8                     2   4   6   8                     2     4   6   8

                    9 doses           9 doses           9 doses       9 doses             9 doses           9 doses           9 doses
                    ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
           50
           40
           30
           20
           10
            0
                    7 doses           7 doses           7 doses       7 doses             7 doses           7 doses           7 doses
                    ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
                                                                                                                                            50
% Trials




                                                                                                                                            40
                                                                                                                                            30
                                                                                                                                            20
                                                                                                                                            10
                                                                                                                                            0
                    5 doses           5 doses           5 doses       5 doses             5 doses           5 doses           5 doses
                    ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
           50
           40
           30
           20
           10
            0

                2     4   6   8                     2     4   6   8                   2     4   6   8                     2     4   6   8
                                                                  Dose selected

                Distribution of selected doses is wide for all methods


                                                                                                                                                 22
                Distribution of selected dose – Umbrella DR (N = 150)
                                    2     4   6   8                     2   4   6   8                     2     4   6   8

                      9 doses           9 doses           9 doses       9 doses             9 doses           9 doses           9 doses
                      ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
           40
           30
           20
           10
            0
                      7 doses           7 doses           7 doses       7 doses             7 doses           7 doses           7 doses
                      ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
                                                                                                                                              40
% Trials




                                                                                                                                              30
                                                                                                                                              20
                                                                                                                                              10
                                                                                                                                              0
                      5 doses           5 doses           5 doses       5 doses             5 doses           5 doses           5 doses
                      ANOVA               Dopt             GADA         MCPMod                MTT              BMA              LOCFIT
           40
           30
           20
           10
            0

                  2     4   6   8                     2     4   6   8                   2     4   6   8                     2     4   6   8
                                                                    Dose selected

                  Distribution of selected doses is wide for all methods

                                                                                                                                                   23
Sample predicted curves – Logistic DR, 9 doses (N = 150)
                                  LOCFIT

                     1

                     0

                     -1                            Sample       Median              True

                     -2

                     -3
                                  MCPMod            MTT                  BMA

                                                                                        1
      Predicted DR




                                                                                        0

                                                                                        -1

                                                                                        -2

                                                                                        -3
                                  ANOVA             Dopt                 GADA

                     1

                     0

                     -1

                     -2

                     -3

                          0   2     4      6   8            0     2       4     6   8
                                                   Dose


 Overall shape of DR was described fairly well by all methods

                                                                                             24
Sample predicted curves: Umbrella, 5 doses and N = 250

                                    LOCFIT

                     0.5
                     0.0
                     -0.5
                                                     Sample       Median              True
                     -1.0
                     -1.5
                     -2.0
                                    MCPMod            MTT                  BMA

                                                                                          0.5
      Predicted DR




                                                                                          0.0
                                                                                          -0.5
                                                                                          -1.0
                                                                                          -1.5
                                                                                          -2.0
                                    ANOVA             Dopt                 GADA

                     0.5
                     0.0
                     -0.5
                     -1.0
                     -1.5
                     -2.0


                            0   2     4      6   8            0     2       4     6   8
                                                     Dose


 Overall shape of DR was described fairly well by all methods

                                                                                                 25
                                                   Conclusions

• Detecting DR is considerably easier than estimating it
• Sample sizes for DF studies, based on power to detect DR, are
  generally inadequate for dose selection and DR estimation
• None of methods had good performance in estimating dose in
  the correct target interval: maximum observed percentage of
  correct interval selection was 60% =⇒ larger N needed
• Adaptive dose finding methods lead to gains in power to detect
  DR, precision to select target dose, and to estimate DR –
  greatest potential in the latter two
• GADA had best overall performance, especially on DR
  estimation

                                                                  26
                                       Conclusions (contd.)

• DR estimation performance of model-based methods are
  superior to methods based on hypothesis testing

• Number of doses larger than 5 does not seem to produce
  significant gains, provided:
   – overall N is fixed =⇒ trade-off between more detail about
     DR and less precision at each dose
   – doses cover range of responses (minimum, intermediate,
     and maximum)

• In practice, need to balance gains associated with adaptive
  dose ranging designs approach against greater methodological
  and operational complexity

                                                                 27
                            Preliminary Recommendations

• Adaptive, model-based dose-ranging designs should be
  considered, as they can lead to substantial gains in
  performance over traditional DF methods

• Sample size calculations for Phase II studies should take into
  account desired precision of estimated target dose and possibly
  also estimated DR (current methods are not adequate)

• When resulting sample size is not feasible, should consider
  selecting two or three doses for confirmatory phase to increase
  likelihood of including “correct” dose – adaptive designs could
  be used in confirmatory phase for greater efficiency (e.g.,
  dropping less efficient doses earlier)


                                                                    28
                   Preliminary Recommendations (contd.)

• Proof-of-concept (PoC) and dose selection should be
  combined, when feasible, into one seamless trial

• Early stopping rules, for both efficacy and futility, should be
  used when feasible to allow greater efficiency in adaptive
  designs – Bayesian methods are particularly well-suited for
  this purpose

• Trial simulations should be used to determine appropriate
  sample sizes, as well as for estimating operational
  characteristics of designs/methods under consideration

• Consider extent of overlap in exposure, when specifying doses
  to be studied

                                                                   29
References

Bretz, F., Pinheiro, J. and Branson, M. (2005). Combining multiple
     comparisons and modeling techniques in dose-response
     studies, Biometrics 61(3): 738–748.

Gallo, P., Chuang-Stein, C., Dragalin, V., Gaydos, B., Krams, M.
     and Pinheiro, J. (2006). Adaptive designs in clinical drug
     development: An executive summary of the phrma working
     group, Journal of Biopharmaceutical Statistics 16: 275–283.

Hoeting, J., Madigan, D., Raftery, A. and Volinsky, C. (1999).
     Bayesian model averaging: A tutorial, Statistical Science
     14(4): 382–417.

Krams, M., Lees, K. R. and Berry, D. A. (2005). The past is the

                                                                     30
     future: Innovative designs in acute stroke therapy trials,
     Stroke 36(6): 1341–7.

Liu, Q. (2006). Adaptive dose-response phase II trials for clinical
     development, Joint Statistical Meeting, Seattle, WA.




                                                                      31

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:20
posted:8/6/2011
language:English
pages:31