Market Intelligence

Document Sample
Market Intelligence Powered By Docstoc
					 ePRO Evolution and
Emerging Best Practice
       Adam Wood
 PDSM PRO Workshop 2010
Content

•   Defining PROs
•   Where are we now?
•   Why ePRO?
•   Obstacles
      – Emerging Best Practice
• What about….?
• For reference




2 August 2012                confidential   Page 2
Defining PROs
Defining PROs

• “A PRO is a measurement of any aspect of a
  patient’s health status that comes directly from the
  patient” - FDA
      – Patient diaries
      – Questionnaires
      – Disease specific measures

• ePRO is any electronic implementation
      – IVRS
      – Hardware device (e.g. PDA, tablet)
      – Digital pen


2 August 2012                 confidential          Page 4
Where Are We Now?
PROs are all round us

• CenterWatch estimate they are used in 75% of
  trials
      – Visit based measures more common that remote
        reported diaries
• Some PROs are more important than others
• Growing need for Health Economic data
      – Proving efficacy is one thing, persuading payers to fund
        your treatment can be quite another
      – Increasing trend to integrate Health Economics data
        alongside efficacy data in Ph II/III


2 August 2012                 confidential                    Page 6
ePRO is Mainstream

• A number of proven vendors with strong track
  records
• Best practice is well established
       – Success is vastly more likely than failure
• Regulators are positive about PROs
       – “Some treatment effects are known only to the patient”*
• ePRO is an integral part of PROs
       – Not a separate, special case
       – And it’s not EDC

* FDA PRO Guidance.


 2 August 2012                  confidential                  Page 7
ePRO and Regulators

• Regulators in Europe and the US are keenly
  supportive of electronic PROs
      – invivodata regularly interacts with EMA and FDA
• Drugs approved based on ePRO data worldwide
      – Japan (at least one)
      – Europe (multiple)
      – US (multiple)
• invivodata inspected by regulators in all 3



2 August 2012                  confidential               Page 8
FDA PRO Guidance

• Finalised in 2009; drafted in 2006
      – Defines new “rules of engagement” for use of PROs
• Questions it triggers
      –   Conceptual framework
      –   Application of scale to patient population
      –   Patient input into PRO design
      –   Reliability, validity, ability to detect change
      –   Translations

      – And, oh yes….mode of administration (p or e)

2 August 2012                      confidential             Page 9
FDA “Spoke and Wheel”*
                                                      i.   Hypothesize Conceptual Framework
                                                      •    Outline hypothesized concepts & potential claims
                                                      •    Determine intended population
                                                      •    Determine intended application\characteristics (types
                                                           of scores, mode and frequency of administration)
                                                      •    Perform literature/expert review
                                                      •    Develop hypothesized conceptual framework
                                                      •    Place PROs within preliminary endpoint model
                                                      •    Document preliminary instrument development
                       v. Modify Instrument                                                                 ii. Adjust Conceptual
                       •    Change wording of items,                                                            Framework & Draft
                            populations, response
                            options, recall period, or                                                          Instrument
                            mode of administration/                                                         •      Generate new items
                            data collection                                                                 •      Create instrument
                                                                                 PRO                        •      Select recall period,
                       •    Translate & culturally adapt
                            to other languages                                     ↕                               response options, & format
                       •    Evaluate as appropriate                              Claim                      •      Select mode of
                       •    Document all changes                                                                   administration /
                                                                                                                   data collection
                                                                                                            •      Conduct cognitive debriefing
                       iv. Collect, Analyze, &                                                              •      Pilot test draft instrument
                                                                                                            •      Document content validity
                           Interpret Data
                       •    Prepare protocol & statistical analysis
                            plan (final endpoint & responder                 iii. Confirm Conceptual Framework &
                            definition)                                           Assess Other Measurement Properties
                       •    Collect & analyze data
                                                                             •    Confirm conceptual framework with scoring rule
                       •    Evaluate treatment response using
                                                                             •    Assess score reliability, construct validity, & ability to
                            cumulative distribution of response &
                                                                                  detect change
                            responder definition
                                                                             •    Finalize instrument content, format, scoring, procedures &
                       •    Document interpretation of treatment
                                                                                  training materials
                            benefit in relation to claim
                                                                             •    Document measurement development



* Figure from: Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, et al. FDA Paper Draft:
Patient-Reported Outcomes to Support Medical Product Labeling Claims. Value in Health 2007

2 August 2012                                                             confidential                                                            Page 10
FDA on Patient Compliance



                                                               “If a patient diary or some other
                                                               form of unsupervised data entry is
                                                               used, the FDA plans to review the
                                                               protocol to determine what
                                                               measures are taken to ensure
                                                               that patients make entries
                                                               according to the study design
                                                               and not, for example, just before a
                                                               clinic visit when their reports will
  (Federal Register, Vol. 71, No. 23; Feb. 3, 2006)            be collected."
                                                               (Pg. 10; 334-337 – emphasis added)




2 August 2012                                         confidential                                  Page 11
Why ePRO?
Is This Familiar?



                                                                                                Missing data




                                                                                          Ambiguous data



                                                                                         Conflicting data


                Extraneous data
From: “Technology Solutions for the collection of patient reported outcomes data”. Bill Byrom. European
Pharmaceutical Contractor, 2004.
2 August 2012                                          confidential                                         Page 13
Benefits of ePRO

• Who hasn’t had a paper diary nightmare?
• Process benefits on the theme of cleaner data
  quicker
      –   More complete – high compliance
      –   Improved integrity – date:time stamps
      –   Increased consistency – logic checks
      –   No free or extraneous text
      –   Automatically coded
• Don’t forget better science - far greater compliance
  with protocol
      – Who’d want to defend paper diary data to a regulator?

2 August 2012                   confidential                Page 14
eDiary v Paper

                      NCI & invivodata Diary Methods Study
                    Stone et al. (2002): British Medical Journal

           Instrumented Paper Diary*                  invivodata eDiary†
                    N = 40                                  N = 40




                *Contained hidden                        †ActualeDiary used
                   photosensor                             in 2002 Study




2 August 2012                          confidential                           Page 15
eDiary v Paper

 Results: eDiary vs. Paper Diary
 Stone et al. (2002): British Medical Journal

  • Paper Compliance                             100

      – Reported: 90%
                                                 80
      – Actual: 11%
                                                 60

  • eDiary Compliance
                                                 40
      – Reported: 94%
      – Actual: 94%                              20


                                                  0
                                                       Paper                 eDiary
                                                       Reported compliance
                                                       Actual compliance

2 August 2012                     confidential                                        Page 16
Obstacles
Obstacles and overcoming them

                   • Fear of change
                          – We’ve always done it this
                            way
                          – Precedent not acceptable to
                            regulators




2 August 2012     confidential                     Page 18
Obstacles and overcoming them

• Lack of awareness of
  modern ePRO
      – No longer something like
        this….
      – Or this….




2 August 2012                confidential   Page 19
Device ePRO examples




2 August 2012    confidential   Page 20
More obstacles

• Not enough time?
      – Not enough time for what?
      – Not enough time to plan for success = planning for
        failure
• Sites won’t like it?
      – Where’s the evidence for this?
      – Who pays them?
      – Doesn’t stop us using EDC widely




2 August 2012                 confidential                   Page 21
PRO Scale Migration

• Consensus has emerged on best practice
• Documented by ISPOR “PRO Good Research
  Practices Task Force”
• Published in Value in Health
      – Coons et al, 1098-3015/08
      – “Recommendations on Evidence Needed to Support Measurement
        Equivalence between Electronic and Paper-Based Patient-Reported
        Outcome (PRO) Measures: ISPOR ePRO Good Research Practices
        Task Force Report”




2 August 2012                    confidential                      Page 22
Levels of Modification

    Level of                      Rationale
    Modification
    Minor                         The modification can be justified on the basis
                                  of logic and/or existing literature. No change
                                  in content or meaning.
    Moderate                      Based on the current empirical literature, the
                                  modification cannot be justified as minor.
                                  May change content or meaning.
    Substantial                   There is no existing empirical support for the
                                  equivalence of the modification and the
                                  modification clearly changes content or
                                  meaning
    Adapted from Shields et al.
2 August 2012                              confidential                       Page 23
Examples of Modification

   Level of                       Examples
   Modification
   Minor                          1) Non-substantive changes in instructions (e.g., from
                                  circling the response to touching the response on a
                                  screen).
                                  2) Minor changes in format (e.g., one item per screen
                                  rather than multiple items on a page).
   Moderate                       1) Changes in item wording or more significant
                                  changes in presentation that might alter interpretability.
                                  2) Change in mode of administration involving different
                                  cognitive processes (e.g., paper [visual] to IVR
                                  [aural]).
   Substantial                    1) Substantial changes in item response options
                                  2) Substantial changes in item wording


    Adapted from Shields et al.
2 August 2012                                 confidential                                Page 24
Evidence Required

     Level of                          Level of Evidence
     Modification

     Minor                             Cognitive debriefing
                                       Usability testing

     Moderate                          Equivalence testing
                                       Usability testing

     Substantial                       Full psychometric testing
                                       Usability testing


         Adapted from Shields et al.

2 August 2012                                   confidential       Page 25
Patients can’t use it?

• Severe Parkinson’s Disease patient




2 August 2012            confidential   Page 26
Living with the PRO Guidance
Emerging Best Practice

• PRO Strategy
      – PRO Dossier
      – PRO input into protocol
      – An Endpoint Development Process




2 August 2012               confidential   Page 28
“Begin with the end in mind”



                      Labeling Claims

                 Disease/Conceptual Model

                    Measurement Strategy

                     Instruments/Items


                   Conceptual Framework

                       Endpoint Model


                         Endpoints

2 August 2012             confidential      Page 29
A Few Words on Instruments

 • When is a PRO instrument adequate to support labeling
   claims?

        – “The adequacy of a PRO instrument as a measure to support
          medical product labeling claims depends on its documented
          measurement properties that demonstrate the instrument is ‘fit for
          purpose’” (Burke, 2008).
                • “Fit for purpose” means that the instrument is specific to and defined by the
                  patient population and the specified disease, treatment, and selected
                  endpoints.

        – In this circumstance, “instrument is defined as a means to
          capture data (i.e., a questionnaire) plus all the information and
          documentation that supports its use” (Burke, 2008).




2 August 2012                                  confidential                                   Page 30
Impact of FDA PRO Guidance

• Study Endpoints and Labeling Division (SEALD) are
  reviewing programs where labeling claims based upon
  PRO
• Feedback from FDA specifically cites PRO Guidance
• Expectations for PRO activity and documentation being
  driven by Guidance
• PRO Submission/dossier template
• Created a need for practical resources to better understand
  the nature of PROs
      – best practices that facilitate the effective development of PRO
        instruments.




2 August 2012                       confidential                          Page 31
Common Pitfalls/Myths

• We can go at risk and then just ask the FDA after-
  the-fact
• Lack of patient-interview based data
      – Needed to support the importance and relevance of the
        concepts and items in the PRO instrument
• The instrument is ‘valid’ in patient population A,
  therefore it is OK to use in patient population B
• We can wait until Ph III to address the PRO
  instrument issues



2 August 2012                confidential                  Page 32
Endpoint Development Process

• A tool for meeting scientific and regulatory needs


                                   Ongoing Synthesis of
                                         Material

  Conceptual
                 Concept        Measurement            Instrument      Conceptual   Endpoint
    Model
                Justification     Strategy            Identification   Framework     Model
 Development




2 August 2012                          confidential                                   Page 33
Endpoint Development Process
                                       Ongoing Synthesis of
                                             Material

 Conceptual
                    Concept        Measurement                    Instrument           Conceptual        Endpoint
   Model
                   Justification     Strategy                    Identification        Framework          Model
Development

     Identify                                                                             Theoretical       Study
                       Expert         Focus of                     Identify Existing
    Program                                                                                Validation       Design
                        Input        Assessment                      Instruments
  Goals & Claims                                                                         Evidence/Plan   Requirements



     Identify                                                                            Psychometric    Construction
                       Patient        Interval of                  Modify Existing
     Relevant                                                                              Validation        of
                       Reports       Assessment                     Instruments
    Population                                                                           Evidence/Plan    Endpoints



                                                                      Identify
     Identify
                      Empirical        Recall                        Alternate
    Relevant
                      Evidence         Period                       Measurement
   Drug Effects
                                                                     Strategies



    Identify                          Timing &
                                                                   Develop Novel
    Relevant                         Schedule of
                                                                    Instruments
  PRO Concepts                       Assessment


 2 August 2012                                    confidential                                               Page 34
Conceptual Model:
Specifying & Substantiating Concepts

                                 Signs/             Disease        General
                               Symptoms             Related        Impact
                                                    Impact
                 Drug Action
                                               Concept Justification

                                                    Impact 1A
                Relevant
    Patient
                Disease        • Signs                             Impact 2A
   Population
                Process        • Symptoms
                                                    Impact 1B

   Diagnosis




2 August 2012                   confidential                                   Page 35
Concept Justification

Concept                                    Expert                  Empirical         Patient          Payor
                                            input                    Data          Interviews       Relevance

Symptom

                      Symptom A

                      Symptom B

Symptom Impact

                Symptom Impact A

                Symptom Impact B


                           Yes indicates that this justification criterion has been satisfied
                        No indicates that this justification criterion has not yet been satisfied
                                           TBD indicates To Be Determined

2 August 2012                                       confidential                                          Page 36
Conceptual Framework

                    PRO
                                 Item 1
                   Concept

                                 Item 2
   Label Claim     PRO
                  Concept
                                 Item 3


                   PRO           Item 4
                  Concept
                                 Item 5
   Label Claim
                   PRO
                  Concept        Item 6



2 August 2012     confidential            Page 37
PRO Strategy Pointers

• Begin thinking of PRO strategy early in
  development
• Interact with regulators
• PRO EDP provides structure for elements of PRO
  Strategy and Dossier
• Think about preparation for trial implementation




2 August 2012          confidential             Page 38
What about….?
What about ePRO Design

• ePRO Solution Design
      – Maximise value of new modalities
                • Don’t simply recreate the weaknesses of paper instruments
      – Less missing data
      – More precise measures
      – More frequent, reliable measures




2 August 2012                           confidential                          Page 40
FDA’s view on PRO Design



                                                               “PRO instruments that require
                                                               patients to rely on memory …
                                                               may threaten the accuracy of
                                                               the PRO data. It is usually better
                                                               to construct items that ask
                                                               patients to describe their current
                                                               state than to ask them to compare
                                                               their current state with an earlier
                                                               period or to attempt to average
  (Federal Register, Vol. 71, No. 23; Feb. 3, 2006)            their experiences over a period of
                                                               time.”
                                                               (Pg. 11; 339-343 – emphasis added)



2 August 2012                                         confidential                                  Page 41
What about…?

• Psychometric validation
      – Luckily the FDA have written us an abridged text book
        on psychometrics
      – It’s called the FDA PRO Guidance
      – Applies to paper as well
      – Eliciting the patient perspective is a central tenet




2 August 2012                 confidential                  Page 42
Psychometric Validation

• Large proportion of the FDA PRO Guidance refers to
  psychometric aspects
• With any PRO you need to show evidence that:
      – “The adequacy of a PRO instrument as a measure to support
        medical claims depends on its development history and
        demonstrated measurement properties”
      – Reliability
      – Validity
      – Ability to detect change
      – Interpretability
• A specialised area that applies as much to paper as
  electronic

2 August 2012                      confidential                     Page 43
What about Linguistic Validation?

• Translations (to a high standard)
      – Full title: linguistic validation & cultural adaptation
•   Best practice well established
•   Plenty of vendors to choose from
•   Timelines typically 8 – 11 weeks
•   Applies to paper as well




2 August 2012                    confidential                     Page 44
For Reference
Scientific Improvements

• Patients are answering questions at the time of
  clinical interest
      – In other words as specified in the protocol and not just
        before a clinic visit
• Patients give more complete answer sets so
  improving statistical power (ref: ACT 2004)
• ePRO allows more sophisticated and targeted
  questioning
• Competing compounds are now likely to be using
  ePRO
      – Who wants to be seen using inferior methods.
2 August 2012                  confidential                    Page 46
Regulatory Motives

• EMA and FDA regularly accept ePRO data for
  primary and secondary endpoints.
      – This is not something they view with suspicion.
• Recently BfArM (the German regulatory authority)
  explicitly directed a sponsor to use ePRO rather
  than paper
• The FDA PRO Guidance sets out clear
  expectations for the use of any form of PROs
  (paper or electronic)
      – http://www.fda.gov/downloads/Drugs/GuidanceComplian
        ceRegulatoryInformation/Guidances/UCM193282.pdf


2 August 2012                 confidential                Page 47
Features of Different ePRO Types

Feature                                     IVRS    eDiary   ePen
(incomplete list)
Automated coding of data                                    
Time-stamps                                                 
Time windows                                         
Alarms / reminders                                   
Branching logic                                      
Prevent question skipping                            
Prevent extra “writing”                              
Graphics                                                     
Replace Clinician Interviews                    
Verbal Recordings                               
High-frequency assessments                            
Episodic, time-based reporting                        
Illiterate subjects                             
2 August 2012                    confidential                     Page 48
    Thank You
      Adam Wood
e: awood@invivodata.com
    +44 78 5492 8430

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:14
posted:8/3/2012
language:English
pages:49