Implementation and Analysis of PROs in Clinical Trials

Implementation and Analysis of PROs in Clinical Trials Jeff A. Sloan, Ph.D. DIA Meeting, D.C., June 26, 2005 CM923712-1 Why is it difficult to deal with PROs? • Relatively recent acceptance • 25 years ago physicians were the sole raters of patient pain • JCAHO 2000 guideline: every patient’s pain to be assessed upon intake on a 0-10 scale • Time and experience alleviates novelty and skepticism CM923712-2 Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the improved integration of health-related quality of life assessment in the drug regulatory process. Chassany O et ERIQA Working Group. Drug Information Journal 2002. Checklist for designing, conducting and reporting HRQL - PRO in clinical trials HRQL / PRO objectives • Added value of HRQL / PRO • Choice of the questionnaires • Hypotheses of HRQL / PRO changes Statistical analysis plan • Primary or secondary endpoint Study design • Basic principles of RCT fulfilled ? • Timing and frequency of assessment • Mode and site of administration... • Superiority or equivalence trial • Sample size • ITT, type I error, missing data Reporting of results • Participation rate, data completeness • Distribution of HRQL / PRO scores HRQL / PRO measure • Description of the measure (items, domains…) Interpreting the results • Effect size, • Evidence of validity • Evidence of cultural adaptation CM923712-3 • Minimal Clinically Important Difference • Comparison with other criteria / scores • Number needed to treat… Take home messages: there is good news • There are problems with using PROs as indicators of efficacy in clinical trials. • There are scientifically sound solutions to these problems. The problems have been disseminated widely and consistently. The solutions have not. CM923712-4 It takes a certain amount of bravery to work with PRO’s CM923712-5 Primary goal: advance the state of the science to help patients soar CM923712-6 How do you analyze PRO data? CM923712-7 Science is a candle in the dark - Carl Sagan CM923712-8 We will use the candle of science to improve the QOL of cancer patients … by answering scientific questions • What is the value added of PROs to treatment trials? • How do you deal with multiple endpoints? • How do you handle missing data? • What is the clinical significance of PRO assessments? CM923712-9 What is the value added of additional questions? CM923712-10 CM923712-11 Single-Item or Multiple-Item PRO? Situations where a single item may suffice Phase II study attempting to assess whether a treatment has any impact on QOL A stratification factor for the presence or absence of depressive issues Need to assess fatigue/pain as a correlate of toxicity (brief fatigue/pain inventory) Identifying patients who have need of further QOL assessment (e.g., score of 6 or less on a single item) A clinical setting wherein a basic idea of which domains of QOL (mental, physical, social) may be affected by a particular treatment or situation Situations where a multi-item index may be needed A Phase III study where it is known that QOL is impacted and more delineation of which QOL components are affected is needed A screen to identify the presence or absence of clinical depression Need to assess the impact of fatigue/pain on the activities of daily living (ADL items for pain/fatigue) Detailing the QOL-related issues once a cut off score on a single item has been obtained A clinical setting wherein precise indications of the way in which the different domains of QOL may be affected by a particular treatment or situation CM923712-12 Guidelines for endpoint determination • Several good references (Beitz, 1996; Chassany, 2002; Fayers, 1999; Sloan, 2002) • Reliability and validity data available • Pilot/focus groups to establish R/V • What aspects of PROs are likely to change? • Can one expect an overall change in well-being, health status or QOL? CM923712-13 Recipe for endpoint determination • List PRO aspects likely to change. • Operationalize each item from a tool. • Survey clinicians/patients if unsure. • Keep the total number of items under 25. • Mock up tables with “perfect world” data, labels with “perfect” results. • Link sample size to a priori clinical significance. CM923712-14 How do you deal with multiple endpoints? CM923712-15 An example of combined symptoms: Gemzar (gemcitabine) • Indication: Advanced pancreatic cancer • Instrument or Method: • Negotiated PRO outcome, “clinical benefit response” • PRO Domains Assessed: • Pain, analgesic consumption, performance status, weight • Results: • Clinical benefit response was experienced by 24% of patients receiving Gemzar versus 5% of patients receiving 5FU, p=0.002 CM923712-16 Gemzar-specific clinical benefit response Analgesic Consumption Pain Intensity Pain Performance Scales Responder Improvement in both parameters Stable in one parameter, improvement in another parameter Stable In both parameters Nonresponder Worsening in either parameter Weight Responder 21% Increase in body weight CM923712-17 Nonresponder Stable or decreased weight A patient was considered a clinical benefit responder to Gemzar if …. • The patient showed >=50% reduction in pain intensity or analgesic consumption, or a 20+ point improvement in performance status (for at least 4 weeks with no worsening of other parameters) • Memorial Pain Assessment Card and Karnofsky Performance Scale • The patient was stable on all of the parameters mentioned and showed a marked, sustained weight gain not due to fluid accumulation ( >7% increase maintained for 4 weeks) CM923712-18 O’Brien Global Test for Multiple Outcomes • Example: Venlafaxine for Hot Flashes • Hot flash frequency per day • Hot flash average severity per day • none, mild, moderate, severe, very severe • scored 0, 1, 2, 3, 4 • Hot flash score (severity times frequency) • Uniscale QOL • Hot flash affect on QOL • Toxicity incidence on 11 variables CM923712-19 O’Brien p-values Endpoints Included Hot Flash Frequency Hot Flash Average Severity Hot Flash Score Uniscale QOL Hot Flash Affects QOL Toxicity CM923712-20 p-value 0.0071 0.0050 0.7528 How do you handle the problem of missing data? CM923712-21 Impact of hydrazine sulfate on colorectal cancer patient QOL Impact of different imputation methods for missing data CM923712-22 Effect of imputation method on treatment comparison CM923712-23 The data are usually trying to tell you something…. …you just have to pay attention CM923712-24 What is the clinical significance of PRO assessments? CM923712-25 Two general methods for clinical significance • Anchor-based methods requirements • independent interpretable measure (the anchor) which has appreciable correlation between anchor and target • Distribution-based methods • rely on expression of magnitude CM923712-26 of effect in terms of measure of variability of results (effect size) The MID method in one slide CM923712-27 The ERES Approach • QOL tool range = 6 standard Deviations • SD Estimate = 100 percent / 6 = 16.7% of theoretical range • Two-sample t-test effect sizes (Cohen): small, moderate, large effect (0.2, 0.5, 0.8 SD shift) • S,M,L effects = 3%, 8%, 12% of range CM923712-28 Assessing Clinical Significance • 1) Methods used to date • 2) Group versus individual differences • 3) Single item versus multi-item • 4) Patient, clinician, population perspectives • 5) Changes over time • 6) Practical considerations for specific audiences • MCP, April, May, June 2002 CM923712-29 The solutions found for tumor response cutoffs may provide guidance • We call a reduction of 50% a response. • Have reductions of 49% all the time, but do not worry about misclassification. • Moertel (1976) basis for 50% cutoff • Find a cutoff and stick to it? CM923712-30 The Good News • Statistical, Philosophical, Empirical, Clinical, Historical, Practical approaches to defining a clinically significant effect for symptom assessments are all in the same ballpark is almost always going to be clinically significant • A 10 point difference on a 100-point scale (1/2 SD) • Smaller differences may also be meaningful (data) • Applies to groups or individuals (just different SD) Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes Research Sept 2004; 4(5): 515 – 519 Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press). CM923712-31 What’s next? CM923712-32 A Mayo/NCCTG meeting on FDA guidances on patient-reported outcomes (PRO) Discussion, Education, and Operationalization • FDA to release guidances for assessing PRO’s in all clinical trials (3rd quarter 2005?) • Meeting co-sponsored with FDA to: • provide a focused process to facilitate discussion among all • • • stakeholders educate stakeholders on background, content, and concerns provide an opportunity for input delineate ways to best operationalize the guidance into clinical trials • February 23-25, 2006, DC (Westfields Marriott, Chantilly, • VA, 7 miles from Dulles) Seeking stakeholders involvement CM923712-33 New ideas have enabled us to make advances in PRO science With your help, there will be more to come CM923712-34 Thank you References: jsloan@mayo.edu CM923712-35