Patient-Centered Outcomes Research Institute by u0m9yC


									   Patient-Centered Outcomes
        Research Institute
What is it and what does it mean for health
            services researchers?

           Patrick S. Romano, MD MPH
UC Davis Center for Healthcare Policy and Research
                  March 28, 2012
Speaker’s Verbal Disclosure Statement:
Have you (or your spouse/partner) had a personal financial relationship
in the last 12 months with the manufacturer of the products or services
that will be discussed in this CME activity?

_x_ Yes

___ No

(If yes, please state disclosures and resolutions)

I received a $200 honorarium and reimbursement of travel
expenses for participating in a PCORI grant review meeting.
Educational objectives for this seminar:
Please type between two and four educational objectives here.

To describe the origin and mission of PCORI

To differentiate Patient Centered Outcomes Research from
Comparative Effectiveness Research (if possible?)

To explain the PCORI grant review process and discuss future
opportunities for UC Davis/CHPR research support
                  What is CER?
• Comparative effectiveness research is the conduct and
  synthesis of systematic research comparing different
  interventions and strategies to prevent, diagnose, treat
  and monitor health conditions. The purpose of this
  research is to inform patients, providers, and decision-
  makers, responding to their expressed needs, about
  which interventions are most effective for which
  patients under specific circumstances. To provide this
  information, comparative effectiveness research must
  assess a comprehensive array of health-related
  outcomes for diverse patient populations…
   – Federal Coordinating Council for Comparative Effectiveness
     Research, definition; [cited 28 July 2010].
               What is PCORI?
• Prior to several years ago, no coordinated effort for
  comparative effectiveness research (CER)
• 2009 stimulus package provided $1.1 billion for
  CER (most went to AHRQ and NIH)
• 2010 Affordable Care Act created Patient Centered
  Outcomes Research Institute
• Funded by tax on government and private insurers
  for each covered life (~$500 million per year)
                    What is PCORI?
• Independent, nonprofit, quasi-governmental organization
   – AHRQ almost eliminated in 1990s after published guideline
     that surgery for back pain was unnecessary
   – Provide political protection
   – PCORI funding does not depend on annual Congressional

• No infrastructure for determining funding priorities,
  issuing opportunities, reviewing and administering grants
   – Outsourcing (NIH, AHRQ, CMS)
      • Peer review, contract management
      • May receive $ from PCORI to originate grants
   – PCORI will be its own entity
1. Identifying national priorities for research.
2. Creating a research agenda based on identified priorities.
3. Funding research consistent with these priorities and agenda.
4. Providing patients and their caregivers with useful research
          PCORI Board of Directors
• Director of AHRQ (Carolyn Clancy)
• Director of NIH (Francis Collins)
• 17 members appointed by the Comptroller General of the
  United States as follows:
   – 3 members representing patients and health care consumers
   – 5 members representing physicians and providers, including at least 1
     surgeon, nurse, State-licensed integrative practitioner, and hospital
   – 3 members representing private payers, including at least 1 health
     insurance issuer and at least 1 self-insured employer
   – 3 members representing pharmaceutical, device, and diagnostic
     manufacturers or developers
   – 1 member representing quality improvement or independent health
     service researchers (Harlan Krumholz or Arnold Epstein)
   – 2 members representing the Federal Government or the States,
     including at least 1 representing a Federal health program or agency
            PCORI Leadership
• Eugene Washington, MD, MSc, (chair), Vice
  Chancellor of UCLA Health Sciences and Dean
  of the David Geffen School of Medicine
• Steven Lipstein, MHA (vice chair), President
  and Chief Executive Officer of BJC HealthCare
• Joe Selby, MD, MPH
  Executive Director
• Anne C. Beal, MD, MPH
  Chief Operating Officer,
              PCORI Mission

PCORI helps people make informed health care
decisions and improves health care delivery and
outcomes by producing and promoting high
integrity, evidence-based information that comes
from research guided by patients, caregivers, and
the broader health care community.
                    PCOR vs. CER
• Why PCORI, not CERI?
• Political concerns that determining which treatments
  better → “rationing”
   – Restrict patients’ ability to choose inferior treatment
• Compromise focus on patient-centered outcomes
• Legislation barred PCORI from
   – Developing or using QALYs
   – Considering cost in making coverage recommendations
   – In practice, PCORI will not fund cost-effectiveness analyses
          Patient-Centered Outcomes Research

• Not just a semantic difference
• Focus on PCOR essential role in shaping funding
  priorities and “rules of the game” for getting funding
• Can’t just replace placebo control with active
  comparison group, do everything else the same
• Distinct elements
   –   Outcomes should be patient-centered
   –   Effectiveness, not efficacy
   –   Heterogeneity of treatment effects
   –   Involvement of stakeholders
   Patient-Centered Outcomes Research

• Outcomes should be patient-centered
  – “…outcomes that people notice and care about
    such as survival, function, symptoms, and health-
    related quality of life”
  – Not biomarkers (BP, serum cholesterol, etc.)
    Patient-Centered Outcomes Research

• Effectiveness
   – Research done in clinically relevant populations
   – Representative of patients
   – Representative of settings

• Heterogeneity of treatment effects
   – “Given my personal characteristics, conditions and
     preferences, what should I expect will happen to me?”
   – “What are my options and what are the benefits and
     harms of those options?”
               Role of stakeholders
• Government payors and regulators; health plans;
  hospitals and other care providers; industry
• Consider how results will shape policy and practice
  through dissemination
• Stakeholders should be intimately involved in the study
  design process to identify important outcomes
   – Trials of anti-seizure drugs in children
   – Traditional measure – seizure frequency
   – Patient-centered measure – parent want to know how
     drug will affect school performance
   – Drug ↓ seizures but ↑ sedation may not be preferable
      Implications for research design
• RCTs
   – Pragmatic - real-world settings; large N’s to assess subgroups
   – Focus on effect size, not just superiority
   – Bayesian and adaptive approaches to design, accounting for
     prior probabilities
• Observational studies
   – Causal inference modeling
   – Tension for studies using pure administrative data –
     powerful and low-cost; lack high-quality patient-centric data
• Cost and cost-effectiveness
   – PCORI not allowed to evaluate cost in making coverage
     recommendations, won’t fund “pure” CEA
   – Cost can be studied and used by decision-makers
          Proposed National Priorities
• Assessment of Prevention, Diagnosis, and Treatment Options. The
  research goal is to determine which option(s) work best for distinct
  populations with specific health problems.
• Improving Healthcare Systems. Focuses on ways to improve healthcare
  services, such as the coordination of care for patients with multiple
  chronic conditions.
• Communication and Dissemination. Looks at ways to provide information
  to patients so that they, in turn, can make informed healthcare decisions
  with clinicians.
• Addressing Disparities. Assures that research addresses the healthcare
  needs of all patient populations. This is needed as treatments may not
  work equally well for everyone.
• Accelerating Patient-Centered and Methodological Research. Includes
  patients and caregivers in the design of research that is quick, safe, and
         PCORI Funding Opportunities
• Developmental
• RFPs (Sept 2011)
     • ~$30-$100K each
     • Methods for setting research priorities

• Pilot Projects Grant Program (Dec 2011)
     •   $13 million in year 1 for ~40 projects
     •   Help develop national research priorities for PCOR
     •   Develop ways to engage patients and bring stakeholders together
     •   Develop measurement tools and data collection methods for novel
         patient-centered outcomes
    PCORI Funding Opportunities
• $150 million in current FY; ultimately perhaps $500
• Mechanisms unknown, priorities under discussion
   – Likely NOT “traditional” research
   – Patient-centered outcomes
      • Including developing methods
   – Real-world populations
   – Heterogeneity of treatment effects
   – Involvement of stakeholders
     Other Funding Opportunities
• Existence of PCORI does not preclude government agencies or
  foundations from funding CER
• AHRQ has Effective Healthcare Program
   – Large evidence-synthesis and dissemination projects
   – EBPCs, DEcIDE network, CERTS
   – Does not support investigator-initiated projects
• Areas of CER explicitly outside PCORI’s mandate
   – CER with “traditional” outcomes
   – Cost-effectiveness analyses
• Remains to be seen
        PCORI Pilot Research Projects
• $13 million in 2012
• Approximately 40 awards
• 2 years maximum support
   – 2nd year of funding must be well justified and will be
     evaluated for successful completion of the first year goals
• $250,000 per year (plus indirect costs)
   – Travel to annual PCORI meeting is an allowable expense
• Requires stakeholder involvement as co-investigators
   – unless not feasible as for an application on analytic
             Principles of grant review
•   Scientific and technical merit
•   Fair, competent and thorough
•   Conflict of interest
•   Confidentiality
•   Difference between overall impact and significance
    – Overall impact: Probability of whether the research will
      exert a sustained, powerful influence on the research field.
    – Significance: Does the project address an important
      problem or a critical barrier to progress in the field? If the
      aims are achieved, how will scientific knowledge, technical
      capability, and/or clinical practice be improved?
                   Structure of application
• Signed Face Page, Table of Contents
• Project Summary: single paragraph using plain language to summarize the
     – Broad objectives
     – Specific aims
     – Relevance to PCORI Objectives
     – Research Design and methods for achieving the goals
• Project/Performance Site(s)
• Key Personnel and Other Significant Contributors
• Biographical Sketches–Principal Investigator, Other Biographical Sketches
    – Personal statement allows each person to explain their role and goals for the project
• Budget and Justification
• Resources
    – Provide information on the facilities to be used in the proposed research project
    – Describe how the research environment contributes to the probability of success
    – Describe how project will benefit unique features of the research environment or
      community involvement or allow for useful collaborative arrangements
    – Describe institutional and community investment in the success of the research
• Checklist
• Addendum Form: Checklist to identify PCORI Areas of Interest
                   Research Plan

•   Specific Aims (1 page)
•   Research Strategy (10 pages)
•   References Cited
•   Protection of Human Subjects
•   Consortium/Contractual
•   Letters of Support

• No appendices are allowed and will be removed by
  PCORI before sending to reviewers
              PCORI Review Criteria

•   Significance (Research Plan)
•   Patient/Stakeholder Engagement (Research Plan)
•   Investigator (Biosketch)
•   Innovation (Research Plan)
•   Approach (Research Plan)
•   Environment (Project/ Performance Sites)
           PCORI Review Criteria -

• Does the project address an important problem or
  critical barrier to PCOR?
• Will the result produce new knowledge that can
  advance PCOR methods or infrastructure?
• Does the investigator demonstrate thorough
  knowledge of previous and ongoing work related to
  their proposed topic?
           PCORI Review Criteria -
      Patient/Stakeholder Engagement

• Will the project make a unique contribution to
  learning about engagement of patients and
• Does the research team demonstrate authentic,
  feasible partnerships with the patients/stakeholders?
• Is there evidence that the stakeholders were involved
  in the preparation of the research proposal?
            PCORI Review Criteria -

• Is the research team well suited to the project?
• Is there appropriate scientific expertise?
• Does at least one member of research team have
  experience in patient/stakeholder engagement?
• Will the research team be able to achieve the study
• Does the research team have complementary and
  integrated expertise? Is the approach and structure
  of the research team appropriate for the project?
           PCORI Review Criteria -

• Does the project address a new method or approach
  for the field of PCOR?
• Does the project apply a proven method or approach
  in a novel way to the field of PCOR?
• Is there a new application or refinement of
  theoretical concepts, approaches, methodologies,
  instrumentation or interventions proposed?
             PCORI Review Criteria -
• Are the overall strategy, methodology, and analyses
  appropriate to accomplish the specific aims?
• Are potential problems, alternative strategies, and
  benchmarks for success presented?
• Will the strategy establish feasibility and will particular
  risky aspects be managed
• Is the budget and timeframe appropriate for the research
• Are there plans for dissemination among key PCOR
  stakeholders in education, practice and policy?
            PCORI Review Criteria -

• Will the environment contribute to the probability of
• Are the institutional support, equipment and other
  physical resources adequate for the project?
• Will the project benefit from unique features of the
  research environment, community involvement,
  patient populations, or non-traditional stakeholder
  collaborative arrangements?
                         PCORI Review
                         Template Based

       Overall Impact

     SCORED REVIEW CRITERIA SECTIONS: The six review criteria, Significance,
     Patient/Stakeholder Engagement, Investigators, Innovation, Approach,
     Environment, with a separate score for the first five criteria.
      1. Significance



                  When Writing Critiques
Overall Impact paragraph –
    – requires a summary narrative (paragraph) of the strengths
      and weaknesses. Do not cut and paste.
    – Focus on major strengths and weaknesses that impacted
      your overall rating of the application
    – Overall impact score is not an average of the criterion
      scores; weights can differ across reviewers
• Use bulleted points to make succinct, focused comments
  (Be careful of being too brief – the reader needs to
  understand your comment)
• Any criterion score between 3 to 9 needs to have weaknesses


9-point score scale is used to provide:
• Criterion Scores for each of the core review criteria
• Overall Impact/Priority Score based on but not an average of the
  core criterion scores plus additional criteria
All applications receive scores:
• Not discussed applications (bottom half by pre-meeting scores) will
  receive only initial criterion scores from the 3 assigned reviewers.
• Discussed applications also receive an averaged overall impact score
  from eligible panel members (i.e., those without conflicts of
• These scores will be averaged to one decimal place, and multiplied
  by 10. The 81 possible priority scores will thus range from 10-90.
                 9-Point Score Scale Descriptors
Impact   Score     Descriptor                       Additional Guidance

           1       Exceptional    Exceptionally strong with essentially no weaknesses

 High      2       Outstanding    Extremely strong with negligible weaknesses

           3        Excellent     Very strong with only some minor weaknesses

           4       Very Good      Strong but with numerous minor weaknesses

           5          Good        Strong but with at least one moderate weakness
           6       Satisfactory   Some strengths but also some moderate weaknesses

           7           Fair       Some strengths but with at least one major weakness

 Low       8        Marginal      A few strengths and a few major weaknesses

           9          Poor        Very few strengths and numerous major weaknesses
           Process During the Meeting
• Chair calls for preliminary overall impact scores
• Primary reviewer
   – Brief description
   – Discuss major/ score-driving strengths and weaknesses
• Second and third reviewers add only major/score-driving
  strengths and weaknesses
• Open discussion among all reviewers
• Human subjects issues
• Final scores from assigned reviewers
• Unassigned reviewers may vote outside the score range
• Record scores (in IAR and on hard copy)
• Budget (does not influence score!)
• 10-15 minutes per application
       Reviewer Voter Sheet
(available only during the meeting)
• Michael Steinman MD, UCSF CTSI Director of
  Comparative Effectiveness Research
• Jonathan Arias, PhD, NIH Center for Scientific
• All of you

To top