Performance Measurement in Pediatric Emergency Care by jennyyingdi


									Performance Measurement in
 Pediatric Emergency Care
       Evie Alessandrini, MD, MSCE
       Center for Health Care Quality
      Division of Emergency Medicine
Cincinnati Children’s Hospital Medical Center

I have no relevant financial relationships
with the manufacturers of any commercial
products and/or provider of commercial
services discussed in this CME activity.

I do not intend to discuss an unapproved or
investigative use of a commercial
product/device in my presentation.
   Performance Measurement

Learning Objectives
• Delineate the different purposes of
  performance measurement
• Become familiar with existing endorsed
  performance measures of emergency care
• Develop a framework for organizing
  pediatric emergency care performance
  measures for your ED
    Measurement Motivators
• Health Care System
  – Growing complexity and costs
  – Transparency initiative, external mandates
• Institute of Medicine Reports
  – To Err is Human, 1999
  – Crossing the Quality Chasm, 2001
  – Performance Measurement: Accelerating
    Improvement, 2005
  – The Future of Emergency Care, 2006
    • Develop national standards for emergency care
      performance measurement

               Stakeholder Survey Analysis

Why Measure Performance?
                             Stakeholder Survey Analysis
 Why Measure Performance?
• Improve
  – Health and Healthcare
    • For patients and populations
    • Within one ED or with one practitioner
    • Within networks of EDs or health systems
• Inform
  – Transparency, consumer decision-making
  – Regionalization of care
• Incentivize
  – Pay for performance
  – National rankings
        Performance Measurement:
         Accelerating Improvement
                   2005 IOM Report
• The ultimate purpose of performance measurement is to
  improve the health of everyone in the United States
• Performance measures are yardsticks by which all
  health care providers and organizations can determine
  how successful they are in delivering recommended care
  and improving patient outcomes
• Public reporting of performance data holds health
  providers accountable to both consumers and
  purchasers of care; transparency builds trust
• Patients can also learn what the expected professional
  standards of care are and where they can go to receive it
       Performance Feedback
Feedback has proven most effective when
• rates of adherence to practice guidelines are low
• the information is directly useful for care
• practitioners are motivated to change

IOM. Rewarding Provider Performance: Aligning
  Incentives in Medicine. Washington DC:
  American Academy Press; 2006.
       Feedback Report: Appropriate Use of
     Amoxicillin for Newly Diagnosed Acute Otitis
                      Media (AOM)
                                              Quality Measure (Numerator): Amoxicillin
        Your            Top Performers        Prescribed
                                              Quality Measure (Denominator): All Visits
                         (December 2009 –
    (December 2009 –                          for Acute Otitis Media Satisfying Inclusion
                            February 2010)
       February 2010)                         and Exclusion Criteria
       81%                 100%               Inclusion Criteria:
                                                 Visit Diagnosis of Acute Otitis Media
                                                 Children 2 months to 12 years of age
                                                 Oral antibiotic prescribed at the visit

                                              Exclusion Criteria: Acute Otitis Media
                                              Visits in the Past 14 Days
                                                  Antibiotics in the Past Month
                                                  Penicillin/Amoxicillin Allergy
                                                  Co-infections: Pneumonia, Sinusitis,
                                              Conjunctivitis, or Pharyngitis

Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042
  Feedback Report: Appropriate Use of Amoxicillin
   for Newly Diagnosed Acute Otitis Media (AOM)
                                    Appropriate Amoxicillin Use

   Time                             You                Your Practice      Network
                      # of Visits   Amoxicillin       Amoxicillin      Amoxicillin
                      with Newly    Appropriately     Appropriately    Appropriately
                      Diagnosed     Prescribed        Prescribed       Prescribed
                      AOM           (N, %)            (%)              (%)
   September 2009         19              18 (95%)         91%              77%
   December 2009          16              13 (81%)         93%              77%

Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042
“Transparency aims to reduce specific risks or performance
   problems through selective disclosure by organizations.
   The ingeniousness of target transparency lies in its
   mobilization of individual choice, market forces, and
   participatory democracy.” Wikipedia

Levels of transparency
     – Within your department
     – Within your hospital
     – Outside your hospital

         Pay for Performance
“Pay for performance systems link compensation
  to measures of work quality or goals”
Perfect Asthma Care
• The pay-for-performance program rewarded
  practices for
  – participating in an improvement collaborative
  – achieving network- and practice-level performance
  – building improvement capability
• The percentage of the network asthma
  population receiving "perfect care" increased
  from 4% to 88%.
       Mandel, KE; Archives of Ped and Adol Med: 161(7): 650-5, 2007 July
     Leadership & Performance
“Turning Doctors into Leaders”
• Organize doctors into teams
• Measure performance by patient
  outcomes, not resource use
• Apply financial and behavioral incentives
• Dismantle dysfunctional cultures
               Harvard Business Review, April 2010;
   Performance Measurement

Learning Objectives
• Delineate the different purposes of
  performance measurement
• Become familiar with existing endorsed
  performance measures of emergency care
• Develop a framework for organizing
  pediatric emergency care performance
  measures for your ED
              National Quality Forum
                      2008 Measures
Hospital-based Emergency Care
• Median time from ED arrival to ED departure for
  admitted patients*
• Median time from ED arrival to ED departure for
  discharged patients*
• Admit Decision Time to ED Departure Time for Admitted
• Door to provider
• Left without being seen

* Measures stratified by
   – Psychiatric diagnoses, observation patients, transfers, all others

            National Quality Forum
                    2008 Measures

Hospital-based Emergency Care (cont.)
• Severe Sepsis and Septic Shock: Management Bundle
• Confirmation of endotracheal tube placement
• Percentage of patients with Chest Pain Symptoms in ED
  receiving Early Therapy including IV, Oxygen,
  Nitroglycerin, Morphine and Chewable Aspirin on Arrival
• Pregnancy test for female abdominal pain patients
• Anticoagulation for Acute Pulmonary Embolus Patients
• Pediatric Weight documented in kilograms
   – AAP is the measure sponsor

Children’s Health Corporation of America
            Existing Measures

Whole System Measures
• ED Left Without Being Seen
• ED Length of Stay

             Urgent Matters
    Robert Wood Johnson Foundations

Urgent Matters Initiative Goals
• Rigorously evaluate the implementation of
  strategies for improving patient flow and
  reducing ED crowding
• Advance the development of standard
  performance measurement in the ED
• Promote spread of promising practices to
  a wider audience / variety of hospitals
                 Urgent Matters
      Robert Wood Johnson Foundations
Performance Measures (NQF and CMS)
• Throughput for admitted patients
• Throughput for discharged patients
• Time to pain management for long bone fractures
  – Admitted patients
  – Discharged patients
• Median time to chest x-ray
  – Admitted patients
  – Discharged patients
• Admit decision time to ED departure time
     Other Existing Measures and
     Measurement Organizations
• Joint Commission
   – ORYX performance measures
       • Children’s Asthma Care measures (inpatient)
• AHRQ Pediatric Quality Indicators (PDIs)
   – 18 risk-adjusted measures
   – Obtained from inpatient administrative data
• Alliance for Pediatric Quality
   – AAP, American Board of Pediatrics, CHCA, NACHRI
   Other Existing Measures and
   Measurement Organizations
ACEP National Report Card on the State of
  Emergency Medicine
• Access to Emergency Care
• Quality and Patient Safety Environment
• Medical Liability Environment
• Public Health and Injury Prevention
• Disaster Preparedness
   Performance Measurement

Learning Objectives
• Delineate the different purposes of
  performance measurement
• Become familiar with existing endorsed
  performance measures of emergency care
• Develop a framework for organizing
  pediatric emergency care performance
  measures for your ED
      Rationale for Framework
Limitations of prior work
  – Single centers or geographic locales
  – Focus on condition-specific indicators
  – Preponderance of process-oriented measures
  – Benchmarks very focused on
     • Timeliness (through put)
     • Satisfaction (ceiling effect)
  – Lack of comprehensiveness regarding
    spectrum of ED care
     • Lindsay et. al., AEM, 2002
     • Guttmann et. al., Pediatrics, 2006
 Performance Measure Framework

Quality indicator set development process
• Adapted from AHRQ
• “Defining Quality Performance Measures for
  Pediatric Emergency Care”
  – Funded by HRSA/EMSC Targeted Issues
    Grant H34MC08512
Performance Measure Development


                                                   Measure by
                        Candidate                   Measure
                        Measures                   Evaluation
           Actual Use
                         N = 60                   for Selection


Evaluation for          Measure Set

                        Adapted from AHRQ PDI development process
Performance Measure Framework

Measure development dimensions
• IOM Quality Domains
• Donabedian’s framework for quality
• PEM disease frequency and severity

Measure evaluation dimensions
• National Quality Forum criteria
         Institute of Medicine
           Quality Domains
Built around the core
 need for health
 care to be
• Safe
• Effective
• Efficient
• Timely
• Patient-centered
• Equitable
       Institute of Medicine
         Quality Domains
• Health care avoids injuries to patients
  from the care that is intended to help
• Health care provides services based on
  scientific knowledge to all who could
  benefit, and refrains from providing
  services to those not likely to benefit
       Institute of Medicine
         Quality Domains

• Health care avoids waste, including waste
  of equipment, supplies, ideas and energy
• Health care reduces waits and sometimes
  harmful delays for both those who receive
  and those who give care
        Institute of Medicine
          Quality Domains
Patient - centered
• Health care provides care that is respectful of
  and responsive to individual patient preferences,
  need and values, and ensures that patient values
  guide all clinical decisions
• Health care provides care that does not vary
  because of personal characteristics such as
  gender, ethnicity, geographic location, and
  socioeconomic status
       Donabedian’s Framework
• Indirect quality-of-care measures related to a physical
  setting and resources: Staff, space, supplies, equipment
  and financial resources
• Measures evaluate the method or process by which care
  is delivered, including both technical and interpersonal
• Outcome elements describe valued results related to
  lengthening life, relieving pain, reducing disabilities and
  satisfying the consumer
        PEM Disease
     Frequency & Severity
• Condition-specific
  – Proportion of patients with croup receiving
• General
  – Proportion of patients returning to the ED within
    72 hours of an initial ED visit
• Cross-cutting
  – Proportion of patients with pain who receive an
Choosing Condition-Specific Measures

                 25.00   24.59

                 20.00                     19.22
% of Diagnosis

                                                                                                                NHAMC S
                                                                                                                PC DP

                                                              9.97             9.65              9.64


                          Trauma          ENT, Dental &    Gastrointestinal   Respiratory     Systemic States
                                         Mouth Diseases       Diseases         Diseases
                                                            Major Group

                                          Alessandrini, Academic Emerg Med; February 2010
  Measure Evaluation Criteria
• The measure reflects a priority or high
  impact aspect of healthcare
• The measure addresses outcomes or is
  strongly linked to improving outcomes
• The measure addresses an area of
  considerable variation or poor
  performance across providers or
  population groups
  Measure Evaluation Criteria
Scientific Acceptability
• There is strong evidence for the specific
  measure focus, such as evidence based
• The measure is reliable, reproducible and
  accurately represents quality of care
  Measure Evaluation Criteria
• The measure provides information that is
  actionable and can be used to make
  decisions that improve the quality of care
• The measure is meaningful and
  Measure Evaluation Criteria
• Data for the measure is generated during
  care delivery and is available in the EHR
  or other electronic sources
• Data collection for the measure can be
• The information provided outweighs the
  costs/burdens of collecting the data
    Results: Performance Measure
     Distribution by IOM Quality
                                                     Applicability of Measures to IOM Domains
                                                     (Measures can apply to more than one IOM domain)
  # of Measures Applicable to IOM

                                    40                         34

                                    20                                                                      17
                                    10                                                            7
                                         Effective            Safe         Patient Centered    Efficient   Timely

Equitable – measures stratified by gender, age, race, ethnicity and payor
                Results: Performance Measure
                 Distribution by Donabedian
                     Distribution of Measures by Donabedian Classification

                40                                37
# of Measures

                15       13
                       Outcome                  Process                Structure
                                             Measure Type
                      Results: Performance
                     Measure Distribution by
                        Diagnosis Type
                                 Distribution of Measures by Category

                30       27

# of Measures


                15                                    13



                       General                   Cross-Cutting          Condition-Specific

  Cross-cutting measures include pain/sedation, severe illness, diagnostic
  testing and medication management
   PEM Balanced Report Card
• Measuring weight in kilograms for ED patients <18
  years of age
• All pediatric equipment present in the ED
     (per ACEP, AAP, ENA policy statement)
• Reducing pain in children with acute fractures
• Systemic corticosteroids in asthma patients with
  acute exacerbation
• Medication error rates
• Parent/caregiver understanding of ED discharge
• Door to provider
• Total ED length of stay
        Further Considerations
• Measures valuable to patients
    – Not minimum level of competency
• Composite measures
    – Conceptual and analytic issues
•   Unit of analysis
•   Measure target
•   Locus of control
•   Balancing measures
    – Are we improving parts of our system at the
      expense of others?
        Measurement & Quality
• Quality is central to achieving affordable care
  that knows patient needs and keeps them
• It’s a three step process
   – The first step toward achieving quality is convening expert
     members across the healthcare industry, including patients to
     define quality with uniform standards and measures that apply to
     the many facets of care patients receive.
   – Second, information gleaned from measuring performance is
     reported and analyzed to pinpoint where patient care falls short.
   – Third, caregivers examine information about the care they are
     providing and use it to improve.

Measure. Report. Improve.
• AAP Policy Statement: Principles for the
  Development and Use of Quality Measures
  – Pediatrics 121 (2), February 2008, pp 411-418
• Pediatric Clinics of North America “Pediatric
  Quality”: Quality Measures in Pediatrics
  – Volume 56 (4), August 2009, pp 816-829
• Institute of Medicine Report: Performance
  Measurement, Accelerating Improvement
  – December 2005
• Joint Policy Statement—Guidelines for Care
  of Children in the Emergency Department
  – Pediatrics 2009;124:1233–1243
    Steps in Measure Specification

•   Numerator statement
•   Denominator statement
•   Denominator exclusions
•   Data source and collection methods
•   Sampling
•   Risk adjustment
•   Stratification to detect disparities
•   Level of measurement / analysis
  Steps in Measure Specification
Risk Adjustment
• Accounts for patient-associated factors
  before comparing outcomes across
• “Levels the playing field”
• Would be unnecessary if patients were
  randomly assigned to treatments, settings

To top