Performance Measurement in Pediatric Emergency Care Evie Alessandrini, MD, MSCE Center for Health Care Quality Division of Emergency Medicine Cincinnati Children’s Hospital Medical Center Disclosure I have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity. I do not intend to discuss an unapproved or investigative use of a commercial product/device in my presentation. Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED Measurement Motivators • Health Care System – Growing complexity and costs – Transparency initiative, external mandates • Institute of Medicine Reports – To Err is Human, 1999 – Crossing the Quality Chasm, 2001 – Performance Measurement: Accelerating Improvement, 2005 – The Future of Emergency Care, 2006 • Develop national standards for emergency care performance measurement Outcomes Quality Stakeholder Survey Analysis Why Measure Performance? Stakeholder Survey Analysis Why Measure Performance? • Improve – Health and Healthcare • For patients and populations • Within one ED or with one practitioner • Within networks of EDs or health systems • Inform – Transparency, consumer decision-making – Regionalization of care • Incentivize – Pay for performance – National rankings Performance Measurement: Accelerating Improvement 2005 IOM Report • The ultimate purpose of performance measurement is to improve the health of everyone in the United States • Performance measures are yardsticks by which all health care providers and organizations can determine how successful they are in delivering recommended care and improving patient outcomes • Public reporting of performance data holds health providers accountable to both consumers and purchasers of care; transparency builds trust • Patients can also learn what the expected professional standards of care are and where they can go to receive it Performance Feedback Feedback has proven most effective when • rates of adherence to practice guidelines are low • the information is directly useful for care • practitioners are motivated to change IOM. Rewarding Provider Performance: Aligning Incentives in Medicine. Washington DC: American Academy Press; 2006. Feedback Report: Appropriate Use of Amoxicillin for Newly Diagnosed Acute Otitis Media (AOM) Quality Measure (Numerator): Amoxicillin Your Top Performers Prescribed Performance Quality Measure (Denominator): All Visits (December 2009 – (December 2009 – for Acute Otitis Media Satisfying Inclusion February 2010) February 2010) and Exclusion Criteria 81% 100% Inclusion Criteria: Visit Diagnosis of Acute Otitis Media Children 2 months to 12 years of age Oral antibiotic prescribed at the visit Exclusion Criteria: Acute Otitis Media Visits in the Past 14 Days Antibiotics in the Past Month Penicillin/Amoxicillin Allergy Co-infections: Pneumonia, Sinusitis, Conjunctivitis, or Pharyngitis Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042 Feedback Report: Appropriate Use of Amoxicillin for Newly Diagnosed Acute Otitis Media (AOM) Appropriate Amoxicillin Use Time You Your Practice Network Period # of Visits Amoxicillin Amoxicillin Amoxicillin with Newly Appropriately Appropriately Appropriately Diagnosed Prescribed Prescribed Prescribed AOM (N, %) (%) (%) September 2009 19 18 (95%) 91% 77% through November 2009 December 2009 16 13 (81%) 93% 77% through February 2010 Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042 Transparency “Transparency aims to reduce specific risks or performance problems through selective disclosure by organizations. The ingeniousness of target transparency lies in its mobilization of individual choice, market forces, and participatory democracy.” Wikipedia Levels of transparency – Within your department – Within your hospital – Outside your hospital Examples • http://my.clevelandclinic.org/Documents/Medicine/PEDS283076A_LR.pdf • http://www.dartmouthatlas.org Pay for Performance “Pay for performance systems link compensation to measures of work quality or goals” Perfect Asthma Care • The pay-for-performance program rewarded practices for – participating in an improvement collaborative – achieving network- and practice-level performance thresholds – building improvement capability • The percentage of the network asthma population receiving "perfect care" increased from 4% to 88%. Mandel, KE; Archives of Ped and Adol Med: 161(7): 650-5, 2007 July Leadership & Performance Measurement “Turning Doctors into Leaders” • Organize doctors into teams • Measure performance by patient outcomes, not resource use • Apply financial and behavioral incentives • Dismantle dysfunctional cultures Harvard Business Review, April 2010; hbr.org Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED National Quality Forum 2008 Measures Hospital-based Emergency Care • Median time from ED arrival to ED departure for admitted patients* • Median time from ED arrival to ED departure for discharged patients* • Admit Decision Time to ED Departure Time for Admitted Patients* • Door to provider • Left without being seen * Measures stratified by – Psychiatric diagnoses, observation patients, transfers, all others www.qualityforum.org National Quality Forum 2008 Measures Hospital-based Emergency Care (cont.) • Severe Sepsis and Septic Shock: Management Bundle • Confirmation of endotracheal tube placement • Percentage of patients with Chest Pain Symptoms in ED receiving Early Therapy including IV, Oxygen, Nitroglycerin, Morphine and Chewable Aspirin on Arrival • Pregnancy test for female abdominal pain patients • Anticoagulation for Acute Pulmonary Embolus Patients • Pediatric Weight documented in kilograms – AAP is the measure sponsor www.qualityforum.org Children’s Health Corporation of America Existing Measures Whole System Measures • ED Left Without Being Seen • ED Length of Stay www.chca.com Urgent Matters Robert Wood Johnson Foundations Urgent Matters Initiative Goals • Rigorously evaluate the implementation of strategies for improving patient flow and reducing ED crowding • Advance the development of standard performance measurement in the ED • Promote spread of promising practices to a wider audience / variety of hospitals www.urgentmatters.org Urgent Matters Robert Wood Johnson Foundations Performance Measures (NQF and CMS) • Throughput for admitted patients • Throughput for discharged patients • Time to pain management for long bone fractures – Admitted patients – Discharged patients • Median time to chest x-ray – Admitted patients – Discharged patients • Admit decision time to ED departure time www.urgentmatters.org Other Existing Measures and Measurement Organizations • Joint Commission – ORYX performance measures • Children’s Asthma Care measures (inpatient) • http://www.jointcommission.org/PerformanceMeasurement • AHRQ Pediatric Quality Indicators (PDIs) – 18 risk-adjusted measures – Obtained from inpatient administrative data – www.qualityindicators.ahrq.gov/pdi_overview.htm • Alliance for Pediatric Quality – AAP, American Board of Pediatrics, CHCA, NACHRI – www.kidsquality.org Other Existing Measures and Measurement Organizations ACEP National Report Card on the State of Emergency Medicine • Access to Emergency Care • Quality and Patient Safety Environment • Medical Liability Environment • Public Health and Injury Prevention • Disaster Preparedness http://www.emreportcard.org Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED Rationale for Framework Limitations of prior work – Single centers or geographic locales – Focus on condition-specific indicators – Preponderance of process-oriented measures – Benchmarks very focused on • Timeliness (through put) • Satisfaction (ceiling effect) – Lack of comprehensiveness regarding spectrum of ED care • Lindsay et. al., AEM, 2002 • Guttmann et. al., Pediatrics, 2006 Performance Measure Framework Quality indicator set development process • Adapted from AHRQ • “Defining Quality Performance Measures for Pediatric Emergency Care” – Funded by HRSA/EMSC Targeted Issues Grant H34MC08512 Performance Measure Development SOURCES Research Literature Measure by Candidate Measure Measures Evaluation Actual Use N = 60 for Selection Concept Selection Criteria Evaluation for Measure Set Application Adapted from AHRQ PDI development process Performance Measure Framework Measure development dimensions • IOM Quality Domains • Donabedian’s framework for quality • PEM disease frequency and severity Measure evaluation dimensions • National Quality Forum criteria Institute of Medicine Quality Domains Built around the core need for health care to be • Safe • Effective • Efficient • Timely • Patient-centered • Equitable Institute of Medicine Quality Domains Safe • Health care avoids injuries to patients from the care that is intended to help them Effective • Health care provides services based on scientific knowledge to all who could benefit, and refrains from providing services to those not likely to benefit Institute of Medicine Quality Domains Efficient • Health care avoids waste, including waste of equipment, supplies, ideas and energy Timely • Health care reduces waits and sometimes harmful delays for both those who receive and those who give care Institute of Medicine Quality Domains Patient - centered • Health care provides care that is respectful of and responsive to individual patient preferences, need and values, and ensures that patient values guide all clinical decisions Equitable • Health care provides care that does not vary because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status Donabedian’s Framework Structure • Indirect quality-of-care measures related to a physical setting and resources: Staff, space, supplies, equipment and financial resources Process • Measures evaluate the method or process by which care is delivered, including both technical and interpersonal components Outcome • Outcome elements describe valued results related to lengthening life, relieving pain, reducing disabilities and satisfying the consumer PEM Disease Frequency & Severity • Condition-specific – Proportion of patients with croup receiving corticosteroids • General – Proportion of patients returning to the ED within 72 hours of an initial ED visit • Cross-cutting – Proportion of patients with pain who receive an analgesic Choosing Condition-Specific Measures 30.00 www.pecarn.org/tools 25.00 24.59 20.00 19.22 % of Diagnosis 16.18 15.33 NHAMC S 15.00 PC DP 12.80 11.91 11.48 9.97 9.65 9.64 10.00 5.00 0.00 Trauma ENT, Dental & Gastrointestinal Respiratory Systemic States Mouth Diseases Diseases Diseases Major Group Alessandrini et.al., Academic Emerg Med; February 2010 Measure Evaluation Criteria Importance • The measure reflects a priority or high impact aspect of healthcare • The measure addresses outcomes or is strongly linked to improving outcomes • The measure addresses an area of considerable variation or poor performance across providers or population groups Measure Evaluation Criteria Scientific Acceptability • There is strong evidence for the specific measure focus, such as evidence based guidelines • The measure is reliable, reproducible and accurately represents quality of care Measure Evaluation Criteria Usability • The measure provides information that is actionable and can be used to make decisions that improve the quality of care • The measure is meaningful and understandable Measure Evaluation Criteria Feasibility • Data for the measure is generated during care delivery and is available in the EHR or other electronic sources • Data collection for the measure can be implemented • The information provided outweighs the costs/burdens of collecting the data Results: Performance Measure Distribution by IOM Quality Domain Applicability of Measures to IOM Domains (Measures can apply to more than one IOM domain) # of Measures Applicable to IOM 50 43 45 40 34 35 Domain 30 25 20 17 14 15 10 7 5 0 Effective Safe Patient Centered Efficient Timely Equitable – measures stratified by gender, age, race, ethnicity and payor Results: Performance Measure Distribution by Donabedian Framework Distribution of Measures by Donabedian Classification 40 37 35 30 # of Measures 25 20 15 13 10 10 5 0 Outcome Process Structure Measure Type Results: Performance Measure Distribution by Diagnosis Type Distribution of Measures by Category 30 27 25 20 # of Measures 20 15 13 10 5 0 General Cross-Cutting Condition-Specific Cross-cutting measures include pain/sedation, severe illness, diagnostic testing and medication management PEM Balanced Report Card • Measuring weight in kilograms for ED patients <18 years of age • All pediatric equipment present in the ED (per ACEP, AAP, ENA policy statement) • Reducing pain in children with acute fractures • Systemic corticosteroids in asthma patients with acute exacerbation • Medication error rates • Parent/caregiver understanding of ED discharge instructions • Door to provider • Total ED length of stay Further Considerations • Measures valuable to patients – Not minimum level of competency • Composite measures – Conceptual and analytic issues • Unit of analysis • Measure target • Locus of control • Balancing measures – Are we improving parts of our system at the expense of others? Measurement & Quality • Quality is central to achieving affordable care that knows patient needs and keeps them healthy • It’s a three step process – The first step toward achieving quality is convening expert members across the healthcare industry, including patients to define quality with uniform standards and measures that apply to the many facets of care patients receive. – Second, information gleaned from measuring performance is reported and analyzed to pinpoint where patient care falls short. – Third, caregivers examine information about the care they are providing and use it to improve. Measure. Report. Improve. References • AAP Policy Statement: Principles for the Development and Use of Quality Measures – Pediatrics 121 (2), February 2008, pp 411-418 • Pediatric Clinics of North America “Pediatric Quality”: Quality Measures in Pediatrics – Volume 56 (4), August 2009, pp 816-829 References • Institute of Medicine Report: Performance Measurement, Accelerating Improvement – December 2005 – www.iom.edu/Reports/2005/Performance- Measurement-Accelerating-Improvement.aspx • Joint Policy Statement—Guidelines for Care of Children in the Emergency Department – Pediatrics 2009;124:1233–1243 Steps in Measure Specification • Numerator statement • Denominator statement • Denominator exclusions • Data source and collection methods • Sampling • Risk adjustment • Stratification to detect disparities • Level of measurement / analysis Steps in Measure Specification Risk Adjustment • Accounts for patient-associated factors before comparing outcomes across settings • “Levels the playing field” • Would be unnecessary if patients were randomly assigned to treatments, settings etc.
Pages to are hidden for
"Performance Measurement in Pediatric Emergency Care"Please download to view full document