Developing Measures of Pediatric Nursing Quality KEY WORDS quality, pediatric, infiltration, pain management, NDNQI ABSTRACT Recent research has highlighted the impact of nursing care on patient outcomes. The link to staffing has also been documented for selected conditions. To date, efforts to establish nurse sensitive measures have focused largely on adults. This article describes the project undertaken to select and pilot two pediatric indicators of nursing quality care to be included in an ongoing national benchmarking program.
1 TITLE Developing Measures of Pediatric Nursing Quality AUTHORS Susan R. Lacey, PhD, RN Assistant Professor of Nursing University of Alabama at Birmingham Birmingham, Alabama Telephone: (205) 966-5681 Fax: (205) 975-6142 firstname.lastname@example.org Susan F. Klaus, MSN, RN Research Instructor University of Kansas School of Nursing Kansas City, Kansas Telephone: (913) 588-1652 Fax: (913) 588-8737 email@example.com Janis B. Smith, RN, MSN Assistant Director, Systems Support Services Vanderbilt University Medical Center Nashville, Tennessee 37332 Telephone: (615) 936-1276 Fax: (615) 936-6785 Janis.firstname.lastname@example.org Karen S. Cox, PhD, RN Senior Vice President for Patient Care Services Children’s Mercy Hospitals and Clinics 2401 Gillham Road Kansas City, Missouri 64108 Telephone: (816) 234-3933 Fax: (816) 346-1333 email@example.com Nancy E. Dunton, PhD Research Associate Professor School of Nursing University of Kansas Medical Center 3901 Rainbow Blvd., Mail Stop 4043 Kansas City, Kansas 66160 Telephone: (913) 588-1456 Fax: (913) 588-4531 firstname.lastname@example.org
2 INTRODUCTION The Institute of Medicine’s third report in its Quality Chasm Series - Keeping Patients Safe: Transforming the Work Environment of Nurses– highlights the critical role of nurses in the health care system.1 Two prior reports – To Err is Human: Building a Safer Health System and Crossing the Quality Chasm: A New Health System for the 21 st Century – provided strong evidence of the need to accurately measure the outcomes of health care for patients and their families.2,3 The reports have served to focus a bright light on health care quality.
Keeping patients safe focuses on the work environment in which patient care is provided from the vantage point of the largest group of health care workers and a critical element in our health care system – nurses. Nurses are the health care providers who patients and their families are most likely to encounter, spend the greatest amount of time with and depend on for recovery when hospitalized, in a nursing home or managing care in their own homes. 1 Nursing actions have been shown to be directly related to patient care quality and outcomes.
Recently, published research has drawn attention to the relationship between hospital nurse staffing and patient outcomes.4,5,6,7 A greater number of hours of care per day provided by registered nurses has been associated with decreased mortality, shorter length of stay and decreased complications among adult medical and surgical patients.4,5 Nurse staffing levels impact nurse sensitive patient outcomes such as urinary tract infections, shock, pneumonia, upper gastrointestinal bleeding and other adverse events.6 Similar studies with pediatric patients have not been conducted.
3 The Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) has worked closely with the National Association of Children’s Hospitals and Related Institutions (NACHRI), Child Health Corporation of America (CHCA) and others to establish performance measures for children’s healthcare.8 The collaborative has developed standardized performance measures for pediatric acute care settings and recently developed six core measures of quality care for pediatric patients with asthma. The standard performance measures for pediatric patients are intended to increase accountability in the care provided to children with asthma, while assisting children’s hospitals to accelerate their own quality improvement efforts. 8 While quality measures have been developed using a medical framework related to pediatric diagnoses, the purpose of this project is to develop and test quality measures that are specific to nursing care with pediatric patients.
The American Nurses’ Association (ANA) advances the nursing profession by fostering high standards of nursing practice, promoting the economic and general welfare of nurses in the workplace, projecting a positive and realistic view of nursing and lobbying the congress and regulatory agencies on health care issues affecting nurses and the public. ANA has been a leader in addressing the safety and quality of nursing care patients receive and the quality of nurses' work lives. ANA’s National Center for Nursing Quality (NCNQ) has focused on nursing care quality. It coordinates a number of projects focused on patient safety, nursing care quality, nurse safety, quality of work life and factors which impact these areas. The chief program of NCNQ is the National Database of Nursing Quality Indicators (NDNQI). The NDNQI provides a data repository for hospitals participating in the national database. Data are collected at the nursing unit level, a unique feature of the measures reported to the NDNQI. 9
The NDNQI was established in 1998 in response to ANA’s Safety and Quality Initiatives. Seven state nurses associations participated in pilot studies to demonstrate the feasibility of collecting comparison data of nursing care quality for adult patients on a national level. Since then, the NDNQI has been a leader in the collection and analysis of patient outcome data and the link with nursing hours worked. From its inception, the NDNQI has collected quarterly data on patient falls, pressure ulcers and staffing measures, including hours per patient day and skill mix. This work provides valuable benchmarking opportunities for participating sites, as quarterly comparative reports are generated by hospital size and unit type. The NDNQI also offers an annual Registered Nurse Job satisfaction survey to its members, in order for participating hospitals to monitor staff satisfaction at the unit level. 10
Currently more than 600 hospitals representing more than 5000 nursing units submit quarterly staffing and quality data. Hospitals submit data to the NDNQI using strict guidelines for data collection. In order to standardize data collection across sites, NDNQI has created educational programs for site coordinators and data entry staff which include on-line training and written resource guidelines. Research staff at NDNQI is available for consultation via email or telephone. The Magnet Hospital Recognition ProgramTM, a “seal of approval for institutions that have superb nursing quality” has strongly encouraged their participating sites to join NDNQI in order to demonstrate their on-going commitment to excellence in nursing care.11,12
5 THE NEED FOR PEDIATRIC NURSING CARE QUALITY INDICATORS Children’s hospitals and pediatric units seeking to participate in the NDNQI nursing care quality project found measuring outcomes on patient falls and pressure ulcers, the indicators of nursing care quality for adult patients, unsuitable for the population of patients cared for in their settings. In order to respond to the need for quality indicators of pediatric nursing care, the NDNQI sought to develop and test credible indicators of pediatric nursing quality. This paper will outline the processes involved in this work. Specifically the following will be explained: The project goal and criteria for indicator consideration and inclusion, The literature reviewed and subsequent discussions with subject matter experts, The creation of potential indicators, Pilot testing and the evaluation process, Subsequent modifications of the indicators and National implementation and future directions.
PROJECT GOAL AND CRITERIA FOR INCLUSION The goal of this project was to create at least two indicators of pediatric nursing care quality for pilot testing by the NDNQI. The overarching criterion for consideration and inclusion of any indicator is this: the presence or absence of the registered nurse impacts the outcome for patients with regard to the indicator.13 That is, the indicator is a sensitive measure of pediatric nursing care quality. Supplemental criteria are as follows: The indicators recommended must be applicable to pediatric patients in both general and intensive care units. The indicators must be scalable.
6 Data collection personnel at each participating site must be able to collect the data without undue burden. Data collection must be feasible. The recommended indicators must be measured accurately and consistently. They must be valid and reliable measures within and across participating sites.
Development process A two-pronged approach was taken in the development of possible pediatric nursing care quality indicators. First, a literature review was conducted. Second, discussions with experts in the field of pediatric nursing care and quality measurement were convened. These two activities were conducted in concert with one another.
Literature review. The literature review was conducted using the following strategy. First, general pediatric quality literature was reviewed. Second, a more specific review of the literature narrowed the findings to empirical evidence associated with pediatric nursing care and quality outcomes. In both cases the focus of the review was the relationship between nursing care and the potential indicator. Table 1 lists the patient care quality indicators that were considered.
Interviews and discussion groups. Key experts in pediatric nursing care reviewed the initial list of potential quality indicators. The experts included a nurse scientist who has conducted three decades of research with pediatric patients; a clinical nurse scientist who actively conducts bedside research at a leading children’s hospital; a nurse expert in clinical data collection; and clinical and administrative nursing leaders at two children’s hospitals in the process of applying for Magnet designation. Since the time of their participation in the discussion groups, both sites
7 have successfully achieved Magnet status. The experts were asked to provide feedback about the sensitivity, scalability, feasibility, validity and reliability of the potential indicators.
Sensitivity: First, is the potential indicator a sensitive measure of the impact of nursing care for pediatric patients? It was imperative indicators be as sensitive a measure of nursing care as possible, given the multidisciplinary nature of the patient care environment. For instance, medication administration accuracy is important for high quality patient care, but the processes which ensure accuracy are not exclusively within the domain of nursing practice. Physicians, pharmacists, as well as nurses, are each responsible for parts of the process that leads to either the safe administration of a medication or a medication error. Another way of thinking about indicator sensitivity is to determine which indicators are most exclusively “owned” by nurses in the delivery of patient care.14
Scalability: Second, is the potential indicator scalable across all types of pediatric units? It was important to select credible indicators that reflected a broad range of pediatric patients regardless of unit or hospital size.
Feasibility: Third, is data collection about the indicator feasible? Participating sites must be able to collect the data without excessive burden. Data must be available from existing sources, such as the electronic or paper medical record or a quality improvement database, or collected in real time by qualified staff.
8 Validity and Reliability: Finally, do the selected indicators accurately measure what each is intended to measure consistently and over time at each site?
PROPOSED INDICATORS OF PEDIATRIC NURSING CARE QUALITY Table 1 lists the six indicators strongly considered as measures of pediatric nursing care quality. The table provides a definition and a short summary of the sensitivity, scalability, feasibility, validity and reliability of each.
Two indicators were recommended for pilot testing. They are peripheral intravenous (PIV) infiltration and the pain assessment/intervention/reassessment cycle. Both indicators were judged to be sensitive measures of professional nursing practice. Maintenance of a patient’s IV is a clear nursing responsibility. Nurses hold a key position in successful pain management. While nurses impact the outcomes of the rejected indicators, the influence of other care providers is indistinguishable.
Pediatric patients are at increased risk for both PIV infiltration and for significant complications of the event. The purpose of measuring PIV infiltration is to determine the prevalence and severity of infiltration in pediatric and neonatal patients and to explore the relationship between PIV infiltration and nursing hours worked. PIV infiltration is graded based on the definitions provided by the Infusion Nursing Society.15
Children remain under-medicated for pain despite the known link between adequate pain management and optimal healing.16 The purpose of measuring the pain
9 assessment/intervention/reassessment cycle is to determine the frequency and completeness of these activities and to explore the relationship between the documented cycle and nursing hours worked.
Following the selection of the two indictors, an initial draft of data collection guidelines and tools was developed by the investigators and key staff at NDNQI. The data final collection tools are presented in Tables 2, 3.
Peripheral intravenous infiltration indicator. The guidelines for data collection instructed each unit to conduct a one-day prevalence study to assess every IV currently being used to administer fluids or medications. Central lines, “locked” IV catheters not currently in use and peripherally inserted central and midline venous catheters were excluded. In order to maximize the likelihood of appropriate sample sizes, the recommended time frame for data collection is monthly.
Pain assessment/intervention/reassessment indicator. The guidelines for data collection directed each site to perform a once quarterly 24 hour chart review to determine the prevalence of completed pain assessment/intervention/reassessment cycles. The data collectors were instructed to evaluate specific pain assessment for evidence of a subsequent intervention and reassessment over the most recent 24 hour period.
INDICATOR EVALUATION AND PILOT TESTING Feedback on the data collection tools. Copies of the guidelines and tools were posted on the NDNQI Website Bulletin Board. The Bulletin Board is a members-only message board for site coordinators and other staff from NDNQI hospitals. Notice was sent to all NDNQI site coordinators requesting review and comment on the draft documents. Bulletin Board participants were asked to focus their comments on specific aspects of the proposed nursing care quality indicators including: the clinical importance of the outcomes to be measured and the sensitivity of each to nursing care, feasibility of data collection, including time considerations, availability of data elements, and personnel needs, clarity of the guidelines and tools, likelihood of pediatric nursing units participation in the measuring the proposed nursing care quality indicators, and willingness to serve as a pilot test site.
Comments were collected on the website for six weeks and then were summarized by the NDNQI staff and investigators. The response to the indicators was positive, with most facilities reporting they monitored peripheral intravenous infiltrations or the pain assessment/intervention/reassessment cycle in some format. In response to feedback received on the Bulletin Board, the instruments and guidelines were modified to address the suggestions of members.
Pilot testing. Pilot sites for testing the pediatric nursing care quality indicators were recruited from within and outside the NDNQI membership. Member hospitals expressing interest in pilot testing were contacted and the final sample was selected. In addition to the volunteer pilot sites, all of which were large hospitals, two small NDNQI member hospitals were purposefully recruited to test the feasibility of the indicators in pediatric units within smaller general hospitals. Finally, two non-member pilot hospitals were selected from institutions that had expressed interest in serving as pilot sites. Ten acute care hospitals agreed to participate, with a total of 26 units willing to test at least one of the two indicators. Table 4 summarizes the unit and hospital characteristics of the ten sites.
Each pilot site was sent the data collection guidelines, the data entry tools and was asked to complete data collection within two weeks. NDNQI staff was available for consultation during the pilot if questions arose.
Evaluation of pilot testing. When data collection was completed, site coordinators and key staff on each pilot unit were asked to complete a written evaluation. The evaluation requested information about the clarity of the guidelines and tools and posed specific questions about the process from patient identification through data collection. Demographic data about each unit was also collected and the survey provided a mechanism for additional comments not addressed with specific questions. For each indicator, the respondents were asked to report how many minutes per chart or assessment were required to collect the data.
12 Select respondents were contacted by phone for interviews by NDNQI staff. The phone interviews provided an opportunity to clarify the written comments on the survey. The process served to enhance guideline clarity and improve data collection feasibility. Finally, three telephone conference calls between pilot site coordinators, NDNQI staff and other member sites were held. Participants from NDNQI member sites included quality improvement staff and pediatric experts. Personnel from 65 sites participated in the teleconferences. The calls provided the opportunity to make final suggestions to improve the data collection tools and guidelines.
Modifications. Several changes were made to the data collection tool for the pain assessment/intervention/reassessment cycle indicator. For example, the initial instructions on the pain management cycle required recording all assessments in the previous 24 hour period. The burden of data collection for intensive care units, where pain was assessed more frequently than in general pediatric units, was high. Data collection procedures were modified in consideration of the timing of nursing documentation. Elements of the cycle could be missing because of delays in documentation, rather than a lapse in compliance. Subsequently, the tools were modified to collect data on the first two cycles of pain assessment/ intervention/reassessment starting at the beginning of the 24 hour period. For example, if data collection begins at 9am Tuesday, the 24 hour study period starts at 9am on Monday and the first two pain cycles that were initiated after 9am Monday would be evaluated. The patient care record is reviewed, and each element of the cycle addressed and reported. In addition, an option to indicate that the patient was asleep was added.
13 For the peripheral IV indicator, the level of scrutiny required to adequately assess the IV site was reinforced. That is, the site is to be examined thoroughly without the presence of an opaque dressing or wrap. The guideline for IV site assessment was modified to more clearly exclude central and midline venous catheters.
NATIONAL IMPLEMENTATION When final revisions were made to the data collection tools and guidelines, revised versions were sent to all NDNQI sites. All participating sites have been invited to implement the indicators of pediatric nursing care quality. Data collected for the two pediatric nursing care quality indicators will be submitted via the internet on the secure NDNQI website. Table 5 provides a description of unit participation for the two indicators in the initial quarter of data collection, fourth quarter 2004.
DISCUSSION At the start, development of indicators of pediatric nursing care quality seemed a straightforward process. In fact, the process has been anything but straightforward. The literature review amply demonstrated the scarcity of empirical evidence of nurses’ impact on specific pediatric patient outcomes. The interviews with experts and focus group discussions revealed diversity of thought about potential indicators and nurses’ direct impact on those outcomes. There seemed to be two schools of thought about the proposed indicators of pediatric nursing care quality. The two camps were illustrative of the age-old differences between the theoretical and the practical or the ideal and the real. One group asserted that nursing should be far beyond evaluating IV sites and staffing levels, recommending complex indicators and sophisticated measurement. The other
14 group stressed it was necessary to keep it simple. Our conclusion is that the simplicity these two indicators offer is necessary to begin the work to establish the impact of nursing care on patient outcomes. The rigor of the process for data collection offers measurement sophistication.
The accepted indicators of pediatric nursing care quality reflect the state of the science linking nursing care with patient outcomes. Pediatric nursing has only begun to move from traditionbased practices toward a more scientific approach. This position is not unique to pediatric nursing or to nursing in general, but includes most health care professions. Accrediting and benchmarking organizations, fiscal constraints and consumer expectations have resulted in a shift from tradition-based care toward evidenced-based practices and accountability. The results, though early, support the development of practice environments and the necessary infrastructures to scientifically evaluate daily clinical practices. Institutions are likely to continue to struggle to meet these new expectations, due to both inadequate resources and longstanding resistance to change. However, as the culture of the healthcare industry moves to focus on quality, these sites may begin to transcend longstanding barriers. ACKNOWLEDGMENT Funding for this work was provided by the American Nurses Association. The authors acknowledge the contributions of the pediatric nursing professionals and thank all of the experts and hospitals that participated in the development of these very important indicators of pediatric nursing quality. This article, however, solely represents the opinion/viewpoint of the authors.
15 REFERENCES 1. Institute of Medicine Committee on Quality of Health Care in America: Keeping patients safe transforming the work environment of nurses. Washington, DC: National Academy Press; 2004:53-64. 2. Kohn LT, Corrigan JM, Donalson MS, eds. Institute of Medicine: To err is human: building a safer health system. Washington, DC: National Academy Press; 1999. 3. Institute of Medicine Committee on Quality of Heath Care in America. Crossing the quality chasm: a new health system for the 21st Century. Washington, DC: National Academy Press; 2001. 4. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J Am Med Assoc. 2002;288:1987-1993. 5. Kovner C, Jones C, Zhan C, Gergen PJ, Basu J. Nurse staffing and postsurgical adverse events: an analysis of administrative data from a sample of U.S. hospitals, 1990-1996. Health Services Researcher. 2002;37:611-629. 6. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. May 30, 2002;346(22):1715-22. 7. Dunton N, Gajewski B, Taunton R, Moore J. Nurse staffing and patient falls on acute care hospital units. Nurs Outlook. 2004;52:53-59. 8. Joint Commission on Accreditation of Healthcare Organization. National pediatric practice and measures: focus on asthma. Executive briefing. Chicago, ILL. 2002.
16 9. Nursing facts: nursing-sensitive quality indicators for acute care settings and ANA’s safety & quality initiative. American Nurses Association Web site. Available at: http://nursingworld.org/readroom/fssafe99.htm. Accessed April 4, 2005. 10. National Database of Nursing Quality Indicators. Available from: http://www.NursingQuality.org. Accessed April 19, 2005. 11. Aiken LH, Havens DS, Sloane DM. “The Magnet Nursing Services Recognition Program: A Comparison of Two Groups of Magnet Hospitals.” American Journal of Nursing. 2000;100(3):26–35. 12. American Nurses Credentialing Center . The Magnet Recognition Program Application Manual 2004. Silver Spring, MD. 2004. 13. American Nurses Association. Nursing report card for acute care. American Nurses Publishing. Washington, DC. 1995. 14. Vollman K. Keynote Address. University of Alabama at Birmingham, 2004 Cardiac Conference. October 14-15, 2004. 15. Infusion Nursing Society. Infusion nursing standards of practice, J. Intraven Nurs. 2000; S57-S58. 16. Howard RF. Current status of pain management in children, JAMA. November 2003;290(18):2464-2469. 17. Comprehensive Accreditation Manuals for Hospitals: The Official Handbook Joint Commission on Accreditation of Healthcare Organizations. Oakbrook Terrace, IL: Joint Commission Resources; 2004:PC-23. 18. Velasco-Whetsell M, Coffin D, Lizardo, L, et al. Pediatric Nursing. New York, NY: McGraw Hill Health Profession Division; 2000:143-146
17 19. Lamagne P, McPhee M. Troubleshooting pediatric peripheral IVs: phlebitis and infiltration. Nurs Spec. July 2004. 20. Terry J. Intravenous Therapy Clinical Principles and Practice. Philadelphia, PA: W. B. Sanders; Company, A Division of Harcourt Brace & Company; 1995: 143-148, 151-156, 379-383, 392-395.
18 Table 1: Potential Pediatric Nursing Care Quality Indicators
Indicator Failure to rescue Definition The caregiver’s inability to save a patient’s life when a complication occurs. Scalability, Validity, Reliability The acuity of today’s Accurate and inpatient population consistent puts every patient at measurement of risk for an untoward “near misses” likely event. to be difficult across diverse sites. Nosocomial infection in vulnerable patient populations is on the increase, as is antibiotic resistance. Valid and reliable data are likely available from hospital infection control statistics. Significance Sensitivity to Nursing Link to nurse hours worked has been empirically established with adult patients.6 REJECTED Link to nurse hours worked has been empirically established with adult patients. Nurses are not the only potential source of hospital acquired infection. 6 REJECTED A link to nursing is likely, however pulmonary management is multidisciplinary. REJECTED
Hospital acquired infection.
The accidental dislodgement of an endotracheal tube from the airway.
Risk of harm is high with this untoward event.
May not be reported as an “incident” in every setting making accurate and consistent measurement difficult.
19 Table 1 continued: Potential Pediatric Nursing Care Quality Indicators
Indicator Medication errors Definition An error in the prescription, dispensing, or administration of a medication. Significance Every patient is at risk for a significant medication error. Scalability, Validity, Reliability Accurate measurement problems are well described. 1 Sensitivity to Nursing Nurses are just one link in a chain of staff that may each be responsible for a medication error. REJECTED Nurses’ key position in successful pain management is well documented. 18 ACCEPTED
The pain assessment, intervention and reassessment cycle.
Children remain undermedicated for pain despite the known link between adequate pain management and optimal healing. 16 The JCAHO has made pain management a clear requirement for hospital accreditation. 17 Pediatric patients are at increased risk for both PIV infiltration and for significant complications of the event. 19
Valid and reliable data should be available from patients’ medical records.
Peripheral intravenous infiltration
The unplanned administration of a vesicant or nonvesicant solution or medication into surrounding tissue.
Valid and reliable data should be available from patients’ medical records, from occurrence reports, or could be collected in a monthly surveillance study. Generalizable to all hospitalized infants and children.
Maintenance of the patient’s IV is a nursing responsibility. 18 or 20 ACCEPTED
20 Table 2: Pain Assessment/Intervention/Reassessment Cycle Report (See enclosed file)
21 Table 3: Monthly Peripheral Intravenous (PIV) Infiltration Report (See enclosed file)
22 Table 4: Pilot Test Sites for the Selected Pediatric Nursing Care Quality Indicators
Pediatric Nursing Care Quality Indicator Pilot Test Sites Beds n=2 100-299 n=3 300-499 >500
Type General Teaching General Nonteaching Children's Specialty n=7
Status Not-For-Profit n=9
Number of units General Care n=13
Pediatric Intensive Care Neonatal Intensive Care
Table 5: Pediatric Nursing Care Quality Indicators-Participating Units Q4, 2004 Indicator Teaching Status Unit Type Step down, Medical, Surgical, & Med/Surgical combined Pediatric Intensive Care Neonatal Intensive Care Step down, Medical, Surgical, & Med/Surgical combined Pediatric Intensive Care Neonatal Intensive Care
Pain AIR Cycle* N=91
Academic Medical Center Teaching Non-teaching
n=49 n=26 n=16
n=46 n=21 n=24
PIV Infiltration^ N=68
Academic Medical Center Teaching Non-teaching
n=33 n=22 n=13
n=36 n=16 n=16
*Total patients=845 ^Total patients=1059, with a total of 1101 sites assessed AIR – Assessment Intervention and Reassessment PIV – Peripheral Intravenous