CREATING AND MAINTAINING A REPORTING CULTURE Michelle L. Harper Robert L. Helmreich University of Texas at Austin This paper presents the theoretical foundation of the concept of reporting culture. The first section of this paper presents theoretical argument defining reporting culture as a critical component of an organization’s ability to create and maintain a proactive approach to safety. Second, a brief description of event reporting in two high risk industries is presented. And, third this paper outlines the parameters of a study to identify the main factors that define an organization’s reporting culture, including proposed methodology for the development of a reporting climate survey and analysis strategies for longitudinal assessment of reporting culture. Introduction Aviation Safety Reporting System (ASRS 1985), and The use of event reporting systems in the safety oriented more recently has been further encouraged by the professions has steadily increased in the past 10 years. Aviation Safety Action Program (ASAP). Other Prompted by a shift to develop proactive approaches to voluntary reporting programs that require informed safety, there has been a huge increase in the number of consent but not manual input of from pilots have also event reporting programs that have sprung in become prominent contributors to this proactive safety organizations that must manage human error. As a result approach. These programs include Flight Operational employees or operators in these industries have found Quality Assurance (FOQA) and the University of Texas themselves being prompted to participate by submitting at Austin Line Observation Safety Audits (LOSA). reports of their own errors. (Note the term operator will be used throughout this paper to refer to an employee or One of the more ambitious yet increasingly popular an individual in either the medical or aviation proactive safety programs is ASAP. This program is industries). In an attempt to further clarify what enables based on the direct voluntary reporting of events by an organization to maintain a reliable and stable event pilots and the airline’s endorsement of non-jeopardy reporting program the theoretical argument and research principles that govern the review of these reports. This methods for assessing attitudes, beliefs and behaviors program is unique in that it is based on a joint surrounding the use of event reporting systems are memorandum agreed on by representative members of discussed. An overview of the current state of event the FAA, the Airline Pilots Association (ALPA) and the reporting in both the medical and aviation industries as participating airline. The ASAP requirement that a joint well as a brief description of where the concept of agreement be met by all three representatives serves as reporting culture lies within larger more commonly an incentive to pilots to submit reports because once referred to cultures is presented. accepted into the program individual disciplinary action by the FAA or the participating carrier is prohibited. As of February 2003, 25 airlines have submitted proposals Review of Event Reporting in Aviation to the FAA to start an ASAP. On average, those airlines that have established programs receive between 3 to 12 There has been a cultural change in the last 15 years in reports per day. Although this program is fairly new to aviation safety programs that has resulted in an increase the industry (American Airlines began an ASAP pilot in the availability of previously unknown information. program in 1996), the increase in carriers starting ASAP The focus of this shift has been to collect and act on suggests the number of reports collected annually may voluntary, protected safety information before it leads to match or exceed the 30,000+ reports received annually a consequential or adverse events. This proactive by ASRS. approach has strengthened by the development and endorsement of non-jeopardy reporting programs. A reporting system is considered non-jeopardy if it Review of Event Reporting in Medicine provides reporters protection from disciplinary action or any negative repercussion that would normally occur. Spurred by a controversial report of the prominence of This protection is often provided through offering medical error (IOM, 1999) the medical industry has seen reporters anonymity or through the de-identification of a similar increase in the number of event reporting reports. This shift started with the development of the systems both at the national and institutional level. As of April 2000, 15 states registered as having mandatory Mallon 1999, Van der Schaff 1991). These findings systems for reporting adverse events, with 6 more states support the assumption that reporting systems can awaiting legislation to start mandatory systems (National provide the means for identifying potential hazards Academy for State Health Policy, 2000). With similar before they become consequential. With the knowledge goals in mind of reducing human error at the institutional that information to proactively address safety is available level, the Joint Commission on Accreditation of the next problem to solve is how to ensure this Healthcare Organizations (JCAHO) introduced the information is reported. requirement that each accredited organization establish “a way for care providers to report adverse events as well as near misses” (O’Leary, 2000). And more Components of Successful Reporting System recently the American College of Physicians called for congressional support of a uniform, voluntary, and non- Mandatory vs. Voluntary Systems punitive reporting program (Roscoe & Krizek, 2002). Regardless of the industry the success of a reporting Following the release of the Institute of Medicine’s system is based on the voluntary reporting of operators. report on the prominence of human error in the medical Even mandatory reporting programs, like Captains industry (IOM, 1999), the National Academy of Irregularity Report Program required by most Sciences who authored the report suggested that health commercial airlines or the nurse reporting requirements care institutions must “create a culture of safety” and depicted in the State of Texas’s Nurse Practice Act, are that it is critical that this culture support the still based on the operator’s willingness to report. identification and prevention of error. Yet this report Findings from a study conducted in both the US and with also suggested that the creation of a safety culture should the British National Health System (NHS) suggests that not be based on attaching blame to those involved but under reporting of adverse events may be as high as 96% rather focus on the identification of root causes (IOM, (Stanhope 1999, Barach & Small, 2000). These findings 1999). suggest that despite the fact that hospitals in both the US and NHS mandate the reporting of adverse events, the use of reporting systems remains a choice of the Objectives of Event Reporting Systems operator. The data collected by event reporting systems in both the Reporter Protection aviation and medical industry varies, but the goals of One of the largest differences found between the aviation event reporting remains the same. These goals can be and medical reporting systems is the level of protection divided into two objectives; that each industry offers to operators who submit reports 1) Collect safety information that would not of potentially incriminating events. One type of have been known through conventional protection commonly characterizing reporting systems is means and provide protection to the reporter. anonymity. The fields of aviation and medicine have 2) Use this information to develop corrective dealt with anonymity issues differently. Aviation actions to reduce the potential for reporting systems, in particular ASAP, is currently reoccurrence of accidents, incidents or safety- experiencing the benefits from the many years of related problems. growing pains experienced by the long established ASRS program. One of the greatest benefits stemming One of the reasons these industries have chosen to invest from ASRS is the influence that it has had on pilot’s resources into the development of event reporting is the level of trust with reporting personal error and commonly held assumption that the operator is privy to a vulnerabilities. large amount of information that is not discoverable by conventional safety programs. Initial findings from On the other hand the medical industry has not benefited ASAP reports collected through several airlines suggests from such shifts, in fact the safety culture of this industry that between 85 to 90% of reports submitted by pilots is commonly referred to as heavily punitive and blame would not have been discoverable by any other means. oriented (Leape & Berrick 2000, Cohen 2000). A recent This statistic suggests that even in a heavily studied, study of physicians in the state of Colorado found that safety conscious industry, the operator is solely aware of although most agreed that there is a huge need to a tremendous amount of untapped safety relevant develop a reliable means for reporting medical error, a information. Further evidence of the unique perspective large percentage stated reluctance to do so without legal that these programs may add to a safety database is the protection (Robinson et al. 2002). finding that close calls or near misses are 3 to 300 times more likely to occur before an adverse event (Battles, From studies of event reporting in both medicine and Kaplan, Van der Schaff & Shea 1998, Gambino & aviation it can be assumed that the success of reporting systems is dependent on the continued, voluntary critical component of a strong safety culture. Further participation of operators and the commitment to protect evidence of the existence of reporting culture adversely the identity of those willing to report or submit affecting the use of reporting systems has been discussed information on potentially incriminating events. by the British National Health System (Vass, 2002) and a series of studies completed in obstetric units in the Reporting culture: Under the umbrella of safety culture United Kingdom (Vincent et al 1999, Stanope et al. 1998). Although clarity surrounding the definition of organizational culture is an ongoing source of debate With this information in mind the following definition of (Glendon & Stanton, 2000 for a review), it is readily reporting culture is suggested. Reporting culture is a maintained throughout management and safety research component of safety culture that is characterized by the that the concept of safety culture exists under the beliefs and attitudes that operators hold towards the umbrella of an organization’s defining values, beliefs, reporting of their own errors and the behaviors that attitudes and behaviors or it’s organizational culture. The characterize the use of reporting systems designed to definition of safety culture is also a source of debate. submit this information to management. Therefore one This debate centers around the premise that traditionally basic assumption of this study is that reporting culture an organization’s culture has been considered a stable not only lies under the umbrella of safety culture but is concept and relatively impermeable to change. Yet, the characterized by beliefs, attitudes and behaviors held commonly held definition of safety culture suggests that towards reporting systems and their use. With this it is a much more dynamic concept and therefore more assumption in mind the following study was developed likely to be susceptible to change. to identify the specific attitudes and beliefs that operators hold towards the use of reporting systems. Adding support to a dynamic definition of safety culture is research completed on high reliability organizations or organizations that have been shown to maintain high Methods levels of safety in complex and technically advanced environments. Case studies and direct comparisons of The proposed study suggests that the following methods these organizations do not report a static or stable will identify an organization’s reporting culture and approach to addressing operator error but instead have assess whether or not the implementation of a reporting been recorded as having a flexible and proactive system will have an impact on traditionally held attitudes approach to safety (Rochlin 1999, La Porte 1996). The towards event reporting. Although survey research is idea of safety culture as a dynamic concept is commonly assumed to assess climate of an organization exemplified in these organizations in that the culture is this study will look at a range of parameters to define often seen as being driven by striving towards specific reporting culture and not simply the results of a single but changeable safety goals. The values held by these survey at a single point in time. The methodology used organizations present evidence for a possible shift in the will encompass the measurement of basic assumptions or traditional views of culture in that they present a strong norms held towards event reporting as well as specific culture that is susceptible to change. attitudes towards the establishment of a new reporting system and the behaviors surrounding the systems use. Defining Reporting Culture Populations to be assessed The reporting climate survey and the behavioral If safety culture is developed through the operators assessments proposed by this study will be completed at attitudes towards and interaction with a safety program two participating medical institutions in the University than it should follow that a set of specific attitudes, of Texas Medical Branch and at two participating beliefs and behaviors should exist surrounding various airlines who maintain both domestic and international types of safety programs. The existence of a sub-culture operations. These four participating organizations are in that both supports and shapes an organization’s safety the process of implementing event reporting systems and culture has been consistently suggested by investigators will be surveyed pre and post implementation of the assessing the success and failure of reporting systems in systems. Behavioral assessments recording the use and high risk industries. Reason (1999) supports the type of errors reported will be completed 3 to 5 months existence of a sub-culture of safety culture pertaining following the implementation of the reporting system. directly to attitudes held towards the reporting of personal error. He further suggests that the larger umbrella cultures are dependent on sub-cultures and goes as far as suggesting “a reporting culture” to be a Assessing reporting culture commitment to safety as the strongest factor found in an employee safety survey. It is therefore hypothesized that The survey was designed to assess reporting climate pre degree and level at which action is taken by and post the development and implementation of a management, be it Quality Assurance, Risk Management reporting system. The survey items were developed or Unit Managers within the medical field or Flight through a series of structured interviews with managers Operations, the Safety Department or Fleet Managers in and operators in both industries. Through the use of aviation, will increase positive perceptions of an confirmatory factor analysis it is hypothesized that the operators ability to make changes and impact reporting following factors will be identified through the rates. administration of the Reporting Culture Survey: Ease of use. Ability to create change The next factor predicted is one that is composed of One primary assumption of this study is that at the questions regarding the ease with which operators can driving point of a reporting culture are the opinions that understand and use a reporting system. There has been a people hold towards their ability to make changes. It is large amount of research devoted to deriving stable therefore hypothesized that one factor of reporting definitions of error in both the aviation and medical culture is based the beliefs that an individual maintains industries. For example, there is currently a debate on their ability to create change. The questions predicted ensuing with the participating members of the rule to load on this factor include assessments of attitudes making committee in charge of overseeing ASAP policy. towards usefulness of the systems and individual’s At the center of this debate is the lack of agreement over experience with making changes in the past. At the the definition of “intentional disregard for safety”. The longitudinal level, it is hypothesized that the degree to medical industry is also struggling with issues of which a person sees a reporting system as a viable place definitions, although these arguments have centered on to create change will be a strong determinant of the definitions of the level of severity of an error. Due to the organization’s reporting rate. lower levels of standardization of this industry in comparison to aviation, definitions and opinions of what Anonymity issues: Fear of litigation and disciplinary constitutes a close call tend to vary according to action institution, unit, and at times profession of the care Two basic conclusions drawn from the study of high provider being asked. These clarity of these definitions reliability organizations is that these organizations tend are critical to the development of a reporting system to reward the reporting of error and that root causes of because they determine what types of events will be error tend to lie with the organization and not the accepted into a reporting program and what operators individual (Weick, 1987). It is hypothesized that this will be told is appropriate to report and what information factor will include questions assessing opinions of trust they will expect to be protected. and issues of disciplinary action, finger pointing and discoverability. The prediction of this factor is based on Personal responsibility to address problems. the assumption that prior to using a reporting system an The final factor predicted is one that loads on questions operator will make an assessment of a risk/benefit ratio. of personal responsibility. These questions address If he/she believes their identity can not or will not be issues regarding both the opinions towards moral and protected or that their report of a safety problem will professional responsibility surrounding the reporting of affect their relationships with other co-workers then even errors. If is further hypothesized that operators with large levels of motivation stemming from possible stronger opinions of personal responsibility will be more benefits of creating a change are at risk of being negated. likely to use a reporting system. Therefore it is also predicted that those reporting systems that offer higher levels of protection will benefit from Behavioral assessments higher reporting rates. This study hypothesizes that degree and level at which Attitudes held towards current reporting systems action is taken by management, be it Quality Assurance, The third factor that is predicted includes questions Risk Management or Unit Managers with in the medical regarding attitudes and beliefs held towards management field or Flight Operations, the Safety Department or that currently run safety programs as well as attitudes Fleet Managers, will increase positive perceptions of an held towards those individuals that will manage the new operators ability to make changes and impact safe systems being developed. These questions address operations. One hypothesis is that manager’s verbal operator’s opinions of how errors were historically endorsement of a reporting system must be followed by handled both formally and informally. O’Toole (2002) actions and without tangible evident of this action the found that employee perceptions of management’s operator will loose motivation to submit reports. By establishing this connection between the verbalization of an organization to tap into the unique information that values and the behavioral endorsement, the current study exist between the operator and the environment, and if hopes to predict that changes in reporting culture can be designed with a reporting culture in mind a reporting driven through behavioral endorsement and this change system may be the most direct outlet through which an can be measured through a longitudinal assessment of employee can play a critical role in establishing a strong reporting climate. safety culture. Practical Applications of Assessing Reporting Culture: References Why is it important to identify an organization’s reporting culture? Barach, P. & Small, S. D. (2000). Reporting and preventing medical mishaps: lessons from non- One possible practical application of assessing reporting medical near miss reporting systems. British Medical culture may be found in the power that it lends to an Journal, 320, 759-763. organizations ability to address safety issues. What has been shown by research conducted in this area is that a Battles JB., Kaplan HS., Kaplan HS., Van der successful safety reporting system can provide a medium Schaaf TW., & Shea CE. (1998). The attributes of through which to identify potentially hazardous medical event-reporting systems: experience with a situations. It then becomes the responsibility of the prototype medical event-reporting system for transfusion organization management to act on this information and medicine. Archives of Pathology & Laboratory disseminate the changes throughout the organization. Medicine., 122, 231-238. The changes resulting from the identification and communication of this information should in turn Berwick, D. M. & Leape, L. L. (1999). promote continued reporting. Reducing errors in medicine: It's time to take this more seriously. British Medical Journal, 319, 136-137. The promotion of a safety change throughout an organization can be referred to as a safety change Cohen, M. R. (2000). Why error reporting process. Each organization is going to complete the systems should be voluntary. British Medical Journal, stages of a safety change process differently based on 320, 728-729. their safety culture. But, because a reporting system can be used to directly feed this process, it may also be Gambino, R. & Mallon, O. Near misses -- an capable of shaping an organization’s safety culture. untapped database to find root causes. Lab Report . 1991. This study hypothesizes that the development of a trusted reporting system will give the operator a tool to Glendon, A. I. & Stanton, N. A. (200). feed the safety change process. Furthermore it is Perspectives on safety culture. Safety Science, 34. suggested that the strength in reporting systems stems from using the operator as the expert in the organization Institute of Medicine. To Err is Human. 19991. and that the operator is not only a powerful determinant in compliance with safety practices but privy to O'Leary M & Chappell, S. (1996). Confidential information of how to best manage regularly occurring incident reporting systems create vital awareness of safety issues. As suggested by the review of the current safety problems. ICAO, 51. state of event reporting in aviation and medicine the development of a non-punitive reporting system not only O'Toole, M. (2003). The relationship between aids in the identification of the sources and locations of employees' perceptions of safety and organizational potential accidents but can also act as a tool to directly culture. Journal of Safety Research, 33. involve the operator in the making improvements to safety. Quality Interagency Coordination Task Force (QuIC) (2000). Doing What Counts for Patient Safety: As this study hypothesizes, it is believed that a strong Federal Actions to Reduce Medical Errors and Their safety culture can only be developed by continued Impact. support of the employee at the front line. Rochlin (1999) suggests that one of the critical components of a strong Reason, J. (2000). Human error: models and safety culture is free information flow and promotion of management. British Medical Journal, 320, 768-770. personal responsibility of all employees. This study hypothesizes that a strong reporting culture may enable Robinson, A., Hohmann, K., Rifkin, J., Topp, D., Gilroy, C., Pickard, J., & Anderson, R. (2002). Physician and Public Opinion on Quality of Health Care and the Problem of Medical Errors. Archives of Internal Medicine, 162. Rochlin, G. (1999). Safe operations as a social construct. Ergonomics, 41. Roscoe, L. & Krizek, T. Reporting medical errors: Variable in the system shape attitiudes toward reporting adverse events. Bulletin of the American College of Surgeons 87. 2002. Ref Type: Generic Stanhope, N., Crowley-Murphy, M., Vincent, C., O'Connor, A. M., & Taylor-Adams, S. E. (1999). An evaluation of adverse incident reporting. Journal of Evaluation in Clinical Practice, 5, 5-12. The National Academy for State Health Policy. State Reporting of Medical Errors and Adverse Events: Results of a 50-State Survey. 2000. Van der Schaaf TW. (1991). Near miss reporting as a safety tool. Oxford. Vass (2002). Patient safety agency admits problems with its pilots scheme. British Medical Journal, 324, 1437. Vincent, C., Stanhope, N., & Crowley-Murphy, M. (2002). Reasons for not reporting adverse incidents: an empirical study. Journal of Evaluation in Clinical Practice, 5, 13-21. Weick, C. (2001). Making Sense of the Organization. In ( Malden Massachusetts: Blakwell Publishers Ltd.