Tom Brower CSP & Brian Tink CRSP TapRooT® Summit 2009
An open discussion on efforts to improve the quality of investigations & reports
Do not have a ToR. Failure to use the tree early in the investigation. Snap chart does not contain all of the baseline evidence for the incident. Bias enters the snap chart – only facts and corrective actions not clearly tied to root causes Not tying causal factors and root causes and other significant learning's to industry, regulatory, and corporate policy.
Not gathering immediate statements. Early sequence causal factors missed. Effectiveness of corrective actions are not measureable (assess, review, enforce). People do not get to deeper, wide and high with generic causes.
Do you have a methodology for reviewing the quality of your investigation? How effective is it and what could improve it.
TapRoot Quality Control Checklist
Incident Description Employees involved in the incident are identified by a coding system such as EE1, EE2, OP1, or OP2 The incident description includes all of the information from the summer/fall SnapChart Can see causal factors and pick them out of the incident description. Incident descriptions provide enough detail to gain a mental picture/understanding of what happened. Report has been given to someone unfamiliar with the incident to see if they can understand what happened. Snap Chart distributed with the incident report. Reports reviewed for descriptors/deflectors that try to frame the issue (cognitive bias) - not present. Photos included if they help explain what occurred. Jargon in reports minimized or eliminated to facilitate understanding. Causal Factors Causal factors have been measured against the definition of a causal factor in the TapRoot book. The reader is able to identify the causal factors from the incident description. The team uses the "So what resulted because of this?" technique to identify causal factor. Identifying causal factors is repeatable by other using the techniques and Snap Chart for the incident. When you ask "so what resulted because of this?" the answer is the incident. Avoided picking causal factors to suit "pre-identified" root causes. This happens due to cognitive bias. Root Causes The root cause includes both the TapRoot code for the root cause and a detailed explanation as to why it is a root cause. All specific root causes considered for generic root causes using the Corrective Action Helper module in the TapRoot software. The root causes make sense based on the incident description, causal factors, and Snap Chart information.
Good
Review more
Corrective Actions
Corrective actions are developed based on the SMARTER criteria. SMARTER worksheets completed as part of the investigation/incident review process. Corrective action selection applies the Hierarchy of Controls, opting for higher-level controls. Corrective action status routinely reviewed and tracked through to closure. Corrective Action Helper reviewed as corrective actions developed.
Post Incident
Verification - is the process of ensuring that something was accomplished according to plan. For example, if training was to be conducted for the operators in the plant on Lockout/Tagout, verfification would mean ensuring that the session was held, the people delivering it were competent, the material was correct, and everyone scheduled attended. Validation - is the process of ensuring that the desired results have been obtained. If training was scheduled as a corrective action because people did not understand the Lockout/Tagout procedure, validation would mean that the problem of equipment not being locked out had been corrected. Employees when quized would know how to lock out, when to lock out and the actual practice in the field was that they were locking out. In other words, the root causes had been removed.
Put an X in the Excellent, Good, Bad, or Ugly column that applies. ONLY one X for each item to rate. Then rate the importance of the item on a scale from 10 to 1 in column H. The ratings provided were the average of a sample of TapRooT® users' opinions. Our measurement system. How we select targets to improve. Our INVESTIGATORS' knowledge of root cause analysis and performance improvement. Our COACHES' knowledge of root cause analysis and performance improvement. Our CERTIFIED INSTRUCTORS' knowledge of root cause analysis and performance improvement. Our GENERAL EMPLOYEES' knowledge of root cause analysis and performance improvement. Our MANAGERS' knowledge of root cause analysis and performance improvement. Our PROGRAM OWNER'S knowledge of root cause analysis and performance improvement. Our TOP MANAGER/SPONSOR'S knowledge of root cause analysis and performance improvement. Our use of trending. Our use of software to improve productivity and quality of investigations.
Excellent
Good
No Improvemen t Needed
Needs Minor Improvemen t
Bad
Ugly
Needs Improvemen ts
We Really Need to Work on This
How important is this Item (Scale of 10 to 1. 10 is most important) 8 5 9 10 10 3 8 10 9 6 6
Put an X in the Excellent, Good, Bad, or Ugly column that applies. ONLY one X for each item to rate. Then rate the importance of the item on a scale from 10 to 1 in column H. The ratings provided were the average of a sample of TapRooT® users' opinions. Our use of the TapRooT® tools (SnapCharT®, Root Cause Tree®, …) Our efficiency in doing investigations (saving investigation time). Our preparations to perform investigations. Our performance of interviews. Our efforts to go from reactive improvement to proactive improvement. Our ability to share lessons learned across the organization. The effectiveness of our corrective actions. Our ability to get corrective actions implemented quickly and correctly. Our system for checking that corrective actions are implemented and effective. Our presentations. Our communication from Users to Program Owner Our communication from Owner to Sponsor Our communication from Sponsor to Owner
Excellent
No Improveme nt Needed
Good
Needs Minor Improveme nt
Bad
Ugly
Needs Improveme nts
We Really Need to Work on This
How important is this Item (Scale of 10 to 1. 10 is most important) 8 6 6 6 8 5 8 8 6 5 5 9 9
Put an X in the Excellent, Good, Bad, or Ugly column that applies. ONLY one X for each item to rate. Then rate the importance of the item on a scale from 10 to 1 in column H. The ratings provided were the average of a sample of TapRooT® users' opinions.
Excellent
No Improveme nt Needed
Good
Needs Minor Improveme nt
Bad
Ugly
Needs Improveme nts
We Really Need to Work on This
How important is this Item (Scale of 10 to 1. 10 is most important)
Our communication from Program Owner to System Improvements
Our succession planning. Our rewards system.
1
8 9
Incident Number Report Title
Team Leader
Evaluation Date:
Report Areas Reviewed 1 2 3 4 Was the format of the report clear and easy to read? Was a Snap Chart used? Does the report include sketches and/or pictures? Were the appropriate causal factors identified?
Results Yes Yes Yes Yes No No No No
If No, include examples and ideas to improve.
5 6 Were the appropriate root causes identified? Is the report written in a clear, concise manner? List areas that were strong and any suggestions on how to improve. Yes Yes No No
7
Report Preparation – Does the report specify/contain: What organizational positions were interviewed? That the team inspected the incident scene? An inventory of documentation reviewed?
Yes Yes Yes
No No No
Comments 8 Do the recommendations effectively correct the root cause(s)? If No, include specific examples. Yes No
9
9 10
Was the corrective action helper used?
Was a SMARTER review conducted? Were any new Safeguards reviewed for effectiveness? I.e. was the hazard removed or a sign put up?
Yes
Yes Yes
No
No No
Evaluator Name & Company
Additional Comments:
What are the common quality problems your review process identifies?
What are the roadblocks that are preventing higher quality investigations? What plans have you used to remove the roadblocks and improve quality?
What is the audience in your company that sees the quality review results? Who asks?
5. Report and Close-Out Was there a written report?
Was the investigation of sufficient Detail?
Were all electronic and hard copy documents filed appropriately ?
Was a PowerPoint Presentation formatted in accordance with the standard template week four and corrective actions entered for tracking?
Was communication effective within the team throughout the investigation?
Tom Brower CSP & Brian Tink CRSP