Docstoc

Lies_ Damned Lies_ and Statistics

Document Sample
Lies_ Damned Lies_ and Statistics Powered By Docstoc
					Lies, Damned Lies, and
       Statistics
                  Hanford Performance Indicator Forum
                          December 11, 2003


         Steven S Prevette
  Fluor Hanford Safety and Health

 509-373-9371
 Steven_s_prevette@rl.gov
 http://www.hanford.gov/safety/vpp/trend.htm
Introduction

• Numbers abound in the Safety profession
• We have all seen great uses and misuses

• How can we make the best use of the data
  we have and make effective decisions that
  will improve the safety of ourselves, our
  coworkers, and our companies?
Liars Figure and Figures Lie 1

          18
          16                                                Injuries by Month
          14
          12
          10
           8
           6
           4
           2
           0
               Jan-02




                                 Jul-02




                                                   Jan-03




                                                                      Jul-03




                                                                                        Jan-04
                        Apr-02




                                          Oct-02




                                                             Apr-03




                                                                               Oct-03
Here are the past 2 years of injury data.
 What is your interpretation? What is your
 prediction of future rates?
Liars Figure and Figures Lie 2

            18
            16                                                         Injuries by Month
            14
            12
            10
             8
             6
             4
             2
                            y = -0.132x + 11.987
             0
                 Jan-02




                                                     Jan-03




                                                                                         Jan-04
                                   Jul-02




                                                                       Jul-03
                          Apr-02



                                            Oct-02



                                                              Apr-03



Let’s add a ―trend line‖. What is your                                          Oct-03


  interpretation now? What is your
  prediction of future rates?
Liars Figure and Figures Lie 3

         18                                       Injuries by Month
         16
         14
         12
         10
         8
         6
         4
         2                 6 Month Moving Average
         0
              Jan-02




                                                  Jan-03




                                                                                      Jan-04
                                Jul-02




                                                                    Jul-03
                       Apr-02




                                         Oct-02




                                                           Apr-03




                                                                             Oct-03
Let’s add a 6 month moving average. What
  is your interpretation now? What is your
  prediction of future rates?
Interlude – Fun with Charts

• It appears I can make this data say
  anything I want it to say . . .
• Courses are taught on ―chartsmanship‖

• Can we interpret the data with a sound,
  rigorous method that is repeatable and
  gives credible results?
Liars Figure and Figures Lie 4
             25                                                                   Average = 10.3
                                                                                  (Jan02 - Jan04)
             20
                    Upper Control Limit
             15


             10


             5
                           Lower Control Limit
             0




                                                                Nov-02




                                                                                                                       Nov-03
                  Jan-02




                                                                         Jan-03




                                                                                                                                Jan-04
                                              Jul-02




                                                                                                     Jul-03
                                     May-02


                                                       Sep-02




                                                                                            May-03


                                                                                                              Sep-03
                            Mar-02




How about the Control Chart?                                                       Mar-03



- The data are stable, within certain bounds.
The Red Bead Experiment

• Many of you have experienced the ―Red
  Bead Experiment‖
• Results changed from person to person,
  but were random and predictable
• The data here were generated by the Red
  Beads
• Which chart gave the proper interpretation
  and predictions?
The Control Chart

• Separates random noise from signal
• Provides prediction capability
• When coupled with management theory
  and action can lead to significant
  improvements
• Does not require great costs to perform
• Actually has saved money
Management Theory For
Improvement

• When there is a trend, find ―special cause‖
  and act upon the trend

• When there is no trend, find ―common
  causes‖ and act upon the system, the
  overall performance
Pareto Charts
       OSHA Cases FY 1997
                                                                       Fluor Hanford
       OSHA Cases FY 1998 through May 1998
                                                              "Overall 25% Reduction to Date"
                         0.40


                         0.35


                         0.30


                         0.25


                         0.20


                         0.15     -48%                             -26%
                                                                                  -8%
                                                                                                       72%
                         0.10
                                                       -63%
                         0.05


                         0.00

                                   OR                  R        HP T              GE
                                                                                     R           IAN
                               R AT                TT E                        RIG            RIC
                            OPE             E FI                          E R/            ECT
                          S              PIP                            RK             EL
                    CES                                               WO
                 RO                                            IRO
                                                                  N
              ARP
          CLE
       NU




Pareto charts can be used to analyze
 common causes – FH ―Top 5 Occupation‖
 initiative from 1998
What is a Trend?

•   One point outside the control limits
•    Two out of Three points two standard deviations
    above/below average
•    Four out of Five points one standard deviation
    above/below average
•    Seven points in a row all above/below average
•    Ten out of Eleven points above/below average
•    Seven points all increasing/decreasing.

See http://www.hanford.gov/safety/vpp/trend.htm
Leading Indicators

•   This system can be used with leading indicators and
    Behavior Based Safety

•    A personal example – my group targeted computer
    ergonomics

•    Goal was to improve behaviors on the computer, to
    reduce risk of Carpal Tunnel / Overuse Syndrome
Leading Indicator Example
BEHAVIOR TO OBSERVE                                                                 YES   NO
Backrest of chair is supporting person's back
Forearms are parallel to floor
Knees at angle of 90 - 110 degrees
Legs supported by chair seat
Person's eyes are 18 - 30 inches from screen
Top of monitor at or below eye level
Monitor viewed straight ahead with no neck turn
Wrists are in proper position (Wrist Support in use, or wrists held at level they
    would be if there was a Wrist Support)


Typing stand (document holder) at same height and distance as screen [leave
    blank if no stand in use]
No glare of outside light, overhead lights, or work lights on screen.
Leading Indicator Results

• Initial rate was 10% safe behaviors
• Plotted on Control Chart
• Pareto chart of individual items
  established
• Safe behavior increased to 83% within 5
  months, a significant increase
Barriers to Good Data Use
• Higher ups will use it as a ―hammer‖

• Subjected to quotas and targets imposed from above

• Fear (―accountability‖) used as a ―motivator‖

• Actions and Explanations as a result of random fluctuations

• Perceived loss of control over portrayal of performance


              The Biggest Barrier = FEAR
Data Sources

• Worker and Customer Opinion
• Expert Review
• Process Measures
  – Top-Down (start with mission/vision)
  – Process Approach
  – Bottom-Up (start with available data)
Data Sources                                          THEORY




Dollars     Process
Hours                                                      OUTCOMES
Materials        Cycle Time                Product      Mission Progress
Data                                       Service      Commitments Met
                 In-Process                             Stewardship
Budget            Inventory                Schedule     Profit
                                                        Safety
                     Procedure                          Satisfaction
                                                        Compliance
            Rework
                       Idle Time



                                   Waste
More Fun with Numbers

•      Boston Herald (Dec 23, 1996):
  "Murders Sink To 30-Year Low "
• "....major crime such as homicide is down.
  Over the last two years there has been a
  52% reduction in shootings..... that's a
  dramatic difference....Police
  Commissioner Paul Evans has succeeded
  where others have fell short...."
More Fun with Numbers 2

                                                  Homicides per Year, Boston MA

                                                                Average = 88.5
                       160
                                                                 (1976 - 1989)
                       140
 Number of Homicides




                                                                                             c-chart UCL
                       120
                       100
                       80
                       60
                                                                                                     c-chart LCL                        New Chief
                       40
                                                                                                                                         in 1993
                       20
                        0
                             1964

                                    1966

                                           1968

                                                  1970

                                                         1972

                                                                1974

                                                                       1976

                                                                              1978

                                                                                      1980

                                                                                              1982

                                                                                                     1984

                                                                                                            1986

                                                                                                                   1988

                                                                                                                          1990

                                                                                                                                 1992

                                                                                                                                         1994

                                                                                                                                                1996
                                                                                     Year
“Murder rate rises in Boston”

BOSTON (AP) — The number of murders in
  Boston rose in 2000 after years of steady
  decline, partially due to an alarmingly violent first
  six months.
The final tally was 37 — up from 31 last year.
[Police Commissioner] Evans noted that there
  were no homicides in October or November


For 2001, AP reported a 67% increase
Conclusion – Data Sanity

We can either react to numbers, with explanations
 of every percent change, with the inherent
 frustrations, fear, and failure
Or
We can understand our data, put it to good use,
 and apply valid management principles

The choice is ours.

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:6
posted:6/24/2011
language:English
pages:22