Sample Record Label Mission Statements by rbp49145

VIEWS: 309 PAGES: 49

More Info
									       Skills Competency Education
                       for
    New PI Directors & Coordinators

Session Three           February 14, 2007

 Data Aggregation and Assessment

Sponsored by: The MT Rural Healthcare PI Network
 Co-Sponsored by: Mountain Pacific Quality Health
                                                    1
    Today‟s Session
   Recap Session 2: Data Collection

   Turning Data into Useful Information
       Step one: aggregate
       Step two: assess
       Step three: data quality issues
       Tools

   Questions
                                           2
    Why Aggregate & Assess Data?
   To increase the usefulness of data

       To help make it „actionable‟

       To identify areas where other or more
        data needs to be collected

       To identify mistakes, poor quality data
                                                  3
    Why Aggregate & Assess Data?
   To provide objective information as the
    foundation of objective decision-making

       Always end with a decision about how to go
        forward

       Ultimately, supports the organization in
        achieving its mission, vision
                                                     4
    A Little Background… Statistics
   The science of probability
       Can become very complex
       We are not statisticians
       We don‟t need to be; someone else has done
        that work for us

   PI uses basic statistical methods and tools
    to scientifically, objectively support
    improvement efforts
       It is a scientifically sound approach
       It is improvement, not research          5
Step One: Aggregate the data




                          6
Step One: Aggregate the data
   Group like-kinds of data together
       Called a data set
       Can start this process during collection

   Aggregation tools
       Log sheets
       Table (matrix)
       Dot plot

                                                   7
     Simple Log Sheet:
     Group one kind of data together

   Data set name    MR Review     H+P
                                 in 24 hrs
                     Record 1       √
   Data label
                     Record 2       √
   Data bit label   Record 3       √

                     Record 4       √
   Data bit
                     etc
                                             8
  Table, Matrix
  Group Several Kinds of Data
Med Rec Review   H&P   Care   Immuniz DC Sum
                       Plan    ations
Record 1          √     √               √

Record 2          √

Record 3          √     √       √       √

Record 4          √     √               √


                                               9
Dot Plot (Scattergram)




                         10
    Data Aggregation Limits
   What do we know so far about the value
    or importance of the data we‟ve collected?

       Can we determine if the variation present is
        “significant”?
       Can we draw overall conclusions from it?
       Can we take constructive action based on it?


                                                       11
    Data Aggregation Limits
   If our data represents a sample, what can
    we say, or infer, about the rest of the
    group (“population”) based on our
    aggregated data?

       Making valid statements of this kind is the
        work of „inferential statistics‟
       Example: reviewing 10% of closed records
                                                      12
Step Two: Assess the data




                            13
Assessment Techniques
   Calculate measures and/or rates

   Construct charts and graphs
       Look for trends and relationships


   Evaluate the variation: is it…
       Normal or an outlier?
       Common cause or special cause?
                                            14
Assessment: Calculations
   Frequency
   Relative Frequency
   Percent, percentage
   Range
   Average (mean)
       Median (middle)
       Quartile
       Decile
                           15
   Calculations: Frequency
    “Count” data: how often something happened or was observed

   Med Rec        H&P        Care    Immun        DC       Total
   Review                    Plan    izations    Sum      (Freq)
Record 1            √         √                    √         3

Record 2            √                                        1

Record 3            √         √         √          √         4

Record 4            √         √                    √         3
Total
                     4         3         1         3
(Frequency)
                                                                   16
     Calculations: Relative Frequency
            Relative Frequency (RF) = x / n
  Med Rec       H&P     Care   Immu      DC    Total   RF
Review (n=4)            Plan            Sum
Record 1          √       √              √      3      0.75
Record 2          √                             1      0.25
Record 3          √       √      √       √      4      1.0
Record 4          √       √              √      3      0.75
Total (x)         4       3      1       3
                4/4 =   ¾=      ¼=      ¾=
RF (Rel Freq)
                 1.0    0.75    0.25    0.75
                                                              17
     Calculations: Percent
           Percent (%) = RF x 100 or ((x/n) * 100)
  Med Rec               Care           DC     RF     Percent
                H&P            Immu
Review (n=4)            Plan          Sum
Record 1         √       √              √     0.75    75%
Record 2         √                            0.25    25%
Record 3         √       √      √       √     1.0    100%
Record 4         √       √              √     0.75    75%
                4/4 =   ¾=     ¼=     ¾=
RF (Rel Freq)
                 1.0    0.75   0.25   0.75
Percent         100%    75%    25%    75%                 18
 Calculations: Range
  Subtract the lowest value from the highest value
Med Rec Review             Care
                 H&P               Immu    DC Sum
    (n=4)                  Plan
Record 1           √        √                 √
Record 2           √
Record 3           √        √        √        √
Record 4           √        √                 √
Percent          100%      75%      25%     75%

Range = 100-25 = 75%
                                                     19
 Calculations: Average, mean
  Sum of all values / n
Med Rec Review            Care
                  H&P            Immu   DC Sum
    (n=4)                 Plan
Record 1            √      √              √
Record 2            √
Record 3            √      √      √       √
Record 4            √      √              √
Percent           100%    75%    25%     75%

Average= 275/4 = 68.7%
                                                 20
    Assessment: Charts, Graphs
   Construct charts and graphs

   Add limits for evaluation
       Control limits: upper, lower
       Threshold: point we will intervene
       Benchmark: internal or external

   Look for trends and relationships
                                             21
Charts and Graphs
   Help us understand and identify normal
    variation in systems and processes and
    leave it alone

       Range of normal body temperatures, pulse
        rates and blood pressures
       Record but leave it alone


                                               22
Charts and Graphs
   Help us understand and identify
    variation that is not normal within a
    system or process and take corrective
    action that will reduce or remove it.

       Ice pack to reduce extreme fever
       Medication to reduce elevated blood
        pressure or heart rate
                                              23
Charts and Graphs: What‟s Normal?
 The Standard Normal Curve

                                                          +/- 1 SD = 68.2 % area
                                                          +/- 2 SD = 95.4 %

                             34.1 % 34.1 %
                                                          +/- 3 SD = 99.8 %; upper
                                                          and lower control limits




    0.1 %           13.6 %                   13.6 %
                                                            0.1 %
            2.2 %                                 2.2 %

   - 3 SD    - 2 SD    - 1 SD   Mean    + 1 SD + 2 SD + 3 SD

                                                                                     24
Charts & Graphs:
Add Control Limits

                                         Upper Control Limit, + 3SD




Mean, average                            Lower Control Limit, - 3SD

 Source: mathematical calculations, internal or external
                                                                      25
                    Charts & Graphs:
                    Add Threshold for Intervention
                                 Mortality Rate % Inpatients

                           2.5
12 Month Rolling Av in %




                                                                       Threshold: a
                           2.0
                                                                       predetermined point
                                                                       at which action will
                           1.5                                         be taken
                                                                       Source: internal
                           1.0
                                                                       discussions
                           0.5

                           0.0
                                 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
                                 '04 '05 '05 '05 '05 '06 '06 '06 '06



                                                                                              26
   Charts & Graphs:
   Add Benchmarks
           Heart Failure Clinical Care Guidelines

                                    120
                                                   Benchmark: a
                                    100
                Success Rate in %


                                                   pre-determined
LVS Assess                          80             level of desired
ACEI/ARB                                           performance
Discharge Ins                       60
Smoke Cess                                         Source: internal
                                                   or external
                                    40

                                    20

                                     0
                                          Q4 '06
                                                                      27
Charts & Graphs:
Look for Trends, Relationships

                        Percent All Falls By Day of Week

          25
          20
Percent




          15
          10
          5
          0
                Mon     Tue     Wed       Thur      Fri    Sat   Sun

               More     Fewer falls:
               falls:   why?
               why?
                                                                       28
Charts & Graphs:
Look for Trends, Relationships
                      CAH Admission Sources, 2004
                 2%
                3%
           8%

                                          Emergency room
    12%                                   Physician clinic
                                          Scheduled surgery
                                          OB
                             59%          SNF
     16%                                  Unexpected post op




                                                               29
    Evaluate Variation
   If know what „normal‟ looks like, you are

       Able to identify outliers: unusual,
        unexpected process/system events

       Able to evaluate relative severity or
        importance when multiple factors contribute

       Able to identify improvement and work to
        maintain gains
                                                   30
Evaluate Variation
   Common Cause Variation
       The expected variation inherent in any
        process due to the normal interaction of
        the process variables.


   Special Cause Variation
       Unexpected variation in the process due to
        a specific reason or cause.
                                                   31
Evaluate Variation:
 The Standard Normal Curve

                                                          +/- 1 SD = 68.2 % area
                                                          +/- 2 SD = 95.4 %

                             34.1 % 34.1 %
                                                          +/- 3 SD = 99.8 %; upper
                                                          and lower control limits




    0.1 %           13.6 %                   13.6 %
                                                            0.1 %
            2.2 %                                 2.2 %

   - 3 SD    - 2 SD    - 1 SD   Mean    + 1 SD + 2 SD + 3 SD

                                                                                     32
     Normal Distribution
     Even and varied distribution of points on both sides of the mean, all
     within control limits; common cause variation; the process is said to be
     ‘in control’ and/or ‘stable’.

                                   Normal Distribution


13
12
11                                                                           Series1
10
 9                                                                           +3 SD
 8                                                                           - 3 SD
 7
 6                                                                           mean
 5
 4
        1   2    3    4   5    6      7     8    9       10   11   12   13
                                                                                       33
    Evaluate Variation:
    Westgard Rules for Control Charts
   1 Point Outside                    1-3SD Warning
    Control Limits
                                       6 point trend
   2-2SD Rule                         7 + point trend

   4SD Rule                           Sawtooth


       Source: http://www.westgard.com/mltirule.htm

                                                          34
  1 Point Outside Control Limits
1 point exceeding the upper or lower control limit is special cause variation

                              1 Point Outside Control Limits

  14
  13
  12
  11                                                                     Series1
  10                                                                     +3 SD
   9
   8                                                                     - 3 SD
   7                                                                     mean
   6
   5
   4
        1     2     3     4        5      6      7       8     9   10


                                                                                  35
     2-2SD Rule
2 consecutive points greater than or less than 2 SD; special cause variation

                                     2:2 SD Rule

13
12
11                                                                    Series1
10
 9                                                                    UCL
 8                                                                    LCL
 7
 6                                                                    mean
 5
 4
     1   2   3   4   5   6   7   8   9 10 11 12 13 14 15 16 17



                                                                             36
     1:4SD Rule
     Change of 4SD up or down is special cause variation

                                   1:4SD Rule

13
12
11
10                                                                 Series1
9                                                                  +3 SD
8                                                                  - 3 SD
7                                                                  mean
6
5
4
       1    2    3   4    5    6      7     8   9   10   11   12



                                                                             37
 3 SD Warning
     Change of 3SD; special cause variation may be present; investigate

                       1: 3SD Warning, Cross Center Line

13
12
11
                                                                      Series1
10
 9                                                                    +3 SD
 8                                                                    - 3 SD
 7
 6                                                                    mean
 5
 4
       1    2     3    4    5     6     7     8    9       10   11



                                                                             38
     6 Points on One Side of Mean
 6 consecutive points on one side of the mean is special cause variation

                         6 Points on One Side of Mean

13
12
11
10                                                                          Series1
9                                                                           UCL
8                                                                           LCL
7                                                                           mean
6
5
4
     1   2   3   4   5   6   7    8    9   10   11      12   13   14   15




                                                                                39
7 Ascending, Descending Points
7 consecutive ascending or descending points is special cause variation

                     7 Point Trend, Ascending or Descending

13
12
11                                                                   Series1
10
 9                                                                   UCL
 8                                                                   LCL
 7
 6                                                                   mean
 5
 4
     1   2   3   4   5   6   7   8   9   10 11 12 13 14 15 16



                                                                            40
       Sawtooth
      A sawtooth pattern is not normal, it is special cause variation

                                       Sawtooth

13
12
 11
10                                                                                 Series1
9                                                                                  +3 SD
8                                                                                  - 3 SD
7                                                                                  mean
6
5
4
       1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17



                                                                                            41
         Practice Assessment
         Are the reductions in immunization failure rate below ‘significant’?
                Reduce Immunization Failure Rate

                12
                                 Standing orders
                10
Number of Pts




                8
                                             Discharge order sheet
                6
                4                                               24 hour review
                2
                0
                     1   2   3   4   5   6    7    8   9   10
                             Week of Improvement
  Hint: think about how the Westgard 3SD and 4SD rules looked                    42
Step Three:
Resolve Data Quality Issues




                              43
Data Quality: Validity
   Is the data itself valid?

   Are your conclusions valid?

       Is the data accurate


       Is the data reliable

                                  44
Valid Data
                Accuracy

                    Precision: how close
                     is the measured value
                     to the true value?

                    Confidence intervals:
                     how confident can
                     you be that they are
                     the true value?

                                             45
    Valid Data
   Reliability:
    do repeated
    measurements
    produce the same
    results?

       Sample size
       Confidence intervals

                               46
    Valid Data: Sample Size
   30 data points                10% of a large
    approximates the               population
    normal curve

   no less than 10               100% of a small
    data points unless             population
    it is 100%
For PI, the data just needs to be valid and actionable!
                                                          47
Questions?

                 Next Time:


             Performance Reports


             Wed, March 14 1pm



                                   48
PI Ed Session 3 References
   Handbook For Improvement, 3rd Edition;
    Healthcare Management Directions, Inc.;
    2002.

   Norman, G. and Streiner, D; Biostatistics The
    Bare Essentials; Mosby-Year Book Inc; 1994.

   http://www.westgard.com/mltirule.htm

   www.mtpin.org
                                                49

								
To top