VIEWS: 10 PAGES: 135 POSTED ON: 11/1/2011 Public Domain
Australian Masterclass Sally Batley Deputy Director of Analysis, NHS Modernisation Agency (UK) Working in partnership with the Patient Flow Collaborative (Victoria AU) So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts Measurement for Improvement Sir Josiah Stamp Public agencies are very keen on amassing statistics - they collect them, add them, raise them to the Nth power, take the cube root and prepare wonderful diagrams. But ... what you must never forget is that every one of those figures comes in the first instance from the village watchman (or admissions clerk?) - who puts down what he damn pleases. There are three kinds of lies: lies, damned lies and statistics After Mark Twain Collecting your data How good is your data? Is the routine data you collect and distribute 100% accurate? Is it complete rubbish? So it must be somewhere in between Issues Definitions Accuracy Consistency Timing The information vicious circle Information is not used Information is: Inaccurate Incomplete Late Inconsistent Task In groups you have to describe the people in the room so answer these questions … How many people are there in the room? How many are wearing something red? How many are tall? How many types of footwear are there? Find one word to describe the group? Issues Timing Definitions Accuracy Consistency Data types Types of data Routine v special collection Qualitative v Quantitative Soft v hard Descriptive v numeric Example of current performance: “Patients are satisfied” v waiting time is 4 hours Example of change: “Communication with patients has improved” v Average X-ray waits reduced by 20 minutes Which types are you collecting? Types of measurement Different types of measurements measurements for judgement league tables Performance Indicators Measure probability not certainty Are better in groups Are better at identifying poorer performance Should not be used for league tables Different types of measurements measurements for diagnosis to show where the problems are lots of measures comparative data useful measurements for improvement to show if improvement are being made linked to the project objectives and aims a few specific measures Measurement for Improvement or how do we know that a change is an improvement? Model for improvement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in the improvements that we seek ? Act Plan Study Do Model for improvement What are we trying to project aims accomplish? How will we know that a change is an improvement? What changes can we make that will result in the improvements that we seek ? Act Plan Study Do Model for improvement What are we trying to project aims accomplish? How will we know that a global measurements change is an improvement? What changes can we make that will result in the improvements that we seek ? Act Plan Study Do Model for improvement What are we trying to project aims accomplish? How will we know that a global measurements change is an improvement? What changes can we make that will result in the improvements change principles that we seek ? Act Plan Study Do Building Improvement Knowledge P Changes that A D result in S improvement D S Improvement P S A D A P A P S D Time Measurement for improvement: Answers the question How do we know change is an improvement ? Is linked to the project objectives or aims usually requires no more than five to seven measures crosses the whole process of care measures change over time Change areas, aims and measures should be related Area - Effective Delivery of Health Care Aim - To improve access to the appropriate treatment Measure - Reduce the number of days between referral and first definitive treatment Example from Action On programme Measuring quantitative outcomes Measuring quantitative outcomes A descriptive goal eg reduce DNAs But by how much? Quantify the starting point (baseline) Set an objective (improve by x%) How will you measure that? (methods) Monitor progress An example - hospital cancellations Monitor progress to target 16 14 12 Percentage 10 8 Baseline = 15% 6 4 Target = 5% 2 0 0 1 2 3 4 5 6 Time How are we doing? Setting the baseline Baseline period must be representative Small numbers issue Baseline period can be greater than monitoring frequency Over what period to measure baseline? DNA rate with large variation 25% Average = 8.7% 20% Percentage 15% 10% 5% 0% 1 2 3 4 5 6 Time Over what period to measure baseline? DNA rate with small variation 25.0% Average = 8.7% 20.0% Percentage 15.0% 10.0% 5.0% 0.0% 1 2 3 4 5 6 Time How will we know? Tips on measurement Measurement periods Census point (particular time of day - eg 12pm) Period of time (eg 24 hour period) Don’t mix the two! Use routine data where possible to allow cross-checking Specify method precisely eg process time in hours for patients from triage to admission onto appropriate ward How much will we improve? - Expressing the measurement of change Be realistic in your expectations Don’t think you can reduce error rate from 50% to 0% Mostly express values to one decimal place DNA rate = 5.6% (not 6%) Express target as a value not as an improvement If baseline is 5 patients/hour and you want to improve by 10% then state target as 5.5 patients/hour Avoid confusion over percentages Baseline is 10% and you want to improve (reduce) by 25% then state target as 7.5% Process Mapping Understand the process before settling on your measures Route A - Self-referral Arrival in A&E W3 W4 W1 Triage Seen by Seen by DTA A&E o/c team Indicative waits W1 - 5 minutes W2 - by category W3 - 1 hour W2 W4 - 1 hour W5 Leave W5 - 4 hours A&E We want to improve the overall patient journey Global measure: % patients seen within recommended waiting times at three key identified stages in care But Changes are made at specific points Global measure: % patients seen within recommended waiting times at three key identified stages in care The Measurement Paradox We want to improve the whole patient experience/ journey but we make changes at specific points. How do we cope with measuring the change? Specific measures can be temporary to monitor change ideas Global measures are permanent to monitor overall improvement Measurement at specific points In addition to reported global measures plotted, additional measures may be required during changes: specific measures related to the change results for sub-groups of patients results by consultant groups results for patients experiencing a particular clinical process Impact of changes on global measures (hopefully!) Average waiting times across the care pathway in days 60 50 Change 1 40 Change 3 30 20 Change 2 10 0 Setting the baseline Or how are we doing right now? Baseline period must be representative Watch out for small numbers! Baseline period can be greater than monitoring frequency Measurement guidelines key measures plotted and reported each month should clarify your project team’s aim and make it tangible. be careful about over-doing process measures. consider sampling to obtain data. integrate measurement into the daily routine. plot data on the key measures each month during the programme Task: Creating measures for your project aims Your Project is Improving Patient Flow what is your measurement strategy? what are you aims what quantified measures could be used? Data collection method what baseline are you going to use? what is the potential performance? frequency of measurement? How are you going to feed it back and to whom? Patient experience monitoring Why? To use patient feedback to improve services Agenda evaluating patient experience quantitative versus qualitative rating versus reporting practical hints and tips Task On your table, brainstorm ideas for measuring and monitoring patients’ experience with a service: How can we measure what patients think of the service? Approaches to monitoring quantitative qualitative •structured •semi-structured •unstructured questionnaires interviews interviews •“tick box” •questionnaires that •patient focus groups surveys combine “tick box” •critical incident with comment spaces technique Report experience don’t rate satisfaction How satisfied were you with the consultation you received with the doctor? very satisfied X quite satisfied satisfied quite dissatisfied very dissatisfied Please answer the questions by ticking the response which most closely matches your experience. All the treatment options were fully explained to me. I was given as much as much information as I wanted to know Treatment options were very briefly discussed with me The doctor did mention different treatments, but I did not really understand I did not feel that I was given a choice about treatment Designing a questionnaire or survey goal of the research What do you want to know? research method How will you find out? questionnaire design What sort of questions? patient sample How many will you ask? frequency of data collection How often will you ask them? data collection methods How will you ask them? systems for analysis How will you analyse the data? reporting systems How will you report the results and to whom? Designing a questionnaire or survey keep it simple plain English small patient sample and track changes over time, little and often (run chart) combine quantitative and qualitative pilot first involve patient / user representatives in questionnaire design, data collection and analysis of results Leave room for comments How satisfied were you with the consultation you received with the doctor? Please answer the questions by ticking the response which most closely matches your experience. All the treatment options were fully explained to me. I was given as much as much information as I wanted to know Treatment options were very briefly discussed with me The doctor did mention different treatments, but I did not really understand I did not feel that I was given a choice about treatment Add any other comments you wish to make in the box below The power of a good quote “The best thing was getting the date for the operation, I was given a date that suited me and was given the letter to show my boss .” “Everything was completed in one morning, I saw the Consultant went to pre-assessment and got my surgery date, this meant that I did not have to take further time off work” Back to you measurement strategy… So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts What do you know? Task What do you know about the following: Mean Variation Special causes Standard deviation What is SPC? P is for Process We deliver our work through processes S is for Statistical because we use some statistical concepts to help us understand our processes C is for Control And by this we mean predictable What is SPC for? A way of thinking Measurement for improvement - a simple tool for analysing data Better way for making decisions Evidence based management Easy, sustainable What Can It Do For Me? To identify if a process is sustainable are your improvements sustaining over time To identify when an implemented improvement has changed a process and it has not just occurred by chance To understand that variation is normal and to help reduce it To understand processes - This helps make better predictions and improves decision making What about this? Where have we come from? Compare to some arbitrary fixed point in the past the average (median) waiting time of those on the list, at 2.97 months, fell slightly over the month, and remains lower than at March 1997 (3.04 months). Show percentage change this month and to some arbitrary fixed point in the past the number of over 12 month waiters fell this month by 3,800 (7.4%) to 48,100, and are now 24,000 (33%) below the peak at June 1998 Comparing this year to last year Delayed Discharges (w eekly Sitreps) 2000/01 7000 2001/02 No . o f d e laye d 6000 d is ch ar g e s 5000 4000 3000 2000 1000 0 2 5 8 42 45 48 51 11 14 17 20 23 26 29 32 35 38 41 W eeks from October Waiting time performance What can you tell me about the following data? 2000 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 85 76 83 58 62 80 53 71 64 82 55 78 2001 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 39 19 31 22 25 51 40 11 31 54 28 16 Is this better? Average wait in days 90 80 70 60 50 40 30 20 10 0 Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Or better still? Average wait in days 120 100 80 60 40 20 0 Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Common management reactions to data take 3 different numbers 6 possible (& random) sequences 3 points can give 6 possible (& random) sequences "Upward Trend"? "Setback"? "Downturn"? 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0 1 2 3 1 2 3 1 2 3 "Turnaround"? "Rebound"? "Downward Trend"? 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0 1 2 3 1 2 3 1 2 3 Unacceptable decision-making Develop polite impatience with guesswork - single figure decision making shooting from the hip anecdotal data debate “known” solutions ?arbitrary targets and standards What else can it do for me? Recognise variation Evaluate and improve underlying process is it stable? can it meet “targets”? Help drive improvement has the process really improved or is it just chance? is it sustainable? Prove/disprove assumptions and (mis)conceptions Use data to make predictions and help planning Reduce data overload What is a control chart 80 Upper 70 process limit 60 50 Mean 40 Lower 30 process 20 limit 10 0 F M A M J J A S O N D J F M A M J J A S O N D So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts What is Benchmarking? Benchmarking compares practice and performance across organisations in order to identify ways to improve It is in essence, the identification, understanding, dissemination and implementation of best practice Benchmarking encompasses… Regular comparison of aspects of performance (functions and processes) with different practitioners Identifying gaps in performance Seeking fresh approaches to bring about improvements in performance Following through with implementation of improvements Monitoring progress and reviewing the benefits Why is Benchmarking important? Benchmarking can be used to improve the overall performance of organisations through sharing and developing different practices What are the benefits of Benchmarking? Improving quality and productivity Improving performance measurement Learning from others and greater confidence in developing and applying new approaches Greater involvement and motivation of staff Comparing performance of different people or services Measuring for judgement The minister has decided that prescribing aspirin for patients on the CHD register is a Good Thing Non-compliance will henceforth be a hanging offence But who to hang? He has been given the latest data on several Health Services Who’s doing well? Gold stars to Health The % patients on CHD register who are being Services A treated with aspirin &B February 2002 100.0% 90.0% 80.0% Average 70.0% Hanging 60.0% for Health 50.0% Services I, A B C D E F G H I J K J&K Why not traditional? Remember who’s doing well? Gold stars to Health The % patients on CHD register who are being Services A treated with aspirin February 2002 &B 100.0% 90.0% 80.0% Average 70.0% Hanging 60.0% for Health 50.0% Services I, A B C D E F G H I J K J&K A different way of presenting it The % patients on CHD register who are being treated with aspirin February 2002 100.0% 90.0% 80.0% 70.0% Average 60.0% 50.0% A B C D E F G H I J K Control limits added The % patients on CHD register who are being treated with aspirin February 2002 100.0% 90.0% 80.0% Average 70.0% Lower Upper 60.0% 50.0% A B C D E F G H I J K So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts What is Variation? Everything varies - no two things are alike Recognising this is a start but not enough: must understand it’s effect on customers and then manage it as appropriate Task In pairs think of reasons why your journey driving to work may be delayed on a morning. – Write on post its You have a few mins and we’ll come back to this later. Different Types of Variation Common Cause = Stable in time & therefore relatively predicatable For example traffic lights which hold us up today would probably hold us up in the next week. Different Types of Variation Special Cause = Irregular in time and therefore unpredictable. For Example a police convoy escorting a wide load Practical interpretation of the Standard Deviation Mean - 3s Mean Mean + 3s 3s and the Control Chart UCL 3s Mean 3s LCL 6s Reducing Variation Walter Shewhart - Statistician 1920’s Bell Telephones: every failure led to an alteration to the telephones. Good idea? Started to look at limits and Common & Special Causes “A phenomenon will be said to be controlled when, through the use of past experience, we can predict, at least within limits, how the phenomenon may be expected to vary in the future” Shewart - Economic Control of Quality of Manufactured Product, 1931 Task Back to the Task-Journey to work Which are common causes of variation? And which are special causes? My trip to work Accident on tyre had Stopped by police for motorway puncture speeding 120 100 80 average Min. 60 40 School holidays 20 Borrowed helicopter 0 COMMON CAUSE VARIATION - Points within the yellow lines is Consecutive trips variation you would expect - normal variation of the process (my trip to work) E.G. traffic lights, pedestrians, rush hour CONTROLLED VARIATION stable,consistent pattern of variation “chance”/constant causes 80 Upper process 70 limit 60 50 Mean 40 30 Lower process 20 limit 10 0 F MA M J J A S O N D J F MA M J J A S O N D UNCONTROLLED VARIATION •pattern changes over time •“assignable”/special causes 100 80 60 40 20 0 F MA M J J A S O N D J F MA M J J A S O N D 2 Ways to improve a process If controlled variation process is stable and predictable variation is inherent to process therefore, process must be changed If uncontrolled variation process is unstable and unpredictable variation caused by factor(s) outside process cause should be identified and “sorted” 2 dangers to beware of Reacting to special cause variation by changing the process Ignoring special cause variation by assuming “its part of the process” Pause: Think of some examples in your own area: - Common cause variation - Special cause variation So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC)-the math Understanding Variation Benchmarking Build you own SPC charts How to interpret the results Rules for special causes RULE 1 Any point outside one of the control limits RULE 2 A run of seven points all above or all below the centre line, or all increasing or all decreasing. RULE 3 Any unusual pattern or trends within the control limits. RULE 4 The number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of points. SPECIAL CAUSES - RULE 1 Point above UCL X UCL UCL X X X X X X X X X MEAN X MEAN X X X X X X X X LCL LCL X Point below LCL Rules for special causes RULE 1 Any point outside one of the control limits RULE 2 A run of seven points all above or all below the centre line, or all increasing or all decreasing. RULE 3 Any unusual pattern or trends within the control limits. RULE 4 The number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of points. SPECIAL CAUSES - RULE 2 Seven points above centre line UCL UCL X X X X X X X X X X MEAN MEAN X X X X X X X X X X X LCL LCL Seven points below centre line SPECIAL CAUSES - RULE 2 Seven points in a downward direction UCL UCL X X X X X X X X X X X X MEAN MEAN X X X X X X X X X LCL LCL Seven points in an upward direction Rules for special causes RULE 1 Any point outside one of the control limits RULE 2 A run of seven points all above or all below the centre line, or all increasing or all decreasing. RULE 3 Any unusual pattern or trends within the control limits. RULE 4 The number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of points. SPECIAL CAUSES - RULE 3 Cyclic Trend pattern pattern UCL X UCL X X X X XX X X X X X XX X X X X X X X X X X X X X X X XX X X X X X X XX LCL LCL Rules for special causes RULE 1 Any point outside one of the control limits RULE 2 A run of seven points all above or all below the centre line, or all increasing or all decreasing. RULE 3 Any unusual pattern or trends within the control limits. RULE 4 The number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of points. SPECIAL CAUSES - RULE 4 Considerably less than 2/3 of Considerably more than 2/3 of all the points fall in this zone all the points fall in this zone UCL UCL X X X X X X XX X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X LCL LCL NOW FOR SOME MATHS! Use individual values to calculate the Mean Difference between 2 consecutive readings, always positive = Moving Range, MR Calculate the Mean MR One standard deviation/sigma = (Mean MR) ÷ d2 * s or σ Upper Process Limit (UPL) = Mean + 3 s Lower Process limit (LPL) = Mean - 3 s * d2 is a constant for given subgroups of size n (n = 2, d2 = 1.128) H.L. Harter, “Tables of Range and Studentized Range”, Annals of Mathematical Statistics, 1960. Construction and Interpretation of (X, Moving R) Chart Run chart, running record, time order sequence Calculate the mean Calculate upper and lower process limits Interpret the chart for process control Find the causes of real change & act to improve Calculation of the mean X1 X2 X3 X4 X5 X6 X7 X8 X19 X20 5.9 0.4 0.7 4.7 2 1.3 0.8 0.7 1.5 2 Mean = X = X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + … + X19 + X20 2 = 5.9 + 0.4 + 0.7 + 4.7 + 2 +0 1.3 + 0.8 + … + 2 2 Mean = 2.545 0 Σ means “ sum of ” ΣX = 50.9 X= n 20 n = number of results SPC 33 Calculation of mean moving range R1 R2 R3 R4 R5 R6 R7 R8 R18 R19 5.5 0.3 4 2.7 0.7 0.5 0.1 1.8 0.8 0.5 Moving Range MR = R1 + R2 + R3 + R4 + R5 + R6 + R7 + R8 + … + R19 19 = 5.5 + 0.3 + 4 +2.7 + 0.7+0.5 + 0.1 + 1.8 +0 + 0.8 +0.7 + 3.7 + 0.5 + 3.8 + 0.1 + 0.2 + 0.6 +0.8 + 0.5 19 = 27.3 19 ΣR = 27.3 R = 1.437 MR= n 19 Σ means “ sum of ” n = number of moving ranges SPC 33 Calculation of σ = 1 standard deviation Calculate σ From the formula = R Never use the standard d2 deviation key on a calculator to get this = 1.437 figure 1.128 = 1.274 d2 is always 1.128 for a sample size of 2 (difference between 2 values) SPC 33 Calculation of control limits Calculate UCLX (Upper Control Limit) for X =X + 30 = 2.545 + 3.822 = 6.367 Plot on graph Calculate LCLX (Lower Control Limit) for X =X - 30 = 2.545 - 3.822 = -1.277 can’t have negative so take to be 0 Plot on graph SPC 33 And that’s how you get one of these! - a Control Chart 80 Upper 70 process limit 60 50 Mean 40 Lower 30 process 20 limit 10 0 F M A M J J A S O N D J F M A M J J A S O N D Things to remember only need 20 data points to set up a control chart “standard deviation” this is not the one used in formulae in Excel or on calculators. d2 constant sample size of 2 refers to the sample size for moving range (which is nearly always 2) - NOT the number of data points 20 data points produces 19 moving ranges Remember the 2 ways to improve a process If controlled variation process is stable variation is inherent to process therefore, process must be changed If uncontrolled variation process is unstable variation is extrinsic to process cause should be identified and “treated” CONTROLLED VARIATION stable,consistent pattern of variation “chance”/constant causes 80 Upper process 70 limit 60 50 Mean 40 30 Lower process 20 limit 10 0 F MA M J J A S O N D J F MA M J J A S O N D Remember the 2 ways to improve a process If controlled variation process is stable variation is inherent to process therefore, process must be changed If uncontrolled variation process is unstable variation is extrinsic to process cause should be identified and “treated” UNCONTROLLED VARIATION •pattern changes over time •“assignable”/special causes 100 80 60 40 20 0 F MA M J J A S O N D J F MA M J J A S O N D DEFINING LACK OF CONTROL A single point falls outside the 3-sigma control limits 2 out of 3 successive values fall on the same side of, and more than 2-sigma units from the central line 4 out of 5 successive values fall on the same side of, and more than 1-sigma unit from the central line 8 (or 7??) successive values fall on the same side of the central line, or all increasing or all decreasing Variation We live in a world filled with variation - and yet there is very little recognition or understanding of variation WILLIAM SCHERKENBACH So what are we going to cover Measurement for Improvement What is Statistical Process Control (SPC) Understanding Variation Benchmarking Build you own SPC charts SPC Spreadsheet Formulae A B C D E F G Date Data Average Moving Average Lower Upper Field Range Moving Control Control Range Limit Limit =AVERAGE( =ABS(B3-B2) =AVERAGE( =MAX(0,C2 =C2+(3*(E2 B2:B10) D3:D10) - /1.128)) (3*(E2/1.12 8))) Average of all The difference Average of the Average Average plus the data list between moving range minus 3 3 multiplied consecutive list multiplied by by Average numbers Average moving range moving range divided by divided by 1.128 1.128 Example Data Set AdmissionsInpatients Average MR Average MRLower LimitUpper limit 01-Feb-02 20 21.90 13.44444 0 57.6565 Table 1. Shows what the data 02-Feb-02 6 21.90 14 0 57.6565 should look like. 03-Feb-02 14 21.90 8 0 57.6565 04-Feb-02 46 21.90 32 0 57.6565 05-Feb-02 41 21.90 5 0 57.6565 Table 2. Shows how the formula 06-Feb-02 32 21.90 9 0 57.6565 should look. 07-Feb-02 40 21.90 8 0 57.6565 08-Feb-02 9 21.90 31 0 57.6565 Average, Lower limit and Upper 09-Feb-02 2 21.90 7 0 57.6565 10-Feb-02 9 21.90 7 0 57.6565 limit should only have the formula in the first row and the value pasted for the entire dataset. Admissions Inpatients Average MR Average MR Lower Limit Upper limit 20 =AVERAGE(B2:B11) =AVERAGE(D3:D11) =MAX(0,C2-(3*E2/1.128)) =C2+(3*(E2/1.128)) 6 21.9 =ABS(B2-B3) 0 57.6565011820331 14 21.9 =ABS(B3-B4) 0 57.6565011820331 46 21.9 =ABS(B4-B5) 0 57.6565011820331 41 21.9 =ABS(B5-B6) 0 57.6565011820331 32 21.9 =ABS(B6-B7) 0 57.6565011820331 40 21.9 =ABS(B7-B8) 0 57.6565011820331 9 21.9 =ABS(B8-B9) 0 57.6565011820331 2 21.9 =ABS(B9-B10) 0 57.6565011820331 9 21.9 =ABS(B10-B11) 0 57.6565011820331 Example SPC Chart 70 60 Inpatients 50 40 Average 30 Lower Limit 20 10 Upper limit 0 03 03 03 03 03 20 20 20 20 20 2/ 2/ 2/ 2/ 2/ /0 /0 /0 /0 /0 01 03 05 07 09 Within this process Trust x could expect to see between 0 and 58 admitted Inpatients per day, with and average of 22. Therefore, there needs to be 58 inpatient beds available everyday to match current demand. Task Split into equal groups around each laptop At least one analyst in each Let someone use the computer who is not use to working with excel Others can coach them on how to use it You have a data file on your computers called example.xls Compose a SPC chart and feedback That’s all Folks !!! Any Last Questions? Useful references Donald Wheeler. Understanding Variation. Knoxville: SPC Press Inc, 1995 Walter A Shewhart. Economic control of quality of manufactured product. New York: D Van Nostrand 1931. American Society for Quality www.asq.org/about/history/shewhart.html WE Deming. Out of the crisis. Massachusetts: MIT 1986 Donald Wheeler. Advanced topics in statistical process control. The power of Shewhart's charts. Knoxville: SPC Press Inc, 1995 Donald M Berwick. Controlling variation in health care: a consultation from Walter Shewhart. Med Care 1991; 29: 1212-25.