Crayons or Laptop dreaming?
Is measurement important in improving
practice?
Don Campbell
Clinical Epidemiology & Health Service Evaluation Unit
Melbourne Health
Improvements
• the will to make the change
• the ideas to make the change
• the execution of the ideas
Commitment to measurement and reporting
Do you need baseline data before you start?
Fundamental Questions for
Improvement
• What is the aim?
• What will be measured to know the aim has
been achieved?
• What are the changes?
Measures
• The key measures should operationalize the
aim
– LOS for admitted, discharge, and fast track
– Clinical improvements
– Patient satisfaction scores
• Collect data on sub-components of the
system judiciously
- ie, only if it is necessary (parsimony)
Some Things to Consider When
Making Improvements
• Multiple PDSA Cycles (and time) are
usually needed to adapt a change
• Pay attention to detail
• Measurement - “useful not perfect”
• Promote the project
• Overcoming barriers to achieving
success
• Hold the gains
Model for Improvement
What is the aim?
What will be measured to know the
aim has been achieved?
What are the changes?
Act Plan
Study Do
Model for Improvement
What are we trying to accomplish?
How will we know that a change is
improvement?
What change can we make that will
result in improvement?
Act Plan
Study Do
Use of Data
Changes That
Result in
A P Improvement
1d
S D
1c
1b
Median LOS for Admitted Patients
A P 320
1a 300 1 2 3
S D LOS (minutes) 280
260
240
220
Theories Goal
200
Ideas 180
160
Week
1. “quick-look” x-rays 3. Bed ahead
2. Work-up done on floor
Useful Measurement
• Data directly related to aims
• Data collected in cycles to determine
the effect of a particular change
• Qualitative data to assist in refining a
change
“Have a narrow bandwidth & stay on the
money”
Collecting Data
• Use purposive sampling to conserve
resources
– Sample data daily for Fast Track, Main ED,
Admitted
– Summarize data weekly using median to
lessen effect of outliers
• Integrate measurement into the daily
routine
Operationalising Data Collection -
Examples
• Time to analgesia
– pen/paper stuck on narcotics safe
– pain scale at triage
• Fast track
– identify on computer (or manually on assigned
cubicle)
• Ottawa ankle rules
– aide memoire at triage and/or in cubicles
Length of Stay for Main ED Discharged
Patients (n=1 per week)
300
Avg=180, SD=50
250
LOS (Min.)
Avg=135, SD=35
200
150
100
50
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Week
Median Length of Stay for Main ED
Discharged Patients (n=14 per week)
200
180
LOS (Min.)
160
140
120
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Week
Median Length of Stay for Main ED
Discharged Patients (n=28 per week)
200
180
LOS (Min.)
160
140
120
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Week
Median Length of Stay for Main ED
Discharged Patients (n=300 per week)
200
180
LOS (Min.)
160
140
120
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Week
A cautionary note
• New performance management structure may support
abstract managerial values at the expense of other
cultures of performance evaluation, and foster fear
rather than QI
• Performance measurement implies a mode of
management
– Guidedog or guard dog?
• Risk that this approach will displace existing formal or
informal internal or professional modes of QA
• What are clinicians already doing that constitutes good
practice, can we build on this to make it better?
A cautionary note
• Successful change: “honour the culture and
respect the past”
• Competition
– clinical culture naturally competitive
– can we manage this for improvement?
– not “competition for competition’s sake”
• Benchmarking
– opportunity rather than threat
– driver to improvement
Conclusion
• Measurement is important
– identifying a problem (helps convince others too)
– demonstrating that change can lead to improvement (data
will convince sceptics)
– holding the gains
• If you aren’t committed to measurement
– how will you know you made a difference?
– your activity is diversionary therapy
• If this was easy we wouldn’t be sitting here