No Slide Title by 3U8ccel


									       What Gets Measured
          Gets Undone
               Dr. Jim Mirabella
        Director of Institutional Research
              Professor of Statistics
Florida Community College - Jacksonville (FCCJ)

               J. Michael Adams
     Corporate Manager, Quality Services
    Florida Power & Light/ FPL Group, Inc.

            Florida Sterling Conference
              Wednesday, May 31, 2000
           Disney World, Orlando, Florida
   What’s getting “undone” ?

• The desired outcome and supporting
  processes that the measure(s) intended to
• Why????
  – Poor measures drive poor performance!

 Objectives of the workshop
• Know good measurements from poor
• Distinguish the implications of good and
  poor measures on performance
• Balance an array of measures that favorably
  tracks a process’s outcome for planning,
  assessing performance, and analysis
• Assess attendees’ measures
• Any other expectation?                     3
“Why” Measure at All?
• Measurement is the language of progress
    and comparison
•   Provides a sense of where we are AND
    where we are going (planning)
•   Can guide a steady advancement toward
    established goals (tracking)
•   Can identify goal shortfalls, or over-
    achievement (analysis)
•   Communicates to the work force what is
    important to the organization (behavioral)
Good measures are born
from SMART goals
 •   Specific
 •   Measurable
 •   Agreed upon
 •   Realistic
 •   Time-Bound

The Challenge of Measuring
• Workers might perceive it as a threat
• Workers might disregard organizational
  goals, customers, products and services
• Workers might focus on obtaining favorable
• Measuring items with no influence on
  organizational success = waste of time
  – result is a bean-counting approach that focuses
    on irrelevant details
• Expected to do something!                      6
   The Challenges- examples

• “Apples and oranges”
• Impact on comparisons, benchmarking and
• turnover
• consistency
• defects vs. defectives
• commutes (measured in time or miles)
• school achievement                        7
“Where” are these

•   Organizational
•   Process
•   Department/work unit
•   Individual
    – Exercise

Your measures                    Jot down your various
performance measures for you, your department, unit and
organization. Indicate the purpose of the measure ( 1=
planning, 2= tracking and analysis; 3: behavioral changes)

• Organization                 • Department/ Unit

• Process                      • Individual

        “What” Do We Measure?

•   Quality      •   Defects
•   Delivery     •   Satisfaction
•   Cycle Time   •   Complaints
•   Waste        •   Financials
•   Cost         •   Price

Measurement Examples
 Operations-related measures
    - reliability
    - timeliness of delivery
    - order processing time
    - errors / defects
    - product lead time
    - inventory turnover
    - cost of quality
    - employee
Measurement Examples
 Customer-related measures
     - customer satisfaction
     - customer complaints
     - customer retention
 Financial measures
     - market share
     - sales per employee
     - return on assets
     - return on sales
Baldrige/ Sterling: Results Category
Private, Education, Health Care
7.1:   Customer Focus Results; Student
       Performance, Patient and other
       Customer Focus
7.2:   Financial and Market; Student and
       Stakeholder Focused Results,
7.3:   Human Resource Result; Budgetary and
       Financial Results, Staff and Work
7.4:   Supplier and Partner Results; Faculty
       and Staff Results
7.5:   Organizational Effectiveness
           Determining and Reviewing
            Measurements for Balance

           Results 7.0    •Financial
                          •Human Resources
            process       •Supplier/ Partner


Desired Outcomes- Airlines
 overall organization balance scorecard
• “Assumed Quality”- arrive safely
• Loyal Customers
  – Market differentiation, key value attributes:
     • on-time arrival
     • baggage handling
     • customer complaints
• Profitability and Market share
  – Satisfied stakeholders (shareholders, partners)
• Employee growth and retention                     15
Airline Measurement System
• Individual measures for each flight process
• Group measures for overall airline
• If problems occur with a flight, who do you
  blame -- the flight crew or the airline?
• What measures do you affiliate with the
  flight crew?
• What measures do you affiliate with the
• What is important to you for a satisfactory
  flying experience?
       Review your measures

• Are the organization measures reflective of
  all Baldrige results items?
• Do the measures reward favorable
• Is there alignment with the contributing
  departments and suppliers?
• Are the overall results managed as an
  outcome of a process?
     What’s the opportunity?
• Many organizational measurement systems
  are too short, too rigid, or used like a strict
  teacher’s ruler ... to whack rather than to
• Need to replace these outdated
  measurement systems with more dynamic
  measurement system that motivates
  continuous improvement in customer
  satisfaction, flexibility, and productivity …
Mis-measurement Systems
• Unless specifically tuned to flight plan,
  measurement systems may:
  – yield irrelevant / misleading information
  – provoke behavior not conducive to
• Traditional measures ignore requirements &
  perspectives of customers (internal/external)
• Bottom-line measures (profitability) too late
  for mid-course correction / remedial action
Mis-measurement Systems
• Many measurement systems overlook key
  non-financial performance indicators
• Measures often used for punishment rather
  than to promote learning
• Many measurement systems are inflexible
  and limited in what they can do

What Should a
Measurement System Do?
• Measures must link operations to strategic
  – departments should know how they contribute
    separately and together toward strategic mission
• System has to integrate financial / non-
  financial info in a way usable by managers
  – managers need right info at right time
• Measurement system’s real value lies in its
  ability to focus all business activities on
  customer requirements
What Should a
Measurement System Do?
• Measure what is important to the customers
• Motivate operations to continually improve
  against customer expectations
• Identify and eliminate waste -- of both time
  and resources
• Help to accelerate organizational learning and
  build a consensus for change when customer
  expectations shift

    Measurement Mismanagement
In an effort to increase market share, a computer service
bureau strategizes to improve the timeliness of its
voicemail service. The goal is to provide new customer
with service w/in 24 hours. To help speed the process, a
program is developed to accept verbal phone orders.
Most new customers were online w/in a day as
promised, causing the company to celebrate their
success. A later audit revealed 70% error rate in order
entry, with 30% of customers disputing their bill and
eventually canceling service..
 Measurement Mismanagement
A Hi-Tech company sets objective to become highly
profitable by being a product leader. To measure the
performance of its marketing and R&D functions, the
number of new products developed is the most watched
barometer. An internal review reveals that in a sample
of 20 new product introductions, 80% were delivered
over a month late, and significant waste piles up in
production. Management cannot understand why the
accountant’s ink is red – after all, their yardstick tells
them they are developing new products at a record rate.

Beware of ……..
Seemingly Simple Measures

  • How satisfied are your customers?
  • What is your employee turnover?
  • What is your client retention rate?
  • Is the value of our service worth the

Beware of ……..
 Treating customer perceptions as
   objective measures
   • Customer satisfaction  product quality
   • Customer satisfaction is a complex
   • A well-handled complaint results in higher
     customer satisfaction than does no
   • More reasons for complaint  more
Beware of ……..
 Non-specific measurements
   • Results are actionable
   • Hard to improve what you cannot assess
 Failing to measure adequately
   • Don’t give employees an outlet for gaming
   • Identify all areas important to customers

Beware of ……..
 Using results incorrectly
   • Don’t tie results to employee pay unless
     employees can directly influence results
   • Don’t base employee pay on results that
     cannot be measured

Recap of Objectives and

• Know good measurements from poor
• Distinguish the implications of good and
  poor measures on performance
• Balance an array of measures that favorably
  tracks a process’s outcome for planning,
  assessing performance, and analysis
• Assess attendees measures
       Thank you!!!!!!

Jim Mirabella
 Mike Adams


To top