; h - NASACT - The National Association of State Auditors_ Comptrollers
Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

h - NASACT - The National Association of State Auditors_ Comptrollers

VIEWS: 0 PAGES: 41

  • pg 1
									        Session Sponsored by the
NASACT Joint Middle Management Conference




A Systems Approach to Performance
  Measurement and Management


               Stephen L. Morgan
                 April 16, 2012


                                            1




         Today’s Presenter

Stephen L. Morgan, President, EGAPP,
 Inc., and former Austin City Auditor can
 be reached at egappmorgan@yahoo.com




                                            2




                                                1
A Systems Approach to Performance
Measurement and Management
  I.   Introduction – Performance
       Accountability System
  II. Historical Overview – Where We Have
       Been in Austin and Beyond
  III. Performance Planning
  IV. Performance Budgeting
  V. Performance M
  V    P f                      t d Reporting
                    Measurement and R    ti
  VI. Performance-Based Decision Making
  VII. Conclusion – What We Have Learned
       and Where We Are Going
                                            3




I. 1. Introduction: Government
      Performance Accountability System

                PLAN
         Strategic & Annual
              Planning


       ACT                         DO
Performance-Based             Performance
 Decisio Maki g
 Decision Making               Budgeting


                CHECK
             Performance
        Measurement & Reporting             4




                                                2
I. 2. Managing for Results Framework
      City of Austin

   PERFORMANCE-BASED                  BUSINESS
       DECISION-MAKING                PLANNING
                                      • Program/Activity Objectives
                         • Citizens
                                      • Organizational and
                         • Council
                                        Individual Performance
                       • Managers       Measures
                      • Employees     • Structural Alignment


                 • Individual SSPR    • Performance Targets
                        Evaluations
                                      • Accounting System
                  • Organizational
          Performance Assessment
                • Performance and
              Measurement Audits

       PERFORMANCE                     PERFORMANCE
      MEASUREMENT &                    BUDGETING
          REPORTING
                                                                      5




I. 3. Characteristics of a Successful
      System
  Use existing data whenever possible
  Find a balance between too few and too
   many measures
  Audit the data regularly
  Modify measures when necessary
  Centrally located staff to analyze data and
   coordinate the system elements
  Technological infrastructure to support the
   system
                                                                      6




                                                                          3
I. 3. Characteristics of a Successful
      System (continued)
   Data forms should have space for
    explanatory information and detail
   Tie measures to budgetary allocation and
    reward system
   Support of top management
   Over the long run should affect bottom line
    performance of the organization
   Citizens will be better informed and more
    participative
                                                  7




II. 1. Where We’ve Been (in the
City of Austin) …
   1992 – Council Resolution on Performance
    Measurement and Reporting
   1994 – First Performance Measurement &
    Reporting System Audit
   1996 – Second Performance Measurement
    and Reporting System Audit; Program
    Budgeting implemented
   1998 – Third Performance Measurement
    and Reporting System Audit

                                                  8




                                                      4
II. 1. Where We’ve Been…
  1998 Corporate Managing for Results Initiative
   Defined
      • Simplify our System
      • Clarify the Information We Provide
      • Develop Measures that are Meaningful to our
        Employees
      • Focus on Cost
  1999 Corporate Partnership Implements CMO
   Initiative
      •                        manual--The
        Developed a Standard manual The Resource Guide
      • Trained over 200 managers
      • Developed a Single Accounting System
      • Identified Key Performance Measures for Executive
        SSPRs
      • Corporate Review Team
                                                         9




II. 1. Where We’ve Been…
  2002 Fourth Audit of the Performance
   Management System
      •   Ongoing Integrated System
      •   Information Used for Operational Management
      •   Measures Are Relevant and Reliable
      •   Budgets Are More Data and Results Driven

  2003-2008 Continuous Improvement
      • Managers and Supervisors Fully Trained
      • Performance Measures Supported by More Robust
        Technology
      • Improvements Made to City’s Website and
        Stakeholder Access to Performance Information
      • Citizen and Employee Surveys Provide Data for
        Selected Performance Measures
                                                        10




                                                             5
 II. 1. Where We Are Now…
    2008-Current
        • Website Robust with Capacity to “Drill Down” and
          Search” through Performance Measures Database
          “Managing f R
        • “M                    lt ” Used
                   i for Results” U d as B iBusiness Pl   i
                                                     Planning
          and Performance Monitoring Model for More than a
          Decade--Now Part of City Culture
        • Performance Report on Website tracks 115 Key
          Departmental Measures, of these 21 are Designated
          Citywide Key or “Dashboard” Measures
        • Performance Comparisons Presented in Graphics
          with Goal/Targets and Measures Tracked Over Five
          Years
          Y
        • Performance Report for 2009-2010 Received
          “Certificate of Excellence” from ICMA in October 2010
        • Annual Citizen Surveys Strengthened to Include
          Focus Groups and Presentations to City Council
        • “Best Practice Citizen Centric” External Performance
                                                            11
          Accountability Report Is Needed




     II. 2. Federal Government Performance
    Management System Continues to Evolve

 G          tP f             d R lt A t f
  Government Performance and Results Act of 1993
 Executive Order 13450-Improving Government
     Program Performance, Nov 13, 2007
 OMB 10-24: Performance Improvement Guidance
  under GPRA for 2011-2012
 Government Performance and Results Act
  Modernization of 2010 (signed Jan. 4, 2011)




                                                                  6
    II. 2. Federal Agencies with Well Developed
    Performance Management Systems

   Social Security Administration
   Department of Interior
   Government Accountability Office
   Nuclear Regulatory Commission
   Office f Personnel Management
    Offi of P         lM           t



                                                         13




    II. 3. Some Local, State, & Provincial Governments
    Have Established Performance Management Mandates




                                                              7
   II. 3. State and Local Governments With
   Well Developed Performance Management
 States of Florida, Washington, Texas, Missouri, and
  Oregon (
  O             have b
           (may h                i d for i di id l St t
                      been recognized f individual State
  departments who are mature and excel in developing
  and applying performance management systems)
 Canadian Provinces are a good model
 Local governments include Austin, King County,
  Phoenix, Bellevue, Charlotte, Portland, Palo Alto, and
     ll h
  Tallahassee
 Auditors have played key roles in many
  performance measurement and
  management initiatives




III. Performance Planning
    III. 1. Establishing programs, activities,
            and potential performance
            expectations
    III. 2. Developing annual
            business/performance plans with
            performance expectations and
            measures
    III. 3. Reviewing business/performance
            plans to support improvement
            and accountability
                                                       16




                                                            8
 III. 1. Service Delivery System (Program
                   Model)

  Input                               Other Contributing
          Process
                                           F t
                                           Factors
            Output



               Intermediate
                 Outcome
                                             Community
                                               Impact
                         Long-term
                          Outcome




III. 1. Service Delivery System
Cause/Effect Relationships


 Inputs        Processes      Outputs       Outcomes


                                      Service
   Service Efforts
                                  Accomplishments
Financial Inputs/Outputs      =      Unit Cost
Outputs/Physical Inputs       =      Productivity
Inputs/Outcomes               =      Cost Benefit and Cost
                                     Effectiveness     18




                                                             9
    III. 1. Service Delivery System:
    Auditing Program
                  Auditing Program or Activity
Inputs              Processes            Outputs           Outcomes
•   Staff           • Audit Process-     • Reports         • Qualitative –
•   Funding           (Survey,           • Briefings         Policy/system/
•   Equipment         fieldwork, &       • Presentations     management
•   Facilities/Rent   reporting)                             improvements
                                                           • Quantitative –
                                                             Cost savings/
                                                             revenue
                                                             enhancement
                                                           • Preventive –
                                                             Deterrence/
                                                             detection 19




    III. 1. Program/Activity Mapping Template
                                       Outputs (Services        Outcomes
         Inputs         Process
                                          Delivered)            (Results)




    20




                                                                              10
III. 1. Austin’s Definition of
Programs
Activity = Input Process       Output Outcome
Program = group of activities with a common
 purpose

Example: Auditing program consists of four
  separate activities:
Performance Audits
Investigations
Consulting and Assistance
Quick Response
                                                                 21




III. 2. Government Performance Expectations
   MISSION PERFORMANCE OBJECTIVES/GOALS/TARGETS

 INPUT                PROCESS               OUTPUT            OUTCOME
                      Efficiency            Effectiveness     Effectiveness
 Economy &
 Sufficiency
 • Financial:         • Productivity        • Quantity        • Mission &
   Amount, timing                                               Outcome Goal
                                            • Quality:          Achievement
 • Physical:
                      • Unit Costs            Products,
   Quantity,                                                  • Financial
                                              delivery
   quality, timing,                                             Viability
   price
                      • Operating           • Timeliness
 • Capacity    vs.      ratios                                • Cost-Benefit
   demand                                   • Price or cost
                                                              • Cost-
                                                                Effectiveness




                                       22




                                                                                11
III. 2. Performance Measures


                               e tent
Q antitati e indicators of the extent to
Quantitative
which performance expectations (goals,
objectives, and targets) are being
achieved. Compare measures to
expectations.




III. 2. Inputs (Resources)

    Resources used to provide a service
    Financial and non-financial resources
    Dollars (actual and budgeted)
    Full-time equivalents
    Direct labor hours
    Equipment
    $, $, $, $, $, $, $, $




                                             12
    III. 2. Outputs (Services Delivered)
    Counts of what was done (often ends with “ed”)
    Units of service or products provided, or numbers of people
     served
    Synonymous with quantity of workload accomplished (not
     potential workload)
    May be conditioned by quality or timeliness (e.g., units
     completed on time, that meet quality standards, with
     satisfactory customer ratings)
     Examples:
        Number of clients served
        % clients served within 15 minutes
        Number of lane-miles resurfaced
        % lane miles resurfaced to high quality condition
        Number of potholes filled within 48 hours of notification
        % fires responded to within 6 minutes




III. 2. Outcomes
Focus on:
    Why the program exists—direct link to general goals
    How the program is expected to benefit or impact people
                   community city region
     or place (the community, city, region, state or province)
    What citizens or customers can expect to find changed or
     accomplished as a result of the program

Examples:
        % solid waste sent to landfills
                                          good        very good”
         % residents rating water service “good” to “very good
        Number of trainees who get jobs and stay employed for at least six
         months
        Number of fire related deaths per 10,000 residents
        % lane-miles of State roads in high quality condition




                                                                              13
  III. 2. Efficiency

  Relates inputs to outputs or outcomes
     Usually expressed as a cost per unit
     Input/output Example: Cost per lane mile paved
     Input/Outcome Example: Cost per life saved
  Also expressed as productivity:
     Outputs/Inputs (units produced/labor hours)
     Example: Tons of solid waste collected per
      employee-year




III. 3. Austin’s Ongoing Effort to Produce
High Quality Performance/Business Plans
    Review Team
      • Budget Office, Organizational Development, City
        Management

    Structure
      • Does it provide for alignment of results?
      • Does it permit illumination of results and cost information
        in a manner useful to decision makers?

    Results
      • Do goals/objectives and measures match?
      • Was template used for best impact?

    Measurability
      • Are goals/objectives measurable?
      • Are program and activity measures useful?
                                                                      28




                                                                           14
III. 3. Overview of the Development of Business Plans
  Service        Common Purpose          Performance Measures
                                                                                                            Environmental
  Service        = Activity A            Result                 Key Result A
                                                                                                                 Scan
  Service                                Output                  Performance
  Service                Activity       Efficiency                  Measure           Common Purpose
                          Objective      Demand
                                                                                        = Program
                                                                                            Program           Change
  Service        Common Purpose          P f         M
                                         Performance Measures                                 Objective       Dynamics
  Service                                                        Key Result B
                 = Activity B            Result
                                                                  Performance           Results =
  Service                                Output
  Service                Activity       Efficiency                  Measure           Accomplishment of
                          Objective      Demand
                                                                                         Key Result A                     M
                                                                                         Key Result B
                                                                                         Key Result C                     I
                                                                                                              G
  Service
                Common Purpose           Performance Measures                                                              S
                = Activity C             Result                 Key Result C                                 O
  Service
                                                                  Performance                                              S
  Service
                         Activity
                                         Output
                                                                                                              A
  Service
                          Objective
                                         Efficiency                  Measure
                                                                                                                           I
                                         Demand
                                                                                                              L
                                                                                        Common Purpose
                                                                                                                           O
                                                                                        = Program             S
                                                                                                                           N
  Service        C         p
                 Common Purpose          Performance Measures                                Program
  Service        = Activity D            Result                 Key Result D                 Objective
  Service                                Output                  Performance
  Service                Activity       Efficiency                  Measure           Results =
                          Objective                                                     Accomplishment of       Change
                                         Demand
                                                                                         Key Result D         Dynamics
                                                                                         Key Result E
  Service        Common Purpose          Performance Measures
  Service
  Service        = Activity E            Result                 Key Result E
                                         Output                  Performance                                 Environmental
  Service                Activity       Efficiency                  Measure                                     Scan
  Service                 Objective      Demand

                                                                                                                             29




 III. 3. Business Plan Alignment Worksheet with
 Definitions
                                  ALIGNMENT WORKSHEET BY ACTIVITY
    Business Plan Element                                                       Results

City of Austin Vision:                We want Austin to be the most livable community in the country.

Vision: (optional)                    Describes the desired future state or set of circumstances.

Mi i
Mission:                              C      h i                  f h D            ’           Id ifi D         ’ i
                                      Comprehensive statement of the Department’s purpose. Identifies Department’s primary
                                      customers and identifies the products or services that are provided.
Goals:                                A broad statement describing the desired outcome for an organization or its programs.
                                      Defines the significant results to be achieved over the next 2 – 5 years.
Program:                              Two or more activities grouped together to form a common purpose to define a program.

Program Objective:                    Clear statement of the objective of the program.

Program Results Measure(s)            Key results this program is expected to achieve.

Activity:                             A set of services with a common purpose that produce outputs and results for customers.

       y j
Activity Objective:                                    f     p p      f            y
                                      Clear statement of the purpose of the activity.

Services that comprise the            A service is a set of actions that produce a product, output, or result directly with or for
Activity:                             customers.
Activity Performance                  Results:      The impact that an activity has on customers/citizens
Measures:                             Efficiency:   Unit cost of an output
                                      Demand:       The amount of services requested or expected by customers of the activity
                                      Output:       Units of services provided, products provided, or people served through
                                                     the activity
Responsible Employee:                 Department Executive/Manager responsible for Activity                                  30




                                                                                                                                     15
III. 3. Sample Business Plan Alignment Worksheet
            Business Plan Element                                                               Results

City of Austin Vision:                   We want Austin to be the most livable community in the country.

Vision: (optional)                       Our community will be the healthiest in the nation.

Mission:                                 The purpose of the Austin/Travis County HHSD is to work in partnership with the community to promote health,
                                         safety, and well being.

Goals:                                   The over-all goal of the Austin/Travis County HHSD is to promote a healthy community which reflects social equity.
                                         This over-all goal will be achieved through:
                                         1) Minimizing the public’s exposure to health and environmental hazards.

Program:                                 Environmental Health Services

Program Objective:                       The purpose of the Environmental Health Services is to provide protection and enforcement service to the public in
                                         order to minimize environmental health hazards.

Program Results Measure(s)               Average response time to complaints/requests
                                         Confirmed cases of food-borne illness
                                         Percent of customers satisfied with complaint/request processing

Activity:                                Health and Safety Code Compliance

Activity Objective:                      The purpose of Health and Safety Code Compliance is to provide inspections, investigations, consultations, and
                                         training for the public in order to minimize public exposure to food-borne illness and other environmental health
                                         hazards.

Services that comprise the Activity:     Inspection services
                                         Investigation services
                                         Provide training to food operation employees

Activity Performance Measures:           Results:
                                         Confirmed cases of food-borne illness
                                         Efficiency:
                                         Cost per food establishment permit
                                         Average inspections/investigations per inspector
                                         Cost per food manager trained
                                         Output:
                                         Number of complaints/requests completed
                                         Number of food establishment, mobile food vendor inspections
                                         Number of temporary food inspections
                                                                                                                                                       31
Responsible Employee:                    Donald Smith




  III. 3. Plans: Consistent Process &
  Product
      Program and Activity Objectives: MFR Template
      Performance Measures: A Family of Measures

                                       The purpose of ________________
                                       is to provide___________________
                                       to ___________________________
                                       so they can __________________

                                                          Result Measure…then
                                                          •      Outputs: How many?
                                                          •      Efficiency: At what cost?
                                                          •     Anticipated Demand
                                                                                                                                                       32




                                                                                                                                                              16
III. 3. Program/Activity Objective
Statement (example)

The purpose of the        Combat Operations
                          (program/activity)
is to provide/produce emergency incident response
                          (service or product)
to                        anyone in the service area
                          (customer)
in order to               save lives and minimize
                          property damage
                          (planned benefit)
                                                          33




III. 3. Performance Measures
(example)
     Result:       Number of fire deaths per capita
                   Percent of fires confined to the
                   room or area of origin after arrival
                   of AFD (per census track)
     Efficiency:   Average cost per call
     Output:       Number of calls (call volume)
     Demand:       Number of fire alarms (calls)
                   expected


                                                          34




                                                               17
  Exercise 1: Achieving a Balance
Between Relevance and Consistency


  1. Why must we allow measures to
      change?
       y
  2. Why must we insist that measures
      stay the same?

                www.Auditor Roles.org   35




IV. Performance Budgeting

  IV. 1.   Link annual plans and budgets

  IV. 2. Establish targets

  IV. 3.   Collect cost accounting
           information


                                        36




                                             18
IV. 1. Link Annual Performance Plans
and Budgets

   Ensure clear linkage between the
    plan’s programs/activities and the
    budget’s programs/activities

   Ensure congruence between the
           goals, objectives,
    plan’s goals objecti es and targets
    and the budget’s goals, objectives,
    and targets
                                             37




IV. 1. The Budget – Linking
Results, $$$, and People
  In the Budget Document
    • Business/Performance Plan
    • Activity and Program Pages
    • Performance Measures: definitions, compared
             to targets, trends, and costs
  Using the Performance Budget to
        g                   g
   “Tell Your Story”
    • Changing the Conversation
    • This Result…At This Cost
                                             38




                                                    19
IV. 2. Establish Targets To Compare
to Actuals

 Targets for each program and activity
  measure
 Sources of criteria for setting targets
          •   Historical trends and baselines
          •   Program requirements or intent
          •   Customer expectations or demands
          •   Industry or sector standards
          •   Benchmarking within the organization
          •   Benchmarking outside the organization

                                                                                    39




  I.V. 2. Examples of Performance Targets and Measures
   Model Component                 Target (Expectation)                 Measure (Actual)
Input Economy               In FY 2011, decrease the purchasing Number of purchasing office
                            office’s personnel allocation by five positions deleted in FY 2011.
                            positions.
Process Efficiency
                 y                      , provide vehicle
                            In FY 2011, p                          Average vehicle preventive
                                                                         g         p
                            preventive maintenance services at     maintenance unit costs in FY
                            the unit cost $500 or less per         2011.
                            vehicle serviced.
Output Quality (accuracy)   In FY 2011, reduce the restaurant      Percentage reduction in the
                            critical inspection error rate by 10   restaurant critical inspection
                            percent.                               error rate in FY 2011.
Output Quantity             In FY 2011, expand “green energy”      Number of additional homes
                            electrical services to 1000            and businesses in FY 2011
                            additional homes and businesses.
                                                    businesses                green energy
                                                                   receiving “green energy.”
Output Timeliness           In FY 2011, all Level 1 emergency      Response times (range) to
                            calls will be responded to with a      Level 1 emergency calls in
                            unit on site within six minutes.       FY 2011.
Outcome Effectiveness/      In FY 2011, increase convention        Change in convention center
customer satisfaction       center customer satisfaction rate      customer satisfaction rate
                            from 4.5 to 4.7 on a 5.0 scale.        during FY 2011.
                                                                                                40




                                                                                                     20
      IV. 2. Sources of Performance
               Expectations
The process for identifying expectations and
  tti t      t h ld b i
setting targets should be rigorous.

Exercise 2:
List the pros and cons of deriving criteria for
performance expectations from the sources
discussed previously.


                    www.AuditorRoles.org           41




IV. 3. Base program budgets on unit
costs that support desired program
outputs and outcomes as reflected in
targets

        Activity-Based            Costing (ABC)

        Identify   Direct and Indirect Costs



                                                   42




                                                        21
IV. 3. Performance Budgeting
 Long-sought “ideal” of budgeting experts:
  Performance-driven budgeting.
                         budgeting

 Best-case reality:
  Performance-informed budgeting.


 Exercise 3: Why might the best-case
   reality be “better” than the ideal?
               www.AuditorRoles.org      43




V. Performance Measurement and
Reporting

  V 1 Individual Performance Appraisal
  V. 1.

  V. 2. Organizational Performance
        Assessment and Reporting

  V. 3. Performance Audits and
        Measurement Certification Audits

                                         44




                                              22
V. 1. Establishing Accountability Key Points
of Business Plan Alignment/ SSPR Integration
  Every employee in the organization contributes to the City Vision

  Every employee in the department contributes to the Mission of the
   department.

  Every employee in the department contributes to at least one Business
   Plan Goal/Objective.

  The Alignment Worksheets show employees how the Services they
   p         pp     p                      g                  j
   provide support specific Activities, Programs, and Goals/Objectives in
   the Business Plan.

  Performance Measures show citizens, City Council and employees how
   well we are doing.

  Every Business Plan Measure must be written into at least one
   employee’s SSPR including departmental executives.         45




V. 1. Individual Performance Appraisal
Alignment Worksheet                         Employee SSPR
      Mission                     Program Objective
      Goals
      Program –
          g
      Activity –
Services that comprise Activity   Activity Objective
  
  
Activity Performance Measures     Activity Results Measure
      Results:

      Efficiency:
                                  Description of Services
                                  D    i ti    fS i
      Demand:

      Output:                     Individual Performance Measure
                                  • Same as the Activity Performance Measure
                                  • Part of the Activity Performance Measure or,
                                  • Contributes to the Activity Performance
                                    Measure                                  46




                                                                                   23
V. 2. What is a Performance
Monitoring System?
 Management
  P f
  Performance       Program
                    P               P f
                                    Performance         I    d d
                                                        Intended
  Expectations      and Levels      Indicators          Uses

 Data Component
       Data Collection                     Data Processing

 Analysis Component
    Measurement of Current       Comparison of Current Performance
    Performance Levels           with Performance Expectations

 Action Component
  Decisions         Decisions Concerning    Decisions Concerning
  Regarding Results Programs and Levels     Monitoring & Evaluation

                                                                      47




V. 2. Ensure Performance Measure
Definitions/Formulas are Established


   Design monitoring system to track
   and analyze the selected measures
   (efficiency, outputs, and outcomes
   are essential). If expectations are
       achieved,
   not achieved determine why and
   take action.

                                                                      48




                                                                           24
V. 2. Ensure the Results of
Performance Measures are Available
for Analysis and Decision Making

   Design a reporting system that is
   easy to use, accessible to all
   interested parties, and enables
   management decisions.




                                       49




V. 2. Establish Performance
Reporting “Best Practices”

  Design reporting formats and
  d id frequency of reporting.
  decide f           f      i
  Reports may include:
       Daily/Weekly Performance
        Reports
       Quarterly Performance
        Quarterly
        Reports
       Annual Performance
        Reports
       Citizen Centric Reports        50




                                            25
 V. 2. Use Performance Reports to
 Improve Performance
     Use performance reports to identify
      and direct analysis of program
      performance
     Use analysis to identify the causes of
      inadequate program performance and
      focus improvements on causes
     Also use performance reports to
      identify high performance programs
                                                 51




       V. 2.1. City of Austin
       Performance Report
 Departmental Performance Measures
     Total f      M          G
   • T l of 115 Measures Grouped i   d into
     Public Safety, Community Services,
     Infrastructure, and Utilities/Enterprise
     Departments
   • Each performance graphic includes:
     M
     Measure D      i ti   C l l ti M th d
              Description, Calculation Method,
     Results, Assessment of Results, Next
     Steps, and Contact for More Information

                                                 52




                                                      26
        V. 2.1. City of Austin
        Performance Report

    Decisions influenced by:
       • Stakeholder/citizen priority or
         demand
       • Stakeholder/citizen satisfaction
         Results h
       • R lt shown
       • City Council and Management
         priorities
                                                53




   V 2.2. Tallahassee Citizen
         Centric Report
We have a responsibility to inform our citizens
 about:
   What we are responsible for doing
   Where the money comes from that runs the City
    and where it goes
   What we have accomplished with monies
    What
    received and expended , and
   What challenges face the City moving forward
We believe informed citizens make for better
 government                                     54




                                                     27
 V. 2.2. Purpose of the Citizen
         Centric Report
• To Demonstrate:
   – Transparency
   – Accountability
• To Promote
   – Dialog
   – Two way communication
             y
• To Build Trust
   – Over time
   – One citizen at a time

                                  55




                                       28
29
  V. 2.2. Issuance of Citizen Centric
    Reports and Media Coverage
Four Citizen Centric Reports have been issued
Received front page newspaper coverage
Received television coverage
Report page 3 data verified by City Auditor
Citizen groups have received reports and have
 been requested to provide audit topic
 suggestions
Reports are available in hard copy and online
                                             60




                                                  30
V. 3. Conduct Performance and
Measurement Audits

   Audit departmental and program
    performance
   Audit relevance and reliability of
    performance measures




                                                              61




         V.3.1. Auditing Government
                 Performance
Measure or assess performance during an audit or other study
based on authoritative auditing standards. (See Austin, Florida
OPPAGA, Kansas City, Phoenix on www.AuditorRoles.org)

     — Identify the program’s inputs, processes, outputs, and outcomes
     — Develop and implement “ad hoc” performance measurement system
     — Using performance expectations as “criteria” and measures as
       “condition,” analyze program performance
     — Identify causes of variances and develop audit recommendations



                        www.AuditorRoles.org                  62




                                                                         31
                                        Available at:
                                        www.theiia.org/bookstor




    V.3.2. Self Assess or Audit
   Performance Measures Using
         Asserted Criteria

Relevance—Measures should be aligned,
 complete, and useful

Reliability—Each measure and its data
 should be accurate, valid, and consistent


                 www.AuditorRoles.org            64




                                                                  32
V.3.2. Test relevance or reliability.
Assessing the Relevance of Performance
Measures: Asserted Criteria
Measures should be:
 Aligned         Linked to mission, goals, objectives

 Complete        Includes essential aspects of
                 performance

 Useful          Timely
                 Understandable
                 U d t d bl
                 Comparable
                 Responsive to change
                 Meets broad needs of users

                   www.AuditorRoles.org                  65




V.3.2. Test relevance or reliability.
Assessing the Reliability of Performance
Measures & Data: Asserted Criteria
 Each measure and its data should be:
 Accurate           Computed correctly
                    Neither overstated nor understated
                    Appropriately precise

 Valid              Corresponds to the phenomena reported
                    Correctly defined
                    Data & calculation comply with definition
                    Unbiased

 Consistent         Consistent with previous periods
                    Controlled by adequate systems


                   www.AuditorRoles.org                  66




                                                                33
VI. Performance-Based Decision Making –
Includes Stakeholders (especially citizens),
Elected Officials, Managers, and Employees


   VI. 1. Using performance information
          to support decision making
   VI. 2. Decision making under greater
          budget
          b d t constraints
                     t i t



                                          67




VI. 1. Performance Information Used
for Different Decisions


                   Budgetary
                 Decision making


              Manage & Improve
                 Operations


               Accountability
                 Reporting
                                          68




                                               34
VI. 1. Performance Information
Supports Decisions to:
  Assess and adjust program performance
       i l l         d
   service levels, and resources,
  improve existing programs and services,
  improve internal management systems,
  revise performance plans and reports,
  initiate new programs and services, and
  bottom-line—use performance information
   to support continuous improvement and
   public accountability.                 69




 V. 2. Strategic Performance Budget Decision
                     Model
    HIGH                                Target for            Strategic Success
                                    Increased Funding           in Achieving
                                                               Community or
                                                             Program Outcomes
                        ce
      Strategic Importanc




                                       Target for           Useful Contributor to
                                      Funding Cuts          Government Success




    LOW
                             POOR            Performance Results                 GOOD

                             Use of Strategic Goals and Performance Results in
                             Prince William County Budget Decisions


                                          www.AuditorRoles.org                          70




                                                                                             35
   VI. 2. Strategic Performance Budget Decision
                       Model:
     In Reality: Varies with Fiscal Environment
        HIGH                                Target for              Strategic Success
                                        Increased Funding             in Achieving
                                                                  Community or Program
                                                                       Outcomes
                            e
         Strategic Importance


                                                                      Less funding available
                                                                                    Useful
                                                                             Contributor to
                                             Target for                       Government
                                            Funding Cuts                           Success
                                            f     g
                                       More funding available


        LOW
                                POOR                Performance Results                 GOOD
                                Use of Strategic Goals and Performance Results in
                                Prince William County Budget Decisions

                                           www.AuditorRoles.org                                71




                                         Exercise 4
Brainstorm ways to encourage government managers and
policymakers to use performance information when making
decisions:

   Political leaders
   Policy staff
   Legislators and legislative staff
   Program administrators
   Service providers
   Consumers of services
   Press and public
                                           www.AuditorRoles.org                                72




                                                                                                    36
             Conclusion
What did Austin learn and where is the
 City going?

What are some of the “best practices”
 f
 from various sources?



                                          73




VII. 1.1 Conclusion: Creating a
Managing for Results Culture for Austin

   S       ti       Vi i
     Supporting our Vision ...
    Creating A Results Orientation:
     Services, Activities and Programs
    Creating Accountability: Measures and
     Indicators
    Creating Integration: Making it Happen
     at the Operational Level


                                          74




                                               37
VII. 1.2 Lessons Learned In Austin

   “Bottom-Up” approach neglected broad
    performance areas and alignment
       • Department key indicators
   Accounting structure is a major hurdle
   Definition of “services” not clear
   Results orientation difficult when
    template not used effectively
   Poor use of template = poor measures
   The “not something I control” syndrome
                                               75




VII. 1.3 Where is Austin going…
   Assessing and Improving the Relevance and
    Reliability of Reported Measures
     • Data Collection Infrastructure
     • Certification Program
     • Citizen-Centric Report
   Providing Further Training to Re-enforce
    Cultural Shift in All Departments
      • Using Information in Management
      • Using Information in Operations
   Passing and Implementing a “Best Practice”
    Performance Accountability Ordinance
                                               76




                                                    38
VII. 2.1 Performance Management and
Accountability: Best Practices Checklist
  Obtain active participation by top-level managers and
   decision makers
  Create a clear vision of why and how performance
   measures will be used internally and externally
  Understand the limits of performance measures what
   they can and cannot do
  Sustain organizational commitment over a long
   period despite barriers and the potential for bad news.
      i d d it b i           d th    t ti l f b d
  Integrate the performance measurement and reporting
   system with organizational planning, service delivery,
   and decision making systems
                                                         77




 VII. 2.2 Performance Management and
 Accountability: Best Practices Checklist
Through planning align mission, goals,
 objectives/targets,
 objectives/targets and measures
Design goals and objectives/targets that specify
 a single aspect of performance
Design aggressive yet realistic goals and
 objectives/targets that encourage progress
 b     d         f
 beyond past performance l llevels
Involve employees, customers, and stakeholders
 in developing goals, objectives/targets, and
 measures
                                                         78




                                                              39
 VIII. 2.3 Performance Management and
 Accountability: Best Practices Checklist

Identify programs and activities to be measured
 Identify
 and define them through and input-process-
 output-outcome model.
 Design a “family of measures” for each
 program which provides key information to
 support decisions
Periodically
P i di ll evaluate current performance
                 l                f
 measure; change when needed but try for
 comparability over time

                                               79




 VII. 2.4 Performance Management and
 Accountability: Best Practices Checklist
Define each measure and identify data sources
 Define
 and data collection procedures
Produce performance information (including
 explanatory information) which is clear and
 useful to all stakeholders (customize)
Educate, encourage, and reward managers for
 using performance information to make
   i       f         i f      i         k
 decisions which improve program management
 and service delivery

                                               80




                                                    40
     Training/Assistance to Get You
          From Here to There
EGAPP Inc. provides training in all
EGAPP, Inc
 aspects of performance management and
 auditing. (Brochure Available)
Auditor Roles Project provides training
 in assessing/auditing performance
 management systems and measures.
 Assistance can also be arranged.
Email to Steve: egappmorgan@yahoo.com
                                       81




              Thank You.
More questions
      questions.

More comments.

Thank you, again.

82




                                            41

								
To top