Performance Measurement Tools for Justice Information Technology by yurtgc548

VIEWS: 1 PAGES: 24

									Performance Measurement Tools
     for Justice Information
       Technology Projects




               Facilitated by:
    Center for Society & Law & Justice
                  Faculty

                                         1
Performance Measurement Tools
     for Justice IT Projects
       A CSLJ ongoing project
            Funded by the
      Bureau of Justice Assistance




         Grant # 2002LD-BX K002


                                     2
Project Background and Purpose




  This project is designed to identify and validate
      an inventory of performance measures
   appropriate for justice information technology
  projects and develop field-friendly performance
                 measurement tools.
                                                      3
              Fact:
There is an increasing focus on
measuring performance in both
  government and industry.




                                  4
              Fact:
 The demand for performance
measures in law enforcement and
criminal justice settings is high.




                                     5
             Question

Why do you think there is currently
 a high demand for performance
 measurement in law enforcement
  and criminal justice settings?


                                      6
Some Reasons Why There is a High
Demand for Performance Measures

 Performance measurement is
  increasingly being mandated
 Tight budgets

 There is a need to justify expenditures




                                            7
            Problem

  Even though the demand for
  performance measures in law
enforcement and criminal justice
   settings is high, technology
 integration projects are not in
advanced stages of performance
          measurement.

                                   8
          Question

 What are some of the unique
challenges or “road blocks” to
 performance measurement?



                                 9
     Some Unique Challenges or
    “Road Blocks” to Performance
           Measurement
   Many people are unfamiliar with
    performance measurement or reluctant to
    use it.
   Measurements are imposed from the
    outside.
   Most measures are not outcome oriented.
   Existing measures are not sufficient.

                                              10
     A Definition of Performance
              Measures
Performance Measures: a particular value or
    characteristic used to objectively measure
    results. This means using valid and
    reliable indicators.

Valid: Do they truly measure what they are
     intended to measure?
Reliable: Do they consistently do so ?
    (Time A, Time B, Time C, etc.)
                                             11
        A Useful Acronym

Valid performance measures are
 S pecific
 M easurable
 A ccountable
 R esults-Oriented
 T imebound
                                 12
              The Two “O’s”
                   of
        Performance Measurement
Performance measurement
  can be defined as the
  process of routinely
  measuring the outputs and
  outcomes produced by a
  project, thereby allowing
  one to assess the
  effectiveness of project
  investments and activities.
                                  13
       Outputs and Outcomes:
     The Logic Model Approach to
      Performance Measurement

    “A program logic model … provides a
  roadmap of your program, highlighting how
   it is expected to work, what activities need
     to come before others, and how desired
             outcomes are achieved.”

Source: WK Kellogg Foundation Evaluation Handbook (1998)

                                                           14
           The Logic Model Approach


  INPUTS         ACTIVITIES       OUTPUTS         OUTCOMES


INPUTS are your resources

OUTPUTS are the product of an activity

OUTCOMES are the consequences of the program/initiative
efforts -- changes in conditions, attitudes or behavior of
individuals or outcomes for agencies and communities, not what
the program or initiative itself does.

                                                          15
             The Logic Model Approach
 INPUTS         ACTIVITIES        OUTPUTS
Example:         Example:        More Rapid
                  Mobile              ID         OUTCOMES
  Money,
Personnel,         Data          of Suspects
Hardware,       Terminals in
 Software       Patrol Cars
                                                  Increased
INPUTS are your resources                         public
                                                  safety

OUTPUTS are the product of an activity

OUTCOMES are the consequences of the program/initiative
efforts -- changes in conditions, attitudes or behavior of
individuals or outcomes for agencies and communities, not what
the program or initiative itself does.
                                                          16
          The Logic Model Approach


 INPUTS         ACTIVITIES        OUTPUTS        OUTCOMES




When creating performance measures, you can use the
Strategic Plan and the Theory of Change to link together the:

          •Inputs
          •Outputs
          •Outcomes


                                                          17
    The Logic Model Approach Helps You
               to Define Your


   A theory of change is a statement of how
    your program intends to proceed from
    initial outputs to produce long-term
    outcomes.
   Once you articulate and reach consensus on
    your theory of change, you can then move
    on to identify those vital, few performance
    measures that are aligned with your
    organization’s mission, goals and objectives.
                                                18
         Thinking Through a Logic Model and
           Articulating a Theory of Change

  Goals        Activities       Targets       Theory             Initial/      Long
                                                of              Short &        Term
                                              Change         Intermediate    Outcomes
In order to                                                      Term
   solve       We will do       For these                     Outcomes
                                             This activity
   these      the following    people and                                    We will know
                                             will lead to
 Problems     activities and     for this                                      we are
                                             changes in
                 Events        amount of                                      reaching
                                            these factors     We will know
                                   time      __________,                      our goals
                                                             these changes        if:
                                            which in turn    have occurred   ___________
                                             will lead to          if:
                                            solving these     ___________
                                              problems




                                                                                  19
        A Logic Model and Theory of Change
                     Example

  Goals        Activities       Targets       Theory             Initial/      Long
                                                of              Short &        Term
                                              Change         Intermediate    Outcomes
In order to                                                      Term
   solve       We will do       For these                     Outcomes
                                             This activity
   these      the following    people and                                    We will know
                                             will lead to
 Problems     activities and     for this                                      we are
                                             changes in
                 Events        amount of                                      reaching
                                            these factors     We will know
                                   time      __________,                      our goals
                                                             these changes        if:
                                            which in turn    have occurred   ___________
                                             will lead to          if:
                                            solving these     ___________
                                              problems




                                                                                  20
    Developing a Theory of Change
Recommended Process for Surfacing &
Articulating a Theory of Change
    Use a Focus Group Format
    Start with Long-term Outcomes
    Work Backwards Toward Initial Activities
    Map Required Existing Resources
    Reconcile Multiple Theories of Change


Adapted from J.P. Connell et al., 1995, “New Approaches to
Evaluating Community Initiatives.” Aspen Institute.

                                                             21
   The Logic Model/Theory of Change Method
     for Developing Performance Measures
                        Program Logic Model and Chain of Events
 Category     Program          Initial    Intermediate    Intermediate          Final
             Feature and     Outcomes      Outcomes        Outcomes        Outcomes/Goals
               Activity                                         II          Accomplished




Measures:   1._______       1._______    1._______       1._______       1._______
            2._______       2._______    2._______       2._______       2._______
            3._______       3._______    3._______       3._______       3._______
            4._______       4._______    4._______       4._______       4._______


                                                                                      22
             Summary:
       Why Measure Performance?

   To monitor project implementation
   Because IT projects are high-risk
   To demonstrate improvements
   To correct problems and make adjustments
   To ensure accountability
   To illustrate progress and justify additional
    funding for your integration effort

                                                23
        Some Suggested Steps…
1.   Invest in Performance Measurement
2.   Carefully select and form a team
3.   Identify the Business Process to be mapped
4.   Specify the logic model using the Theory of
     Change method
5.   Decide the Audience/Level of the Measures
6.   Keep in mind that the Model should be
     Plausible, Doable and Testable
7.   Reconcile conflicting assumptions among
     stakeholders
8.   Revise and Refine model and corresponding
     measures as needed                            24

								
To top