The Blueprint

Document Sample
The Blueprint Powered By Docstoc
					Integrating Performance Measures
     into University Endeavor

                Victor M. H. Borden, Ph.D.
                       Associate Vice President
  University Planning, Institutional Research, and Accountability (IU)
              Associate Professor of Psychology (IUPUI)
                               Or

Becoming an Evidence-Driven
   Learning Organization

              Victor M. H. Borden, Ph.D.
                     Associate Vice President
University Planning, Institutional Research, and Accountability (IU)
            Associate Professor of Psychology (IUPUI)
                                Or

 How I Learned to Stop Worrying
and Love Performance Measures

               Victor M. H. Borden, Ph.D.
                      Associate Vice President
 University Planning, Institutional Research, and Accountability (IU)
             Associate Professor of Psychology (IUPUI)
      Integrating Performance Measures into University Endeavor




If this were a simple matter, you
would have figured it out long ago and
I wouldn’t be here. Do not expect my
explanations to be simple nor my
advice to be straightforward.
This will be more like a graduate-level
seminar than an introductory course


      Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




  The Institutional Research Credo

I realize that I will not succeed in answering all of
your questions. Indeed, I will not answer any of
them completely. The answers I provide will only
serve to raise a whole new set of questions that lead
to more problems, some of which you weren’t aware
of in the first place. When my work is complete, you
will be as confused as ever, but hopefully, you will
be confused on a higher level and about more
important things

           Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




     Why Not “Data-Driven?”
 Data,   per se, are not what we need




           Victor M. H. Borden – February 9, 2006 – Ohio University
                           Integrating Performance Measures into University Endeavor

NMPSID     Tracked     DBIRTH     Age     GENDER     RACE   MARSTAT     SATV    SATM   ACTC HSCTYP   HSGPA
 AS0022       1      05/24/1962   43.3       1         1        2         -9      -9    -9    0.00     -9
 AS0023       0      09/09/1954   51.0       1         3        1         -9      -9    -9   -9.00     -9
 AS0024       1      07/02/1982   23.2       0         6        1        530     530    20    1.00     -9
 AS0201       1          8/9/76    26        1         6       -9         -9      -9    -9   -9.00     -9
 AS0202       1         8/17/83    19        0         6        1         -9      -9    -9    1.00     -9
 AS0203       1         11/9/70    32        1         6       -9         -9      -9    -9    1.00     -9
 AS0204       1         8/26/74    28        1         1        1         -9      -9    20    1.00     -9
 AS0205       1         3/26/68    34        0         6       -9        520     460    -9    1.00     -9
 AS0206       1         8/31/83    19        1         6        1         -9      -9    -9    1.00     -9
 AS0207       1          9/8/76    26        1         6       -9         -9      -9    -9   -9.00     -9
 AS0208       1         11/6/55    47        1         6       -9         -9      -9    -9      0      -9
 AS0209       0        1/7/1985   17.6       0         3       -9         -9      -9    -9   -9.00     -9
 AS0210       0       3/19/1977   25.5       1         6        1         -9      -9    20      1      -9
 AS0211       1      11/15/1971   30.8       1         3       -9         -9      -9    -9     -9      -9
 AS0212       0       6/29/1971   31.2       1         1       -9         -9      -9    -9      0      -9
 AS0213       0                    -9       -9         7       -9         -9      -9    -9     -9      -9
 AS0214       0       7/5/1956    46.2       1         6       -9         -9      -9    -9   -9.00     -9
 AS0215       1      10/21/1960   41.9       1         1        4         -9      -9    -9   -9.00     -9
 AS0216       0                    -9       -9         7       -9         -9      -9    -9   -9.00     -9
 AS0217       0       7/23/1975   27.1       1         3        1         -9      -9    -9   -9.00     -9
 AS0218       0        7/6/1972   30.2       1         6        1         -9      -9    -9     -9      -9
 AS0219       1       2/23/1977   25.5       1         4        1         -9      -9    -9      1      -9
 AS0220       0        2/5/1983   19.6       1         6       -9         -9      -9    -9      1      -9
 AS0221       1       8/15/1967    35        1         2       -9         -9      -9    -9     -9      -9
AS031282      0      11/21/1961   41.8       1        -9       -9         -9      -9    -9     -9      -9
AS031284      0       7/21/1962   41.1       1        -9       -9         -9      -9    -9     -9      -9
AS031288      0       3/17/1982   21.5       0        -9       -9         -9      -9    -9     -9      -9
AS031310      0        9/2/1974    29        1         3       -9         -9      -9    -9     -9      -9
AS031327      0       5/30/1975   28.3       1         3       -9         -9      -9    -9     -9      -9
AS031328      0       7/30/1968   35.1       1         3       -9         -9      -9    -9      1     2.47

                           Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




 If Not Data-Driven, Then What?
 Evidence-based             practice to decide…
   What     to do
   How best to do it
   If it is working as desired

 So that we can learn from what we do and
  improve
 We want to be part of a Learning Organization


           Victor M. H. Borden – February 9, 2006 – Ohio University
        Integrating Performance Measures into University Endeavor




     Learning Organizations
 …organizations   where people continually
 expand their capacity to create the results
 they truly desire, where new and
 expansive patterns of thinking are
 nurtured, where collective aspiration is set
 free, and where people are continually
 learning to see the whole together.
 (Senge, 1990)

        Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




     Learning Organizations
 …are  characterized by total employee
 involvement in a process of collaboratively
 conducted, collectively accountable
 change directed towards shared values or
 principles. (Watkins and Marsick 1992)




         Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




                       Overview
 Lessons I’ve learned (the hard way) about
  developing university performance
  measures
 Performance measures as the “tip of the
  evidence-based iceberg”
   Going   below the surface
         an organizational learning lens
 Applying
 Some implications and related thoughts

         Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor




            Lessons Learned
 Earlylessons on measurement theory
 1994 NDIR Volume
 Measuring Institutional Performance
  Outcomes (APQC-MIPO)
 Developing campus PIs to link planning,
  budgeting, evaluation and improvement
 Taking it to the next level


          Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Measurement Theory
Inductive – Deductive Cycle




Victor M. H. Borden – February 9, 2006 – Ohio University
                  Integrating Performance Measures into University Endeavor




               Measurement Theory
   Validity
       Warranted assertion (Dewey)
       Degree to which the measure accurately represents the concept
        (what you are attempting to measure)
         • Size of a person (weight, height, circumference, body mass)
         • Quality of instruction (course ratings, peer review, student
           learning)?
   Reliability
       Degree to which measure faithfully represents the concept
         • Course ratings taken mid-term/end-term
   Unless very careful attention is paid to one’s theoretical
    assumptions and conceptual apparatus, no array of
    statistical techniques will suffice – Blalock, 1982

                  Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor




          1994 NDIR Volume
 UsingPerformance Indicators to Guide
 Strategic Decision Making (Borden and
 Banta, Eds.)




          Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




                        Lessons
 Borden and Bottrill: Where you stand on PIs
  depends on where you sit
 Ewell and Jones: Think before you count
 Joengblood and Westerheijden (Europe): PIs
  out, Quality Assurance in
 Dorris and Teeter (TQM): PIs are fine, if P
  stands for Process
 Dolence and Norris: KPIs are the fuel of a
  strategic decision engine
 DeHayes and Lovrinic (ABC): Show me the
  money…and what you use it for doing.
         Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




            Lessons (continued)
 Banta     and Borden Criteria for Effective PIs
   Start with purpose
   Align throughout organization
   Align across input, process, output
   Coordinate a variety of methods
   Use in decision making




            Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor


       Measuring Institutional
       Performance Outcomes
   American Productivity and Quality
 An
 Center (APQC) benchmarking study




         Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




          APQC MIPO Findings
 The best institutional performance measures
  communicate the institution’s core values
 Good institutional performance measures are carefully
  chosen, reviewed frequently, and point to action to be
  taken on results
 External requirements and pressures can be extremely
  useful as starting points for developing institutional
  performance measurement systems
 Performance measures are best used as “problem
  detectors” to identify areas for management attention
  and further exploration
 Clear linkages between performance measures and
  resource allocation are critical, but the best linkages are
  indirect (and non-punitive)
            Victor M. H. Borden – February 9, 2006 – Ohio University
             Integrating Performance Measures into University Endeavor




                       MIPO Cont.
   Performance measures must be publicly available,
    visible, and consistent across the organization
   Performance measures are best considered in the
    context of a wider transformation of organizational
    culture
   Organizational cultures supportive of performance
    measures take time to develop, require considerable
    “socialization” of the organization’s members, and are
    enhanced by stable leadership
   Performance measures change the role of managers
    and the ways in which they manage


             Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




        MIPO – Boiling it Down
 You  cannot ‘lead’ with performance
  measures
 Performance measures emerge from a
  broader culture of evidence, that is, they
  are part of something bigger




         Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
                     www.iport.iupui.edu



  E.G.: PIs@IUPUI




Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
 Taking it to the Next Level:
  Accountability at Indiana
          University


Articulating and Attaining Strategic
       Goals and Objectives
           Integrating Performance Measures into University Endeavor




                       Audiences
 Board    of Trustees
   Most   comprehensive, University-wide view
 Campus      accreditors and (prospective)
 partners
   Campus-specific             objectives and indicators
 Targeted      packaging for…
   Media;legislators; alumni; current and
   prospective students and their parents;
   research agencies and collaborators
           Victor M. H. Borden – February 9, 2006 – Ohio University
              Integrating Performance Measures into University Endeavor




                            Purposes
 Position IU strategically
 Improve the effectiveness and quality of
  programs and services
 Provide a common framework to align efforts
  across campuses
 Communicate a clear and consistent message
  about IU’s broad goals
 Enhance IU’s image
       Define and document IU’s contributions to the state,
        students, and communities
       Demonstrate integrity in accounting for the use of
        public and private resources
              Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




                      Principles
 Mission-centered
 Research-driven
 Transparency
 Inclusive   dimensions of excellence and
  quality
 Empowerment and responsibility
 Influenced by “best practices”
   NationalCommission on Accountability in
    Higher Education
         Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




                    Framework
 University-wide   strategic goals and core
  performance indicators
 Campus performance objectives and
  indicators derived from mission, aligned to
  university goals and core indicators
 Explicit link to administrative area goals
  and objectives
 Annual performance reports and reviews
   University    and campuses
         Victor M. H. Borden – February 9, 2006 – Ohio University
Advance University   Rankings and recognitions      Expand the Scope        Funding
Distinction and       Integrating Performance Measures into University Endeavor
                                                   and Impact of
                                                                            Research collaborations
Distinctiveness      Focused areas of distinction   Research and
                                                    Creative Activities     Faculty
                     Centers of Excellence                                  participation/productivity
                     Overall campus quality                                 Space and equipment

                                                                            Academic Impact
Enhance Academic     Quality of faculty
Program Quality                                                             Practical Impact
                     Program accreditation and
                     review                         Advancing Indiana       Economic development and
                                                                            impact
                     Teaching and learning
                     development                                            Cultural development and
                                                                            impact
                     Information/technology                                 Educational development
                     resources
                                                                            Indiana professional practice:
                     Physical resources
                                                                            Preparation and service
                     Program demand and                                     Civic engagement
                     delivery
                                                    Increase                Finances and budgeting
Improve Student      Preparation and support
                                                    Operational
Achievement and                                                             Enrollment
                     Access and affordability       Efficiency and
Success
                                                    Effectiveness           Leadership development
                     Student engagement
                                                                            Administrative overhead
                     Progress
                                                                          Quality of administrative
                     Outcomes                                             services to
                      Victor M. H. Borden – February 9, 2006 – Ohio University
                                                                          Faculty/staff/student
         Integrating Performance Measures into University Endeavor




Limitations of Measures/Metrics
 Inherentlyimperfect
 Overly simplistic
 Not everything that counts can be
  counted, and not everything that can be
  counted counts – Albert Einstein




         Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




Accommodating the Limitations
 An  imprecise answer to the right question
  is much better than a precise answer to
  the wrong question (paraphrasing John Tukey)
 Triangulation
   Using  multiple, convergent measures to better
    reflect the underlying
 Performance measures as the tip of the
  evidence-based iceberg

            Victor M. H. Borden – February 9, 2006 – Ohio University
                                     Integrating Performance Measures into University Endeavor


Performance Measures as the Tip of the
   Evidence-Based Practice Iceberg
 Vertical (hierarchical) alignment

                                                                                 Performance
                                                                                  measures


                                                                                            Evidence
                                                                                          Based Practice


                                                           Plan
                                     Improve                           Implement
                                                         Assess
                                          Horizontal February 9, 2006 – Ohio University
                                     Victor M. H. Borden –(cross-unit) alignment
          Integrating Performance Measures into University Endeavor




     Evidence-Based Practice
 Commonly        used in clinical domain
   Validityderived from rigorous research
    conducted by others and believed to
    generalize to other settings
 For university endeavor there are limits to
  generalizability across settings
 Focus shifts to more continuous use of
  process-generated data using less
  rigorous methods to monitor, reflect, and
  adjust
          Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




Methods of Evidence-Based Practice
 Themany faces of evidence-based
 practice
   Student  learning outcomes assessment
   Program evaluation
   Program review
   Quality improvement
   Balanced score card
   Benchmarking
 The   role of collaborative inquiry
           Victor M. H. Borden – February 9, 2006 – Ohio University
                                                         Integrating Performance Measures into University Endeavor




                                                        The Evaluation Cycle
1. THINK ABOUT
   ISSUES                                                     2. ENGAGE WITH THE
                                                                 PROBLEM
                                                                                                                3. DEVELOP
                            Back to the drawing board
On to something else




                                                                                                                   RESOURCES/
                                                                                                                   STRATEGIES
                                                                                                                   TO IMPROVE
                                                                 6. PLAN TO
                                                                    IMPROVE

                                                                                             4. IMPLEMENT
                       5. EVALUATE IMPACT                                                       INTERVENTIONS
                       * did it work as I intended?                                            * experiment
                       * how did people respond?
                       * what were the results?
                                                                                             Adapted from
                                                         Victor M. H. Borden – February 9, 2006 – Ohio University Norman   Jackson
              Integrating Performance Measures into University Endeavor




           The Assessment Matrix
    1.          2.         3.       4.                            5.          6.
  What        How      How are How could                        What        What
 general   would you you trying    you                        have you    changes
outcome    know it if to achieve measure                       found?      will you
 are you    you saw       the    what you                                  make?
seeking?       it?    outcome?   hope to
                                  see?




              Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
             Integrating Performance Measures into University Endeavor




        The Support Unit Matrix
    1.          2.          3.                    4.         5.         6.
  What       Why is it  What are             How will      How do   Based on
objective   important you doing             you know you know evidence
 are you      to the   to achieve           when you       you are   from #4
trying to   University     it?                  are        doing it   and #5
achieve?        ?                           achieving efficiently? what can
                                                   it               you do to
                                           (effectiveness)          improve?
                                                  ?




             Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




  Quality Improvement Models
 Advantages
   Focus   on process provides best chances for
    identifying points of improvement
   Collaborative teams empower staff and help
    improve communication across units
   Formulaic method and external staff support
    help guide and keep on track
 Sample     methods
   Penn  State’s Fast Track
   U of Wisconsin Accelerated Improvement
           Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




    PSU Fast Track




Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor



UWisc Accelerated Improvement
 http://www.wisc.edu/improve/improvement/accel.html

Define        Goals and measures of success
              Document process
              Understand customer needs
              Check/refine goals

Design        Develop potential solutions
              Analyze solutions/options
              Finalize solution develop implementation plan

Implement     Inform affected people
              Conduct training, if needed
              Execute action plans w/timeline

Follow-up     Collect data to track improvement
              Review and refine process changes
              Issue final report with results
         Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor




             Program Review
        self-study, site visit by “peers”
 Program
 Common method for academic programs
   Increasing     use for administrative programs
 Fitswell with accreditation framework
 Guidelines shape tone and tenor
   Content standards
   Review team composition
            accommodates range of inquiry
 Flexibility
  orientations
          Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




    Limits of Program Review
 Expensive and time-consuming
 Can be done with little participation
   Or   with a lot
 Results    not always directly useful for
  change
   Memorandum             of understanding helpful
 Episodicnature not responsive to
  changing environment
           Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




  Balanced Score Card (BSC)
 Kaplan   & Norton propose business model
   Financialperformance
   Customer service and satisfaction
   Process effectiveness and efficiency
   Organizational learning




           Victor M. H. Borden – February 9, 2006 – Ohio University
               Integrating Performance Measures into University Endeavor




         BSC in Higher Education
   Ruben (1999)
       Teaching/Learning
         • Programs/Courses, Student Outcomes
       Service/Outreach
         • University, profession, alumns, state, prospective students,
           families employers
       Scholarship/Research
         • Productivity/Impact
       Workplace satisfaction
         • Faculty/staff
       Financial
         • Revenues/expenditures

               Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
              Integrating Performance Measures into University Endeavor




                    Benchmarking
 Best practices in organizations sharing similar
  internal work procedures
 HE focus often on peer or aspirational
  institutions
 NACUBO study searched for measures
 APQC introduces qualitative benchmarking to
  higher education
       Measuring institutional performance outcomes
       Electronically supported student services

              Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor




Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor




        More Complex Models
 The   Evaluation Center
  Stufflebeam, Eastern Michigan University
  http://www.wmich.edu/evalctr/checklists/
   CIPP Model
   Constructivist Evaluation
   Deliberative Democratic Evaluation
   Key Evaluation Checklist
   Qualitative Evaluation
   Utilization-Focused Evaluation
          Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




    Limits of Complex Models
 Too complex and expensive to be practical
 They require an…
   “evaluation unit as a staff operation at a high
   level of the organization in order to help
   insulate the unit from inappropriate internal
   influences and enhance its influence on
   decision making .”
                                                   Daniel J. Stufflebeam


    http://www.wmich.edu/evalctr/checklists/institutionalizingeval.htm
            Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor




    Collaborative Action Inquiry
 Continuous cycle of data collection  data
  analysis  data feedback  action plans 
  data collection
 Stakeholder empowerment through active and
  on-going participation
 Data feedback meetings promote collaboration,
  dialogue, and collective analysis
 Active learning and discovery fostered by critical
  reflection process
 Data-driven action plans developed = research
  linked to action

          Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




  Linking Research and Action
 Who   does what?
   Decides  what actions are taken?
   Is responsible for effective implementation?
   Can devise appropriate evaluation protocols?
   Has access to or can collect appropriate
    evidence?
   Reviews the results and decides what to do?

 What can be done to get these people to
 work together and in concert?
         Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




       A Learning Paradigm
 Typical  data-driven focus supposes
  rational world
 Learning incorporates uncertainty,
  ambiguity, and multiple styles
 Individual learning and organizational
  learning are compatible concepts
 Evidence-based practice is compatible
  with learning approach
            Victor M. H. Borden – February 9, 2006 – Ohio University
              Integrating Performance Measures into University Endeavor




 Single- and Double-Loop Learning
 Argyris and Schön
 Learning is the detection and correction of error
  (unintended consequences)
 “Governing Variables” are those things what we
  feel are important to keep within limits
 “Action Strategy” is what we do or plan to do to
  keep the governing variables within limits
 “Consequences” are the intended and
  unintended outputs and outcomes
       Intended: confirm our theory in use
       Unintended: suggests error in our theory in use

              Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




       Single-Loop Learning
 Governing      variables not called into
  question
 Adjustments made to action strategies at
  best
 Defense mechanisms can readily arise to
  maintain single-loop learning
     Governing                 Action                        Conse-
     Variables               Strategies                      quences

         Victor M. H. Borden – February 9, 2006 – Ohio University
              Integrating Performance Measures into University Endeavor




           Double-Loop Learning
 Questioning the role of the framing and learning
  systems which underlie actual goals and
  strategies
 Reflection is fundamental
       Basic assumptions are confronted
       Hypotheses publicly tested
       Falsification is sought
       Ego is laid aside

         Governing                   Action                       Conse-
         Variables                 Strategies                     quences
              Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




   Model I and II Org Learning
 Single- and double-loop learning at the
  organizational level
 Model I: Organizational members
  prescribe to a common theory in use
   Organizational           policies and practices inhibit
    change
 Model II: Governing values, policies, and
 practices promote double-loop learning

            Victor M. H. Borden – February 9, 2006 – Ohio University
John Seely Brown & Paul Duguid
The Social Life of Information
(2000) Harvard Business School Press




Organizational Learning and Communities-of-Practice:
Toward a Unified View of Working, Learning, and
Innovation. (1991) Organization Science, 2(1), 40-57.
          Integrating Performance Measures into University Endeavor




  Learning To Be / Know How
 Based   on collaborative practice
   Communities          of practice
 Knowledge        as inseparable from the
  knower
 Evidence from a variety of sources,
  including practitioner experience
 Sharing interpretations as process
 Common priorities and strategies as
  output
          Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




           Learning is Good
 We  promote (lifelong) learning for students
 We seek to contribute to the creation of
  knowledge within our disciplines and
  professions
 What about in our practice as...
   Classroom  teachers
   Conferrers of degree credentials
   Managers and administrators
   Support staff

         Victor M. H. Borden – February 9, 2006 – Ohio University
          Integrating Performance Measures into University Endeavor


      The Learning/Performance
        Measure Conundrum
 If our general objective is to collectively learn
  how to do our work better, then we must accept
  that our current thinking, practices, structures,
  etc., need to change
 Our current best thinking about what measures
  reflect progress toward desired changes may
  change through the learning process
 We should not be rigid about our performance
  measures but rather allow our evidence-based
  collaborative learning efforts to guide their
  evolution
          Victor M. H. Borden – February 9, 2006 – Ohio University
           Integrating Performance Measures into University Endeavor




Implications for Faculty/Staff/Org Development

 There   are many viable ways to integrate
  inquiry into organizational practices
 Administrative support focus may need to
  shift from information provision and toward
  collaborative inquiry
 Someone needs to focus on how this all
  fits together
    Theinstitutional portfolio provides one such
     mechanism
           Victor M. H. Borden – February 9, 2006 – Ohio University
         Integrating Performance Measures into University Endeavor




Implications for Information Use
 Data sources
 Types of needs
 Types of users
 Sources of information
 Tools for user needs




         Victor M. H. Borden – February 9, 2006 – Ohio University
                Integrating Performance Measures into University Endeavor




                        Data Sources
   Sources of evidence
       Documented
       Provider/practitioner experience
       User/client experience
       Contextual
   Derived from Institution’s operational information
    systems
       Student, Human Resource, Finances
       Space, program inventory, courseware
   Surveys
       Students, faculty, staff, prospects, community
   External data sources
       Federal and State (K-16) education data, national efforts (CDS,
        rewards and recognitions, media)
       Census, labor, workforce development, licensing boards
                Victor M. H. Borden – February 9, 2006 – Ohio University
                Integrating Performance Measures into University Endeavor




     Types of Information Needs
   Operational
       Directly support the ongoing operation of a system
       Formatted presentations of transactional
       Often use data from a single operational domain
   Tactical
       Monitor and respond quickly to a variety of short-term situations
       Typically more aggregate (less granular) than operational reports
       Includes both recurrent and ad hoc information needs
       Often requires merging data from multiple operational domains
        as well as data from non-operational sources
   Strategic
       Focuses on higher level policy and practice issues, often with
        longer timeframes
       Often requires more significant analysis of institutional, survey,
        and external data sources
                Victor M. H. Borden – February 9, 2006 – Ohio University
               Integrating Performance Measures into University Endeavor




                          User Roles
   Casual
       occasional use that demands relatively little technical
        expertise
   Recurrent
       more frequent use but modest technical expertise OR
        insufficient time to employ technical skills
   Power
       modest to frequent use with capacity for using more
        complex technical systems
   Individuals may occupy different roles at
    different times
              Victor M. H. Borden – February 9, 2006 – Ohio University
                     Integrating Performance Measures into University Endeavor




        Information Needs and Users
                                                    Type of User

                                    Casual              Recurrent                Power

              Operational       Pre-packaged          Report Modules ODBC Access to
Type of Use




                                 Operational          With Parameter Data Warehouse
                                   Reports               Choices          Tables

              Tactical
                                                         Web-based
                                Research Briefs                                  OLAP
                                                          Report
                                 And Analyses                                    Tools
              Strategic                                  Generators

                     Victor M. H. Borden – February 9, 2006 – Ohio University
               Integrating Performance Measures into University Endeavor




                 Implications for IT
   Analytic data warehouse is essential, but
   Think more broadly about data sources
       Not just enterprise system as we now know it
       Data from courseware platform
       Mechanisms for collecting “droppings” from other
        important activities
         • Faculty vitae and annual reports
         • Portfolios of faculty and student work
         • Civic engagement inventory
   Access/reporting technology should focus on
    enabling “value-added resellers” to deliver to
    broad range of users

               Victor M. H. Borden – February 9, 2006 – Ohio University
            Integrating Performance Measures into University Endeavor




Responsibility-Centered Budgeting
 Similar   to Churchill’s opinion of democracy
   Ithas been said that democracy is the worst
    form of government except all the others that
    have been tried
 Concerns        about changing to RCB
   It changes everything and yet nothing really
    changes
   [I] have known a great many troubles, but
    most of them never happened ~Mark Twain
            Victor M. H. Borden – February 9, 2006 – Ohio University
        Integrating Performance Measures into University Endeavor




            Parting Thought
 Itis good to have an end to journey
  towards; but it is the journey that
  matters in the end - Ursula Le Guin




        Victor M. H. Borden – February 9, 2006 – Ohio University

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:1/5/2013
language:Unknown
pages:75