Key Developments in Management Accounting by sdfwerte


									Research Excellence Framework

       Position at October 2009

                 David Otley

 Distinguished Professor of Accounting & Management
       Lancaster University Management School

        Chair Main Panel I for RAE 2008

                            REF October 2009          1
Issues with RAE 2008

   This is seen by HEFCE / government as:
       Too burdensome
          Mainly for them and university administrations
          But also for the panels

       Too incestuous
          Captured by academics
          Not enough attention given to impact

       Problematic funding outcomes for old 5*s
   ‘A few simple metrics’ should replace it

                                REF October 2009            2
Other issues with RAE 2008

   Past sell-by date
       Burden on panels to produce profiles
       Game-playing helped by early publication of
        rules and procedures
       Late introduction of sub-profile reporting
       Strong advice not to use citation data
       B&M panel over-stretched

    ‘Never again’ as concluding motto

                              REF October 2009        3
The REF – likely ground rules

   Much very similar to RAE
       Work associated with individuals
       Selective inclusion of staff by universities
       4 (or perhaps 3) research outputs
       Profiles will be used
       Much the same panel structure
            Although B&M (and A&F) need re-organization
       Panels will exercise judgment, informed by
        other data

                                 REF October 2009          4
REF consultation

   Consultation document now issued for
    response by mid-Dec 2009
       Informed by Expert Advisory Groups
   Much already non-negotiable
       Need to be acceptable to government
       Context of likely reduced public expenditure
   But some important changes, and issues
    for subject groups to respond to

                              REF October 2009         5
    Government Policy
   A strong and innovative national research base is
    essential to support national prosperity in a
    globalised knowledge based economy
   Need to strengthen links between investment in
    research and the economic and social benefits it
        Creating new businesses and improving performance
        Developing new products and services
        Highly skilled and educated workforce
        Improving public services and public policy

(Science and innovation investment framework

                                       REF October 2009      6
      The REF framework
  Outputs                 Impact                    Environment

  Quality of outputs:                                 Quality of the
  assessed through a                                  research
  combination of                                      environment:
  bibliometrics and                                   assessed through
  expert review                                       narrative and

                    Impact of              Engagement with
                    research:              users: assessed
                    assessed through       through narrative
                    a portfolio of         and indicators

Weights: 60%                      25%                          15%

                             REF October 2009                            7
Three components
   Outputs
       Much as before but supplemented with
        bibliometric information if recommended
       Need for careful benchmarking
       Need for adequate database
       Maybe only 3 items
       New 4* definition
   Environment
       Much as before, but now includes ‘esteem’ and

                              REF October 2009          8

   New area of ‘impact’
       Includes ‘impact’ but ‘engagement with users’
        now under ‘environment’
       Has to be based on research work, but current
        period impact can be based on prior period
       Not necessarily specific papers, more a body
        of work in a department
       Assessed by ‘case examples’

                             REF October 2009           9
Other issues

   Profiles
       Often dichotomised with single point alternative
        but continuous scale possible (e.g. 2.4)
       Same for all three areas?
       Need for point definitions
   Funding
       Gradient of funding line (less in 2008), but
        contentious especially with reductions likely

                               REF October 2009         10
    Incentive effects                                [Van der Stede, 2007]

Effort   “Productive “ effort

                                “Perverse” effort

                                          Incentive strength

                                  REF October 2009                           11

   Need to respond to consultation
   Role of bibliometric data
   Information requirements
   Assessment of impact
   Determination of panel composition
   Funding principles (economies of
    concentration ???)

                     REF October 2009    12
Citation issues

   Different discipline features
       Average citation rates
       Citation lags and half-life
       Databases variable quality & coverage
   Issues for attention
       Inclusion of comments with papers?
       Provision of comparative information
       Use in esteem measures

                          REF October 2009      13
Bibliographic issues

   Use of journal „lists‟
   Impact factors
       Imputes quality of journal to paper
       Maybe ok in aggregate?
   Individual panels may well differ in
       Allocation of people to subject panels?

                           REF October 2009       14

To top