Final Report - Review of the National Indigenous Reform Agreement by Wk87dH9l

VIEWS: 5 PAGES: 38

									               Review of the

National Indigenous Reform Agreement

      Performance Framework



            NOVEMBER 2011


                   FINAL REPORT

     authored by the NIRA Review Working Group

              As at 2 November 2011
Table of Contents

EXECUTIVE SUMMARY                                                                                             1
  Table 1: Current and proposed indicator sets                                                                 2

Recommendations                                                                                                4


OVERVIEW                                                                                                      6

BACKGROUND                                                                                                    7

The National Indigenous Reform Agreement                                                                       7

The COAG Reform Council (CRC)                                                                                  8

Heads of Treasuries (HoTs) Review and Treasury’s analysis using the HoTs conceptual framework                  8

The Commonwealth Treasury initial analysis                                                                     9


REVIEW OF EACH INDICATOR AGAINST THE CLOSING THE GAP TARGET                                                   10

Health Performance Indicator Set                                                                              10
  TARGET ONE: Close the gap in life expectancy between Indigenous and non-Indigenous Australians by
      2031                                                                                                    10
  Performance Indicator 1 – Estimated life expectancy at birth                                                10
  Performance Indicator 2 – Mortality rate (and excess deaths) by leading cause                               11
  Performance Indicator 3 – Hospitalisation rates by principal diagnosis                                      11
  Performance Indicator 4 – Rates of current daily smokers                                                    11
  Performance Indicator 5 – Average daily alcohol consumption and associated risk levels; rates of alcohol
      consumption at long term risk to high risk levels                                                       11
  Performance Indicator 6 – Levels of obesity – Body mass index                                               11
  Performance Indicator 7 – Level of physical activity                                                        12
  Performance Indicator 8 – Access to health care compared to need                                            12
  TARGET TWO: Halve the gap in mortality rates for Indigenous children under five by 2018                     12
  Performance Indicator 9 – Child under five mortality rate (and excess deaths)                               13
  Performance Indicator 10 – Mortality rates (and excess deaths) by leading cause (for children under five)   13
  Performance Indicator 11 – Child under five hospitalisation rates by principal diagnosis                    13
  Performance Indicator 12 – Proportion of babies born of low birth weight                                    14
  Performance Indicator 13 – Tobacco smoking during pregnancy                                                 14
  Performance Indicator 14 – Antenatal care                                                                   14

Education Performance Indicator Set                                                                        14
  TARGET THREE: Ensure access to early childhood education for all Indigenous four year olds in remote
      communities by 2013                                                                                  14
  Performance Indicator 17 – The proportion of Indigenous children who are enrolled in (and attending, where
      possible to measure) a preschool program in the year before formal schooling                         14
  TARGET FOUR: Halve the gap in reading, writing and numeracy achievement for Indigenous children by
      2018                                                                                                 15
  Performance Indicator 15 – Percentage of students at or above the national minimum standard in reading,
      writing and numeracy for Years 3, 5, 7 and 9                                                           15
  Performance Indicator 16 – Rates of participation in NAPLAN reading, writing and numeracy tests – Years 3,
      5, 7 and 9                                                                                             15
  TARGET FIVE: Halve the gap in Year 12 or equivalent attainment rates for Indigenous young people by
      2020                                                                                                   15
  Performance Indicator 18 – Proportion of 20-24 year olds having attained at least a Year 12 or equivalent or
      AQF Certificate II (or above)                                                                          15
  Performance Indicator 19 – Apparent retention rates from Year 7-8 to year 10 and to year 12                16
  Performance Indicator 20 – Attendance rates for Year 1 to 10                                               16

Employment Performance Indicator Set                                                                      16
  TARGET SIX: Halve the gap in employment outcomes between Indigenous and non-Indigenous Australians
      by 2018                                                                                             16
  Performance Indicator 21 – Employment to population ratio for the working age population                16
  Performance Indicator 22 – Unemployment rate                                                            17
  Performance Indicator 23 – Labour force participation rate                                              17
  Performance Indicator 24 – CDEP participations and off CDEP job placement                               17
  Performance Indicator 25 – Three month employment outcomes (post program monitoring)                    17
  Performance Indicator 26 – Proportion of Indigenous 18-24 year olds engaged in full-time employment,
      education or training at or above Certificate III                                                   18
  Performance Indicator 27 – Proportion of Indigenous 20-64 year olds with or working towards post school
      qualification in Certificate III or above                                                           18
  Table 2: Revised Indicator Set                                                                          18


SUMMARY                                                                                                     21

OTHER ISSUES RAISED AT THE WORKSHOP                                                                         21

Attachment A - Working Group Terms of Reference                                                             22

Attachment B - Terminology                                                                                  29

Attachment C – List of the 6 NIRA Targets and 27 performance indicators                                     31

Attachment D - Write up from the CRC Recommendations                                                        32
  CRC Findings - NIRA Performance Report for 2008-09                                                        32
  CRC Findings - NIRA Performance Report for 2009-10                                                        33

Glossary                                                                                                    34
Executive Summary
The National Indigenous Reform Agreement (NIRA) contains 27 performance indicators which are
reported on by the COAG Reform Council (CRC) to demonstrate Government performance against the
six Closing the Gap targets in Overcoming Indigenous Disadvantage.

Many of these indicators have significant data limitations which impact on performance reporting.
In some instances data are available infrequently, while in others, data are subject to quality issues.
However, all jurisdictions are committed to improving data quality through the NIRA and a number of
significant improvements have already been made.

In February 2011 the Council of Australian Governments (COAG) agreed to review the six National
Agreements (NAs) under the Intergovernmental Agreement on Federal Financial Reform (IGA FFR),
including the NIRA. In April 2011 a NIRA Review Working Group was established to review each of the
27 indicators against the Heads of Treasuries (HoTs) Conceptual Framework for Performance Reporting
(the conceptual framework). The Framework was designed to assess the conceptual and data
robustness of each performance indicator under existing agreements, including the NIRA, and guide the
development of performance indicators in future agreements.

The NIRA Review Working Group recommends that a revised set of revised performance indicators form
the basis of the NIRA performance reporting framework, noting that these changes will not take place
until the 2011-12 reporting period. The revised indicator set represents a balance between conceptually
adequate indicators with acceptable data and forms a performance reporting system that encourages
more direct public accountability for performance against the Closing the Gap targets. Jurisdictional
performance against the revised indicator set, across the Commonwealth, State and Territory
Governments, will be more representative and reflective of progress towards the goal of Closing the Gap
in Indigenous disadvantage.

Table 1 on the following page provides both the current and proposed set of indicators and allows a
comparison across the previous indicator set and the recommended indicator set. The recommendations
for consideration by COAG are provided after this table.




                                                                                               1|Page
Table 1: Current and proposed indicator sets
    Target                       Current Performance Indicator set                                Modification             Proposed performance indicator set
    TARGET ONE: Close the gap    PI 1 – Estimated life expectancy at birth                        Nil – Keep               PI 1 – Estimated life expectancy at birth
    in life expectancy between
                                 PI 2 – Mortality rate (and excess deaths) by leading cause       Remove excess deaths     PI 2 – Mortality rate by leading cause
    Indigenous and
    non-Indigenous Australians   PI 3 – Hospitalisation rates by principal diagnosis              Remove
    by 2031
                                 PI 4 – Rates of current daily smokers                            Nil – Keep               PI 3 – Rates of current daily smokers
                                 PI 5 – Average daily alcohol consumption and associated risk     Change title             PI 4 – Levels of risky alcohol consumption
                                 levels; rates of alcohol consumption at long term risk to high
                                 risk levels
                                 PI 6 – Levels of obesity – Body mass index                       Change title and measure PI 5 – Prevalence of overweight and obesity
                                                                                                  to include obesity,
                                                                                                  overweight, normal and
                                                                                                  under weight
                                 PI 7 – Level of physical activity                                Remove
                                 PI 8 – Access to health care compared to need                    Remove
    TARGET TWO: Halve the gap    PI 9 – Child under five mortality rate (and excess deaths)       Merge into new PI 6      PI 6 – Under five mortality rate by leading cause
    in mortality rates for                                                                        and remove excess
    Indigenous children under                                                                     deaths
    five by 2018
                                 PI 10 – Mortality rates (and excess deaths) by leading cause     Merge into new PI 6
                                 (for children under five)                                        above
                                 PI 11 – Child under five hospitalisation rates by principle      Remove
                                 diagnosis
                                 PI 12 – Proportion of babies born of low birth weight            Nil – Keep               PI 7 – Proportion of babies born of low birth weight
                                 PI 13 – Tobacco smoking during pregnancy                         Nil - Keep               PI 8 – Tobacco smoking during pregnancy
                                 PI 14 – Antenatal care                                           Nil – Keep               PI 9 – Antenatal care



                                                                                                                                                                   2|Page
TARGET THREE: Ensure           PI 17 – The proportion of Indigenous children who are            Nil - Keep                PI 10 – The proportion of Indigenous children who
access to early childhood      enrolled in (and attending, where possible to measure) a                                   are enrolled in (and attending, where possible to
education for all Indigenous   preschool program in the year before formal schooling                                      measure) a preschool program in the year before
four year olds in remote                                                                                                  formal schooling
communities by 2013
TARGET FOUR: Halve the         PI 15 – Percentage of students at or above the national          Nil - Keep                PI 11 – Percentage of students at or above the
gap in reading, writing and    minimum standard in reading, writing and numeracy for Years                                national minimum standard in reading, writing and
numeracy achievement for       3, 5, 7 and 9                                                                              numeracy for years 3, 5, 7 and 9
Indigenous children by 2018
                               PI 16 – Rates of participation in NAPLAN reading, writing and    Keep as a supporting      PI 11 measure 1: Rates of participation in NAPLAN
                               numeracy tests – Years 3, 5, 7 and 9                             measure under P11         reading, writing and numeracy tests – years 3, 5, 7
                                                                                                                          and 9
TARGET FIVE: Halve the gap     PI 18 – Proportion of 20-24 year olds having attained at least   Keep as a supporting      PI 12 – Attainment of Year 12 or equivalent
in Year 12 or equivalent       a Year 12 or equivalent or AQF Certificate II                    measure under new
                                                                                                                          PI 12 measure 1: Proportion of 20-24 year olds
attainment rates for                                                                            PI12
                                                                                                                          having attained at least a Year 12 or equivalent AQF
Indigenous young people by
                                                                                                                          Certificate II level (or above)
2020
                               PI 19 – Apparent retention rates from Year 7-8 to year 10 and    Replace with              PI12 measure 2: Year 12 certification
                               to year 12                                                       certification data as a
                                                                                                new measure under
                                                                                                new PI12
                               PI 20 – Attendance rates Year 1 to 10                            Nil - Keep                PI 13 – Attendance rates Year 1 to 10




                                                                                                                                                                  3|Page
TARGET SIX: Halve the gap    PI 21 – Employment to population ratio for the working age        New PI 14              PI 14 – Level of workforce participation
in employment outcomes       population
between Indigenous and
non-Indigenous Australians   PI 22 – Unemployment rate                                         Keep as a supporting   PI 14 measure 1: Employment to population ratio for
by 2018                                                                                        measure under new      the working age population
                                                                                               PI14
                             PI 23 – Labour force participation rate                           Keep as a supporting   PI 14 measure 2: Unemployment Rate
                                                                                               measure under new
                                                                                               PI14
                                                                                                                      PI 14 measure 3: Labour force participation rate
                                                                                               Keep as a supporting
                                                                                               measure under new
                                                                                               PI14
                             PI 24 – CDEP participations and off CDEP job placement            Remove
                             PI 25 – Three month employment outcomes (post program             Remove
                             monitoring)
                             PI 26 – Proportion of Indigenous 18-24 year olds engaged in       Remove
                             full time employment, education or training at or above
                             Certificate III
                             PI 27 – Proportion of Indigenous 20-64 year olds with or          Nil - Keep             PI 15 – Proportion of Indigenous 20-64 year olds
                             working towards post school qualification in Certificate III or                          with or working towards post school qualification in
                             above                                                                                    AQF Certificate III level or above




                                                                                                                                                           4|Page
Recommendations
It is recommended that the Steering Group recommend to COAG:

1. That NIRA performance indicators be rationalised.

    a.    retain, with no changes, performance indicators 1, 4, 12, 13, 14, 15, 17, 20 and 27 with a note
          that further work is currently underway to improve the data for performance indicator 17, with
          the data set to improve iteratively from 2012 onwards;

    b.    delete performance indicators 3, 7, 8, 9, 10, 11, 16 18, 19, 21, 22, 23, 24, 25 and 26 with
          indicators 9, 10, 18, 19, 21, 22 and 23 to be consolidated as recommended at 1c;

    c.    note performance indicator 16 is to be removed and included as a supporting measure to
          performance indicator 15 ‘Percentage of students at or above the national minimum standard
          in reading, writing and numeracy for years 3,5,7 and 9’;

    d.    add three new improved indicators incorporating some of the previous indicators:

          i. ‘Under 5 mortality rate by leading cause’. This new indicator incorporates performance
             indicators 9 ‘Child under 5 mortality rate (and excess deaths)’ and 10 ‘Mortality rates (and
             excess deaths) by leading cause (for children under 5)’;

         ii. ‘Attainment of year 12 or equivalent’. This new indicator incorporates two measures
             ‘Proportion of 20-24 year olds having attained at least a year 12 or equivalent AQF
             Certificate II’ and administrative data on the proportion of young people who have attained
             a Year 12 or equivalent certificate (noting that further work is required to improve
             certification data. Refer to Recommendation 4); and

         iii. ‘Level of workforce participation’. This new indicator incorporates three supporting
              measures, these being performance indicators: 21 ‘Employment to population ratio for the
              working age population’; 22 ‘Unemployment rate’; and 23 ‘Labour force participation rate’.

    e.    modify;

                      i. performance indicator 2 (‘mortality rate [and excess deaths] by leading cause’)
                         to no longer report on excess deaths;

                      ii. performance indicator 5 (from ‘Average daily alcohol consumption and
                          associated risk levels’) to ‘Levels of risky alcohol consumption’; and

                     iii. performance indicator 6 (from ‘Levels of obesity – body mass index’) to
                          ‘Prevalence of overweight and obesity’;

2. That by December 2012, the Working Group on Indigenous Reform (WGIR), in consultation with the
   Standing Council on Health, will review the indicator on 'access to health care compared to need' in
   order to determine its re-inclusion in the NIRA;

                                                                                                  4|Page
3. That, subject to a cost benefit analysis, the survey questions used in the National Aboriginal and
   Torres Strait Islander Health Survey (NATSIHS) be included in the National Aboriginal and Torres
   Strait Islander Social Survey (NATSISS) for reporting against performance indicators 5 and 6. This will
   provide a three yearly reporting cycle for these indicators;

4. That further work be done to develop a robust annual proxy indicator using existing administrative
   data to enhance measurement of progress under the employment target;

5. Data on Year 12 or equivalent attainment should be considered as part of the review of the National
   Education Agreement (NEA). It may be possible to make greater use of administrative data on Year
   12 certification to measure progress against the Year 12 or equivalent target noting that
   considerable work on Year 12 certification data is required;

6. Refer the performance indicators and data sets requiring further developmental effort to the NIRA
   Performance Information Management Group (PIMG) work plan for inclusion and action in
   consultation with relevant Standing Councils;

7. Note that recommended changes to the NIRA performance framework will take effect in the
   2011-12 reporting period; and

8. Note that at its February 2011 meeting, COAG reaffirmed its ongoing commitment to the Closing the
   Gap agenda and agreed to its inclusion as one of the ‘five themes of strategic importance that lie at
   the intersection of jurisdictional responsibilities’ for focus of COAG effort over the next three years.
   To achieve this objective all NAs and NPs should consider how well they address the Closing the Gap
   targets and associated performance outcomes.




                                                                                                5|Page
Overview
At its meeting of 13 February 2011, COAG agreed that the underlying reform principles of the IGA FFR
continues to provide a strong foundation for progressing COAG’s agreed reform agenda and achieving
better policy and service delivery outcomes for all Australians. COAG also agreed to take forward the
key recommendations of the HoTs review and reports from the CRC and consider improvements to the
governance and performance reporting frameworks under the six NAs.

Following this, the HoTs Review and CRC Recommendations Implementation Steering Group (the
Steering Group) was established. The Steering Group tasked separate Working Groups, comprising
senior officials from First Ministers’, Treasuries and select Portfolio Agencies, to undertake reviews of
the performance reporting frameworks of the NAs.

In April 2011 the NIRA Review Working Group (the Working Group) was established to review the
performance reporting issues identified by the HoTs review and reports of the CRC by 31 July 2011. The
Working Group was asked to undertake a review of the NIRA performance framework using the
conceptual framework developed by the HoTs Review (www.federalfinancialrelations.gov.au/). The
Working Group was also asked to consider the performance data currently collected through the NIRA
and what data and reporting is required to demonstrate progress. The terms of reference governing the
Working Group are at Attachment A.

The Steering Group also advised the Working Group that:
     a reduction in the number of indicators is a critical part of the review and it should be aiming for
       the minimum number necessary, ideally 10-15, and
     there is a need for cost-benefit analysis of the recommended indicators.

In its deliberations, the Working Group agreed to reduce the number of indicators only if this was
supported by clear rationale, strengthened reporting outcomes and improved public accountability.

The Working Group had its first meeting via tele-presence on 31 May 2011 to finalise the terms of
reference (Attachment A) and develop a forward work plan to meet the tight timeframes. Following
this, a face-to-face workshop was convened in Sydney on 24 June 2011 where members discussed each
of the 27 performance indicators against the agreed principles of conceptual adequacy, data limitations,
driving performance and public accountability.

Draft recommendations from the review process were distributed to all jurisdictions and a
tele-presence held on 3 August 2011 to finalise the recommendations. Following further consideration,
the revised indicator set was finalised at a tele-presence on 13 September 2011.

The final report from the Working Group was submitted to the Steering Group on 12 October 2011.

A terminology paper for the review of the NIRA performance framework is at Attachment B. Although
this paper notes the technical differences in the definitions used, the majority of these terms are derived
directly from the IGA FFR.

                                                                                                 6|Page
Background
The National Indigenous Reform Agreement

In 2007-08, COAG agreed to a number of ambitious targets to Close the Gap in Indigenous disadvantage
by improving outcomes between Indigenous and non-Indigenous Australians in the areas of life
expectancy, health, education and employment. The NIRA sets out this policy framework and provides
links to those NAs and National Partnership Agreements (NPs) which include elements aimed at Closing
the Gap in Indigenous disadvantage.

The NIRA sets out six targets and seven strategic platforms or building blocks which support the targets.
The targets are:
    1. close the gap in life expectancy between Indigenous and non-Indigenous Australians by 2031;
    2. halve the gap in mortality rates for Indigenous children under five by 2018;
    3. ensure access to early childhood education for all Indigenous four year olds in remote
        communities by 2013;
    4. halve the gap in reading, writing and numeracy achievement for Indigenous children by 2018;
    5. halve the gap in Year 12 or equivalent attainment rates for Indigenous young people by 2020;
        and
    6. halve the gap in employment outcomes between Indigenous and non-Indigenous Australians by
        2018.

The seven building blocks address specific areas of Indigenous disadvantage which are recognised as
supporting the reforms aimed at the closing the gap targets including: early childhood; schooling; health;
economic participation; healthy homes; safe communities; and governance and leadership. It should be
noted that while the healthy homes, safe communities and governance and leadership building blocks
do not currently have any associated NIRA performance indicators against them, in most instances these
have performance indicators under NPs.

The NIRA currently contains 27 performance indicators (Attachment C) to measure progress against the
six targets. The NIRA Indicators are based on the conceptual framework and data specifications
contained and agreed in the original NIRA developed in 2009 and the NIRA Refresh in February 2011. It
should be emphasised that the 27 NIRA indicators were largely based on already collected data and
statistics, mostly from the Australian Bureau of Statistics (ABS) and Australian Institute of Health and
Welfare (AIHW). They do not generally impose additional need for collection of information by agencies
or by states and territories.

The NIRA, through Schedule F Agreed Data Quality Improvements, commits jurisdictions and data
agencies to a range of data improvements. The Australian Government has committed $46.4 million
over four years from 2009-10 to address data issues related to the NIRA. This investment will help build
a better evidence base against which to measure progress. Data quality improvements will lead to




                                                                                               7|Page
better estimates of mortality and life expectancy through improved identification of Indigenous people
in the Population Census and in health administrative datasets.

At its February 2011 meeting, COAG reaffirmed its ongoing commitment to the Closing the Gap agenda
and agreed to its inclusion as one of the ‘five themes of strategic importance that lie at the intersection
of jurisdictional responsibilities’ for focus of COAG effort over the next three years.

The COAG Reform Council (CRC)

The CRC has the role of assessing and publicly reporting on the performance of governments against the
objectives, outcomes and indicators in the NAs, including the NIRA. It is also tasked with annually
assessing whether there has been genuine improvement against each of the Closing the Gap targets.
The CRC has authored two NIRA performance reports based on performance in 2008-09 and 2009-10.
The CRC made recommendations in both reports, which where relevant have been considered as part of
this review (Attachment D).

In these reports, the CRC highlighted concerns relating to data limitations and proposed
recommendations to improve the performance reporting framework. The CRC concluded that some
data is reported infrequently, such as through the Population Census on a five year reporting cycle. In
some instances, this makes it difficult to measure annual progress. For this reason the NIRA includes a
range of additional indicators that allow for progress to be assessed in the years in which the target
cannot be directly measured. In other cases, data quality issues preclude meaningful comparisons over
time and between jurisdictions.

The CRC concluded that the establishment of targets is a strength of the NIRA and the NIRA provides a
comprehensive framework to measure change across the key areas of disadvantage experienced by
Indigenous Australians. However, the CRC has stated that given the data limitations and the likely pace
of change, it may be some years before they are able to make a comprehensive assessment of the
performance of governments against these targets. The CRC notes that data development is a priority
under the NIRA and is specifically addressed at Schedule F of the NIRA, as part of the Agreed Data
Quality Improvements.

Heads of Treasuries (HoTs) Review and Treasury’s analysis using the HoTs
conceptual framework

In December 2009, COAG requested that HoTs undertake a review of the implementation of the IGA FFR.
In considering the effectiveness of the performance frameworks of the existing NAs and NPs, the HoTs
Review identified two broad issues affecting performance indicators, namely data limitations and
conceptual inadequacy.

The review identified that data limitations include data that: is of poor quality, is unreliable or
infrequent; is not comparable over time or between jurisdictions; and cannot be sufficiently
disaggregated by Indigenous or socio-economic status. It concluded that indicators with data limitations


                                                                                                 8|Page
are problematic where they do not provide sufficient information to allow an assessment of progress
against the outcomes of the agreement.

In relation to conceptual adequacy, the HoTs Review concluded that to be a useful measure, a
performance indicator should be clearly and logically linked to the outcome and these links should be
evident to the general public. In addition, a performance indicator must detect a real change in the
outcome over time, and a shared understanding of the direction of change that represents progress is
required.

The Commonwealth Treasury initial analysis

Consistent with the Working Group Terms of Reference, the first stage of the NIRA Working Group
review was a preliminary analysis of the NIRA’s 27 performance indicators using the HoTs conceptual
framework. The Commonwealth Treasury undertook this analysis, and assessed the conceptual and
data robustness of each indicator and the role that each indicator plays in measuring the targets of the
NIRA.
The analysis identified that there were strengths and weaknesses associated with the current set of
performance indicators. It identified three broad categories of indicators:
     headline or outcome performance indicators;
     progress measures; and
     indicators that are neither headline indicators or strong progress measures (as they do not
       provide strong support to the broader headline indicators).

Headline (or outcome) indicators directly measure performance towards achieving the target while
progress measures demonstrate relative performance in an area that is critical to achieving the target.

The existence of conceptually strong outcome indicators (such as the headline indicators) would usually
negate the need for additional progress measures. However there are a couple of exceptions to this
general principle. For example, if the outcome indicators are conceptually strong but suffer from
significant data issues, that limit reporting against them, then additional progress measures may be
required to meet the public accountability objective of the IGA FFR.

Another exception would be where the outcome indicator is a long-term measure such as estimated life
expectancy, which shows changes over a long timeframe but may only show small year-on-year
variations in this outcome. A good progress measure therefore should demonstrate a strong conceptual
link with the target, and provide timely, accurate, reliable and regular information that supports an
associated outcome performance indicator.

The Treasury analysis concluded that headline (or outcome) indicators exist for five out of the six Closing
the Gap targets. For the target ‘ensure all Indigenous four year olds in remote communities have access
to early childhood education within five years’, there is currently no data of sufficient quality to measure
progress, however, data should be available from 2012.




                                                                                                 9|Page
The analysis further identified that there are a range of progress measures that assist in evaluating
jurisdictions’ performance against the targets, for example where data limitations prevent sufficient
reporting against the headline indicators. However, the analysis also identified some performance
indicators that suffer from data limitations and/or conceptual issues and therefore do not provide
adequate measures of performance.

Review of each indicator against the Closing the Gap target
A total of 27 performance indicators are currently reported under the NIRA performance framework.
This section summarises the conclusions reached by the Working Group and provides the supporting
analysis to the recommendations made in this report. As discussed above the key considerations include
whether the performance indicators are conceptually sound, provide evidence to drive performance
(requires sufficient data), have sound policy justification and support public accountability of reporting.

A number of the performance indicators that this review has recommended for removal from the NIRA
performance framework are subject to data improvements under Schedule F of the NIRA and have
attracted Commonwealth funding for improvement. The fact that an indicator has been dropped from
the NIRA does not mean that these data are not important nor does it mean that these data should no
longer be collected. In this context it is important to note that there is a range of performance reporting
on government service delivery and activity outside of the IGA FFR which provides transparency and
public accountability on the level of government service provision. Performance reporting under the IGA
FFR should complement these activities rather than duplicate reporting efforts with the primary focus of
demonstrating measurement of progress towards the outcomes of agreements.

In some instances consideration could be given to reincorporating indicators into the NIRA once the
quality of underlying data has been enhanced. All jurisdictions should be consulted through the NIRA
PIMG and the WGIR before any further changes are made to the indicator set.

In this regard it should also be noted that the NIRA, like other NAs, is a living document that is subject to
enhancement over time to reflect additions and changes to existing and new NAs and NPs. As COAG
agrees to additional reforms to Close the Gap in Indigenous disadvantage, these will also be reflected in
the NIRA.

Health Performance Indicator Set

TARGET ONE: Close the gap in life expectancy between Indigenous and
non-Indigenous Australians by 2031

Performance Indicator 1 – Estimated life expectancy at birth

It is recommended that this indicator be retained as a direct measure of the target. This indicator is
considered a strong outcome indicator that demonstrates strong conceptual links to the target. As this
performance indicator relies on life expectancy estimates, progress can only be directly measured once
every five years. Considering the small year-on-year change in the life expectancy measure, there is


                                                                                                 10 | P a g e
likely to be little benefit from measuring this indicator more frequently. Therefore, performance
indicators 2, 4, 5 and 6 are considered appropriate progress measures that support this indicator.

Performance Indicator 2 – Mortality rate (and excess deaths) by leading cause

It is recommended that this indicator be retained. This indicator is considered a strong progress
measure which supports performance indicator 1 and has only minor data limitations.

Reporting on excess deaths should be removed. Data on excess deaths are age standardised using an
indirect method that does not allow for time series analysis. Given the importance of measuring change
over time for the NIRA, data on excess deaths does not add value to the performance framework. The
concept of excess deaths is not readily understood by the public.

The recommended change to the indicator does not affect the data collection and will not result in
additional cost to any parties.

Performance Indicator 3 – Hospitalisation rates by principal diagnosis

It is recommended that this indicator be removed. This indicator does not strongly support the broader
headline indicator as it is difficult to determine what direction equates to progress when changes are
measured (i.e. whether an increase in hospitalisation rates reflects an increase in the occurrence of
illnesses and conditions requiring inpatient hospital treatment; or an increase in access to and use of
hospital inpatient treatment by people with such conditions).

Performance Indicator 4 – Rates of current daily smokers

It is recommended that this indicator be retained. This is considered an adequate progress measure that
supports performance indicator 1. Smoking is estimated to contribute to 17% of the gap in health
outcomes between Indigenous and non-Indigenous Australians. The health impact of smoking is evident
in the incidence of a number of key chronic diseases including ischemic heart disease, chronic
obstructive pulmonary disease and cancer. Data is available three yearly through the ABS survey
program and this is considered sufficient given the slow pace of change in health risk factors and also the
long term nature of the 25 year life expectancy gap.

Performance Indicator 5 – Average daily alcohol consumption and associated risk
levels; rates of alcohol consumption at long term risk to high risk levels

It is recommended that the title of this indicator be changed to ‘levels of risky alcohol consumption’ to
improve public interpretation. With this change this indicator is considered an adequate progress
measure that supports performance indicator 1. The measurement remains the same.

Performance Indicator 6 – Levels of obesity – Body mass index

It is recommended that this indicator be retained but changed to ‘Prevalence of overweight and obesity’.
In reporting this new indicator the prevalence of overweight, underweight, normal weight and obesity


                                                                                               11 | P a g e
should be reported separately in addition to the combined prevalence, as is done in the Aboriginal and
Torres Strait Islander Health Performance Framework (HPF). Obesity is estimated to contribute to 16%
of the gap in health outcomes between Indigenous and non-Indigenous Australians. With these changes
this indicator is considered an adequate progress measure that supports performance indicator 1. For
reporting data should be split by obesity, overweight, normal and under weight.

Subject to a cost benefit analysis, it is recommended that associated questions for performance
indicator 5 and 6 be standardised across and included in both the NATSIHS and the NATSISS. This would
provide data on a rotating three yearly cycle across the two Indigenous data collections, rather than the
current six year gap between reports. It is important to have a more regular indicator of change
regarding significant risk factors that negatively affect life expectancy. The cost benefit analysis could
also consider if the questions regarding alcohol consumption should be consistent in the National Drug
Strategy Household Survey report.

Performance Indicator 7 – Level of physical activity

It is recommended that this indicator be removed. While the level of physical activity is an important
contributor to the health gap between Indigenous and non Indigenous Australians, this indicator was not
judged to be as important as some other indicators under the life expectancy target.

Performance Indicator 8 – Access to health care compared to need

It is recommended that this indicator be removed because of conceptual inadequacies. Work needs to
be done to better measure both ‘access’ and ‘need’. Noting the importance of this indicator (the gap in
health system performance is linked to life expectancy outcomes), it is recommended that the data
development work be included in Schedule F of the NIRA and put onto the NIRA PIMG work plan. The
NIRA PIMG should develop options for a better measure of this concept.

The NIRA PIMG should build on work that is currently underway to provide more appropriate measures
for this indicator. The Department of Health and Ageing (DoHA) and AIHW are developing an enhanced
measure of Access Compared to Need using mortality and hospital data that could allow the indicator to
be reported annually. The inclusion of this indicator in the NIRA could be reconsidered once a
conceptually sound measure is developed, given the importance of appropriate health care for the life
expectancy target.

It is recommended that, by December 2012, the WGIR (in consultation with the Standing Council on
Health) review this indicator in order to determine its re-inclusion in the NIRA.

TARGET TWO: Halve the gap in mortality rates for Indigenous children under
five by 2018

The headline indicators for this target (performance indicators 9 and 10) are not able to be reported for
all jurisdictions due to problems with the quality of the Indigenous identification and the small number
of child deaths in any one year in the smaller jurisdictions. Data for performance indicators 9 and 10 are
currently only able to be reported for NSW, QLD, WA, SA and the NT. Data on under-five mortality for

                                                                                              12 | P a g e
NSW, QLD, WA, SA and the NT are provided annually however the data are combined across five years.
This makes it impossible to track progress from one year to the next.

Due to the small numbers involved annual progress cannot be reliably assessed at the jurisdictional level
due to the inherent volatility associated with small numbers. For the three remaining jurisdictions (VIC,
TAS and the ACT) progress against this target cannot be tracked at all without the addition of progress
indicators as the target cannot be directly monitored again due to small numbers.
Without the retention of some progress measures against this target there will be no ability for
Government to track progress on an annual basis by jurisdiction.

Performance Indicator 9 – Child under five mortality rate (and excess deaths)

It is recommended that performance indicators 9 and 10 be merged to ‘Under five mortality rates by
leading cause’ with reference to excess deaths removed. This will strengthen reporting outcomes. The
concept of excess deaths is not readily understood by the public and does not allow for time series
analysis. These indicators are considered strong outcome indicators that demonstrate strong conceptual
links to the target and as a direct measure of the target. However, while conceptually strong, data for
some progress measures should be retained to track progress on an annual basis for all jurisdictions.

Data for performance indicators 9 and 10 are only currently reported for NSW, QLD, WA, SA and the NT.
For VIC, TAS and the ACT progress against this target cannot be tracked without the addition of progress
measures.

The recommended change to the indicator does not affect the data collection and will not result in
additional cost to amend.

Performance Indicator 10 – Mortality rates (and excess deaths) by leading cause (for
children under five)

As discussed above, performance indicator 9 and 10 should be merged but reporting on excess deaths
ceased and the reference to excess deaths removed. The rationale for no longer reporting excess death
is similar to that under performance indicator 2.

Performance Indicator 11 – Child under five hospitalisation rates by principal
diagnosis

It is recommended that this indicator be removed. This indicator does not strongly support the broader
headline indicators. It is difficult to determine what direction equates to progress when changes are
measured (i.e. whether an increase in hospitalisation rates reflects an increase in the occurrence of
illnesses and conditions requiring inpatient hospital treatment; or an increase in access to and use of
hospital inpatient treatment by people with such conditions).




                                                                                             13 | P a g e
Performance Indicator 12 – Proportion of babies born of low birth weight

It is recommended that this indicator be retained. This indicator is considered an adequate progress
measure to support performance indicators 9 and 10.

Low birth weight (new-born babies weighing less than 2,500 grams) is associated with premature birth
or sub-optimal intrauterine environments (fetal growth retardation). Low birth weight infants are at a
greater risk of dying during the first year of life and are prone to ill health. Other maternal factors that
contribute to low birth weight include smoking and no antenatal care (Middleton 2009).

Performance Indicator 13 – Tobacco smoking during pregnancy

It is recommended that this indicator be retained. This indicator is considered an adequate progress
measure which supports gaps in performance indicators 9 and 10.

Smoking is a risk factor for adverse events in pregnancy, and is associated with poor perinatal outcomes
such as low birth weight (less than 2,500 grams) (AIHW 2004c), pre-term birth, fetal growth restriction,
congenital anomalies and perinatal death.

Performance Indicator 14 – Antenatal care

It is recommended that this indicator be retained, noting that it is an output measure. The link between
antenatal care and under five mortality is sufficiently robust to support its continued inclusion.
Antenatal care provides parents with important education and advice about pregnancy and birth, and
ensures early screening to identify and manage issues likely to impact on the pregnancy and its
outcomes. Inadequate or late access to quality antenatal care has been associated with poor pregnancy
outcomes, such as low birth weight, prematurity and increased delivery intervention.

Education Performance Indicator Set

TARGET THREE: Ensure access to early childhood education for all Indigenous
four year olds in remote communities by 2013

Performance Indicator 17 – The proportion of Indigenous children who are enrolled in
(and attending, where possible to measure) a preschool program in the year before
formal schooling

Data of sufficient quality are not currently available to report against this indicator. Data development
work is underway and it is expected that sufficient quality data will be available from 2012 onward and
available for reporting in 2013.




                                                                                                 14 | P a g e
TARGET FOUR: Halve the gap in reading, writing and numeracy achievement
for Indigenous children by 2018

Performance Indicator 15 – Percentage of students at or above the national minimum
standard in reading, writing and numeracy for Years 3, 5, 7 and 9

It is recommended that this indicator be retained as a measure of the target. This is a strong outcome
indicator that demonstrates strong conceptual links to the target.

Performance Indicator 16 – Rates of participation in NAPLAN reading, writing and
numeracy tests – Years 3, 5, 7 and 9

It is recommended that this indicator be removed from the NIRA performance indicator set, but retained
as a supporting measure to performance indicator 15. This provides critical contextual information and
the data is of high quality. Reporting on these data as a supporting measure, needs to occur to assist in
interpreting trends in performance indicator 15. This is as important, as indicator 15 can be quite
sensitive to changes in the proportion of students who participate in tests.

TARGET FIVE: Halve the gap in Year 12 or equivalent attainment rates for
Indigenous young people by 2020

Performance Indicator 18 – Proportion of 20-24 year olds having attained at least a
Year 12 or equivalent or AQF Certificate II (or above)

It is recommended that this indicator be retained as the direct measure of the target under the new
‘attainment of Year 12 or equivalent’ indicator. The new indicator will also use administrative data on
the proportion of young people who have a Year 12 or equivalent AQF Certificate II level (or above)
certificate as a supporting measure. With these changes the indicator will provide a direct measure of
the target and will be an adequate outcome performance indicator, however, further work to improve
administrative data and to develop appropriate denominators for use with the administrative data
needs to occur.

Measurement of the universal Year 12 or equivalent attainment target should be considered under the
review of the NEA. The current approach to measurement of Year 12 or equivalent attainment based on
self reports of people aged 20-24 years is not ideal as it is affected by an individual’s interpretation of
their educational attainment, interstate mobility and time lags. This means that the relative
performance of jurisdictions can be hard to discern as a substantial number of 20-24 year olds have
completed their education in other jurisdictions, and the attainment could have occurred up to seven
years prior to the time of data collection (as is the case for 24 year olds).

It is recommended that the Steering Group recommend to COAG that data on Year 12 or equivalent
attainment should be reconsidered as part of the review of the NEA recognising work is required to
enable the collection of comparable administrative data on Year 12 certification.

                                                                                              15 | P a g e
In its 2009 NIRA report on the NEA, the CRC also called for investment in improvements to
administrative collections to support national performance reporting, including enhancements to
schools’ administrative data collections for enrolment and attendance information and Year 12
completions administrative data.

Performance Indicator 19 – Apparent retention rates from Year 7-8 to year 10 and to
year 12

It is recommended that this indicator be removed and replaced by administrative data on Year 12 or
equivalent certification once data of sufficient quality is available. The existing performance indicator
suffers significant data limitations and it is recommended that it be replaced with administrative data on
the proportion of young people with a Year 12 or equivalent AQF Certificate II level (or above)
certificates. Given the small numbers of Indigenous enrolments, Indigenous apparent retention rates
are particularly affected by the data limitations of this indicator and are particularly affected by
interstate migration, repeating students and movement between school sectors.

Performance Indicator 20 – Attendance rates for Year 1 to 10

It is recommended that this indicator be retained. It is considered an important progress measure that
supports the target. If students are not regularly attending school during years 1 to 10, they are unlikely
to attain a year 12 certificate or equivalent. School attendance is a critical issue for Indigenous
education particularly in remote areas.

The issue of attendance rates, the ability to disaggregate data (currently not available by geo-location for
all jurisdictions) and compare data across school sectors and across jurisdictions, is already subject to
data quality improvements. This work should continue and should be discussed in the context of the
review of the NEA.

Employment Performance Indicator Set

TARGET SIX: Halve the gap in employment outcomes between Indigenous and
non-Indigenous Australians by 2018

Performance Indicator 21 – Employment to population ratio for the working age
population

It is recommended that this performance indicator be retained as a direct measure of the target and
incorporated under the new performance indicator ‘Level of workforce participation’ as explained below
under performance indicator 23.




                                                                                               16 | P a g e
Performance Indicator 22 – Unemployment rate

It is recommended that this performance indicator be retained as a supporting measure of the target
and incorporated under the new performance indicator ‘Level of workforce participation’ as explained
below under performance indicator 23.

Performance Indicator 23 – Labour force participation rate

It is recommended that this indicator be retained as a supporting measure of the target and
incorporated under the new performance indicator ‘Level of workforce participation’. Data collected
under performance indicators 21 as the direct measure and performance indicators 22 and 23 as
supporting measures will be used under this new outcome performance indicator. The amalgamation
of these measures as support measures under the one indicator will clarify that they are all different
approaches to measuring the one concept. Reporting under the three measures will still occur through
alternative processes.

The recommended change to the indicator does not affect the data collection and will not result in
additional cost to amend.

Performance Indicator 24 – CDEP participations and off CDEP job placement

It is recommended that this indicator be removed. This indicator does not strongly support the target
given the small number of participants.

Performance Indicator 25 – Three month employment outcomes (post program
monitoring)

It is recommended that this indicator be removed. This indicator does not appear to strongly support
the target and has conceptual and data limitations. It also provided only partial information on
employment outcomes for Indigenous Australians.

It is recommended that the Steering Group propose that COAG agree to undertake further work to
develop a robust annual proxy indicator using administrative data to enhance measurement of progress
under the employment target. The incorporation of a proxy indicator would need to consider similar
criteria used to analyse these indicators (i.e. conceptual adequacy, data robustness, overall
appropriateness and proportionality of the performance framework).

Such a measure would likely be developed jointly by the Commonwealth Department of Families,
Housing, Community Services and Indigenous Affairs (FaHCSIA) and the DEEWR and considered by all
jurisdictions and the ABS through the NIRA PIMG and WGIR.




                                                                                            17 | P a g e
Performance Indicator 26 – Proportion of Indigenous 18-24 year olds engaged in
full-time employment, education or training at or above Certificate III

It is recommended that this indicator be removed. This progress indicator demonstrates a relatively
weaker conceptual link to the target, given its relatively limited age grouping, compared with
performance indicator 27. This indicator is also covered under performance indicators 21 to 23 and 27.

Performance Indicator 27 – Proportion of Indigenous 20-64 year olds with or working
towards post school qualification in Certificate III or above

It is recommended that this indicator be retained, noting data limitations. This progress indicator
supports the target providing important information on skilling the population and work readiness.
Evidence suggests that attaining a AQF Certificate III level results in a greater likelihood of transitioning
from education into employment.

Reporting against this performance indicator 27 should be disaggregated to include reporting against
both categories ‘with’ or ‘working towards’ a post school qualification. This issue should be referred to
the review of the National Agreement for Skills and Workforce Development (NASWD) for further
consideration as there may be conceptual and data issues in splitting the indicator in this way.

FINAL REVISED INDICATOR SET

A summary of the final revised indicator set is provided below for ease of reference. The indicators have
also been renumbered. There are a total of 15 recommended indicators in the revised set. The final
revised indicator set represents the balance reached by the Working Group. The set contains
conceptually adequate indicators, with acceptable data and forms a performance reporting system that
encourages more direct public accountability for performance against the Closing the Gap targets.

The reduced number of indicators means that all Governments will be responsible for a smaller, more
focussed indicator set. Jurisdictional performance against the indicator set will now be more
representative and reflective of progress towards the goal of Closing the Gap in Indigenous
disadvantage.

Table 2: Revised Indicator Set

The orange shading indicators that the performance indicator directly measures the target. Where this
indicator has supporting measures, this has been reflected in the table.


Health Target: Close the life expectancy gap within a generation (by 2031)


    Revised PI No. 1                      Estimated life expectancy at birth
                                          (indicator directly measures target)



                                                                                                  18 | P a g e
    Revised PI No. 2                    Mortality rate by leading cause

    Revised PI No. 3                    Rates of current daily smokers

    Revised PI No. 4                    Levels of risky alcohol consumption

    Revised PI No. 5                    Prevalence of overweight and obesity

There are five final performance indicators under the life expectancy target, reduced from eight.


Health Target: Halve the gap in mortality rates for Indigenous children under five within a decade (by
2018)


    Revised PI No. 6                    Under five mortality rate by leading cause
                                        (indicator directly measures target)

    Revised PI No. 7                    Proportion of babies born of low birth weight

    Revised PI No. 8                    Tobacco smoking during pregnancy

    Revised PI No.9                     Antenatal care

There are four final performance indicators under the child mortality target, reduced from six.


Education Target: Ensure all Indigenous four year olds in remote communities have access to early
childhood education (by 2013)


    Revised PI No. 10                   The proportion of Indigenous children who are enrolled in (and
                                        attending, where possible to measure) a preschool program in
                                        the year before formal schooling
                                        (indicator measures progress towards target)

There is one performance indicator against the early childhood education target. The number of
indicators remains unchanged.


Education Target: Halve the gap for Indigenous students in reading writing and numeracy within a
decade (by 2018)


    Revised PI No.11                    Percentage of students at or above the national minimum
                                        standard in reading, writing and numeracy for years 3, 5, 7 and 9


                                                                                             19 | P a g e
                                       (indicator directly measures target)

   Revised PI No. 11, measure 1        Rates of participation in NAPLAN reading, writing and numeracy
                                       tests – Years 3, 5, 7 and 9

There is one performance indicator against the education target, reduced from two. There is one
supporting measure under the indicator.


Education Target: At least halve the gap for Indigenous students in Year 12 attainment or equivalent
attainment rates (by 2020)


   Revised PI No.12                    Attainment of Year 12 or equivalent
                                       (indicator directly measures target)

   Revised PI No. 12, measure 1        Measure: Proportion of 20-24 year olds having attained at least
                                       a Year 12 or equivalent AQF Certificate II level (or above)

   Revised PI No. 12, measure 2        Measure: Year 12 certification

   Revised PI No.13                    Attendance rates - Year 1 to 10

There are two performance indicators against this education target, reduced from three. There are two
supporting measures under the first indicator.


Employment Target: Halve the gap in employment outcomes between Indigenous and non-Indigenous
Australians within a decade (by 2018)


   Revised PI No.14                    Level of work force participation
                                       (indicator directly measures target)

   Revised PI No.14, measure 1         Direct Measure: Employment to population ratio for the working
                                       age population

   Revised PI No.14, measure 2         Supporting Measure: Unemployment rate

   Revised PI No.14, measure 3         Supporting Measure: Labour force participation rate

   Revised PI No.15                    Proportion of Indigenous 20-64 year olds with or working
                                       towards post school qualification in AQF Certificate III level or
                                       above

                                              Disaggregate to include reporting against ‘with’ and


                                                                                              20 | P a g e
                                                 ‘working towards’

There are two performance indicators against the employment target, reduced from seven. There are
three supporting measures under the first indicator.

Summary
The NIRA Review Working Group recommends that 15 revised performance indicators be reported on in
future under the NIRA performance framework, noting that these changes will not take place until the
2011-12 reporting period.

The revised indicator set represents conceptually adequate indicators, with acceptable data and forms a
performance reporting system that encourages more direct public accountability for performance
against the Closing the Gap targets. Jurisdictional performance, across the Commonwealth, State and
Territory Governments, against the revised indicator set will, as a result, now be more representative
and reflective of progress towards the goal of Closing the Gap in Indigenous disadvantage. By removing
performance indicators that do not sufficiently demonstrate progress, reporting against the indicator set
should provide for increased transparency and public accountability.

The revised performance framework will not only assist in improving public reporting and accountability,
but also reflects the ongoing role of the NIRA as the pre-eminent agreement guiding Governments’
approach to Closing the Gap and Overcoming disadvantage in Indigenous communities.


Other issues raised at the workshop
The Working Group agreed to preserve the CRC’s annual performance reporting against the NIRA
recognising that new data for each indicator will not always be available.

The annual NIRA Performance Report released by the COAG Reform Council may need to include
supplementary data and analysis to provide contextual information. Under the IGA FFR the CRC is able
to seek additional information providing it is available in a timely manner and the cost of the reporting
burden on the jurisdiction is reasonable. However, most of the supplementary data and analysis can be
drawn from existing reporting.

It was agreed that the Steering Group recommends that COAG should note the recommended changes
to the NIRA performance framework will take effect in the 2011-12 reporting period.

There is some alignment between the NIRA and other NA performance frameworks and these linkages
should be considered in terms of data availability and consistency. This will assist to ensure that
Indigenous issues are included as a priority under other NAs. As part of this point it is also recognised
that NIRA indicators sourced from other NAs may need to be reviewed in light of any changes made
through the review processes for the other NAs.




                                                                                               21 | P a g e
Attachment A - Working Group Terms of Reference

NIRA Review Working Group Terms of Reference
Preamble
At its meeting on 13 February 2011, the Council of Australian Governments (COAG) agreed that the
underlying reform principles of the Intergovernmental Agreement on Federal Financial Relations
continue to provide a strong foundation for progressing COAG’s agreed reform agenda and achieving
better policy and service delivery outcomes for all Australians. COAG established a HoTs Review and
CRC Recommendations Implementation Steering Group to review and improve implementation of the
IGA.

Role/purpose
The Working Group has been established by the HoTs Review and CRC Recommendations
Implementation Steering Group to address the performance reporting issues associated with the
refreshed National Indigenous Reform Agreement (NIRA) identified by the Heads of Treasuries (HoTs)
review and reports of the COAG Reform Council (CRC) to reinforce COAG’s commitment to enhancing
performance and public accountability.

Term
The Terms of Reference are effective from May 2011 and continue until the review is complete (due 31
July 2011, final recommendations to be back to Steering Group by 15 July 2011).

Membership
Membership will consist of Senior Officers from First Ministers’, Treasuries and relevant portfolio
agencies from the Commonwealth and the States and Territories.

The Secretariat for the Steering Committee for the Review of Government Service Provision and the
COAG Reform Council are invited to attend as observers.

Key tasks
The Working Group will:

           undertake a review of the NIRA performance framework using the conceptual framework
            developed by the HoTs review
            (http://www.federalfinancialrelations.gov.au/content/performance_reporting/downloads/c
            onceptual_framework_for_performance_reporting_feb11.pdf ) as per the process outlined
            below;


                                                                                              22 | P a g e
         limit indicators to those necessary to measure performance and inform the public about
          reform progress;
         consider the performance data currently collected through the NIRA and what data and
          reporting is required to demonstrate performance (including the ability to identify
          Indigenous Australians within data collections and to further disaggregate by geo-location);
         include an assessment of the cost of developing data to support proposed performance
          indicators against the benefit of improved public accountability;
         consider the role of the NIRA in supporting the achievement of the COAG closing the gap
          targets including measuring the achievement of Indigenous outputs and outcomes;
         consider whether reporting timeframes should be revised to improve timeliness;
         respond to the relevant CRC recommendations (see attachment);
         seek advice as required from the National Indigenous Reform Agreement Performance
          Information Management Group (NIRA PIMG) whose membership includes the Productivity
          Commission and the COAG Reform Council;
         seek advice from other relevant Ministerial Council data groups (e.g. health);
         provide a final report including recommendations to the HoTs Review and CRC
          Implementation Steering Group on the review and provide recommendations for action; and
         have regard to other work, such as the reviews of the performance frameworks of other
          National Agreements which share indicators with the NIRA.

Meetings
         The Working Group will meet as required.
         Meetings will be chaired jointly by the Department of the Prime Minister and Cabinet and
          the Queensland Department of Premier and Cabinet.
         The Commonwealth will provide secretariat support for the Working Group.
         As many meetings as possible will be held via telepresence.

Process
     1. Initial analysis of the NIRA performance framework against the HoTs Review conceptual
        framework, taking into consideration:
             o that the NIRA includes a number of performance indicators from other National
                  Agreements and National Partnership Agreements; and
             o the recommendations from the CRC and Steering Committee for the Review of
                  Government Service Provision.
     2. Analysis of existing performance indicators and performance benchmarks that are identified
        as poor quality (at step 1) against the HoTs Review Conceptual Framework’s guiding
        principles for rationalisation or change.
     3. Analysis of additional and alternative performance indicators and performance benchmarks
        against a framework for prioritisation, including consideration of the costs and benefits of
        new or expanded data collections.
     4. Consideration of the overall appropriateness and proportionality of any revised performance
        framework.
     5. Draft recommendations and a report with supporting analysis (including a response to the
        CRC’s recommendations).




                                                                                         23 | P a g e
Key Deliverables and Outcomes
        Analyse the NIRA Closing the Gap performance indicators against Treasury’s Conceptual
         Framework, in order to assess the effectiveness of the indicators in demonstrating national
         progress and increasing public accountability.
        Provide a set of recommendations for action to the HoTs Review and CRC Implementation
         Steering Group by 18 July 2011.
        Provide a final report on the review to the HoTs Review and CRC Implementation Steering
         Group by 29 July 2011.

Attachments
  COAG response to CRC recommendations are included as attachments to this Terms of Reference.




                                                                                         24 | P a g e
COAG Reform Council: NIRA 2008-09

At its meeting on 13 February 2011 COAG agreed to adopt a revised NIRA which has been refreshed to take account of recommendations made in CRC reports.
Where CRC recommendations have not already been reflected in the NIRA these are to be considered in a review of the NIRA to be undertaken for COAG, by
Senior Officials and Heads of Treasuries prior to 31 July 2011. The response to the 2008-09 report has been agreed COAG and sent to the CRC.


   Recommendation                                                                                    COAG Response


   Recommendation 1:

   The COAG Reform Council recommends that COAG confirm that administrative data provided            Agreed.
   for each report should relate to the relevant reporting year



   Recommendation 2:

   The COAG Reform Council recommends that COAG agree to revise the indicator of the                 Agreed.
   proportion of Indigenous 20 to 64 year olds with or working towards a post school qualification
   in Certificate III, IV, Diploma or Advanced Diploma to ‘the proportion of Indigenous 20 to 64
   year olds with or working towards a post school qualification in Certificate III or above’
                                                                                                     The indicator has been revised and was incorporated in the
                                                                                                     refreshed NIRA adopted by COAG at its meeting on 13 February
                                                                                                     2011.


   Recommendation 3:

   The COAG Reform Council recommends that COAG note the issues and suggestions for
                                                                                                     Noted and incorporated in the refreshed NIRA or referred to the
   improvements to the performance reporting framework listed below, and refer them to the
   Heads of Treasuries Committee on Federal Financial Relations for further consideration and        working group to consider as part of its review of the NIRA


                                                                                                                                                               25 | P a g e
        prioritisation, in consultation with the NIRA PIMG.                                                   outlined above.

        a)       Overall data improvements

 (i)             Give high priority to finalising the national and jurisdictional trajectories during 2010.

(ii)             Give priority to improving Indigenous identification in the Census and administrative
        data collections.

(iii)             Develop methods to allow mortality and child mortality data to be reported for
        jurisdictions with small Indigenous populations (where issues of Indigenous identification are
        also improved).


(iv)             Improve the disaggregation of performance indicators by geo-location (including
        survey coverage in remote areas) and investigate the desirability of providing disaggregations
        by geo-location for other indicators in the National Indigenous Reform Agreement.

(v)            Investigate the feasibility of developing a model for an Indigenous index of socio-
        economic disadvantage.

(vi)             Consider the appropriateness of the frequency of the data collections for indicator data
        and the impact on the council’s ability to analyse the year to year performance of jurisdictions,
        taking into account both the benefits and costs of more frequent reporting.

        b)      Target: Life expectancy
 (i)            Support the work being undertaken by the Australian Institute of Health and Welfare
        (AIHW) to develop a better measure of access to health services compared to need.

        c)       Target: Child mortality
        (i)      Support the work currently undertaken by the AIHW to:

        - improve the collection of data on access to antenatal care and tobacco smoking during
        pregnancy to allow jurisdictional comparisons


                                                                                                                                26 | P a g e
        - collect data on the Indigenous status of the child, as well as the mother, in the National
        Perinatal Data Collection.

        d)       Target: Early childhood education
        (i)      Develop, as a matter of priority, a data source that can be used to report against this
        indicator.

        e)       Target: Literacy and numeracy

        (i)      Better understand non-participation in NAPLAN testing by separately reporting absent
        and withdrawn students, and by disaggregating students by geo-location.

        f)       Target: Year 12 attainment

 (i)              Consider inclusion of a timely and accurate measure of Indigenous Year 10 completion
        rates for use in both the National Indigenous Reform Agreement and the National Education
        Agreement

(ii)             Continue to develop alternative measures of secondary school engagement to
        apparent retention rates for use in both the National Indigenous Reform Agreement and the
        National Education Agreement.

(iii)           Continue to investigate a more timely and informative measure of Year 12 attainment
        based on the actual number of Year 12 completions through certificate identification.

        g)       Target: Employment

 (i)              develop the relevant data sources to be able to report on engagement with study by
        level of study, and report separately on higher education and vocational qualifications




                                                                                                           27 | P a g e
COAG Reform Council: Report on the NIRA 2009-10

The response to the 2009-10 report is to be considered at the COAG meeting of 19 August 2011.
The response below is draft and has not yet been agreed.


   Recommendation                                                                       COAG Response

   Recommendation 1(a):

    The COAG Reform Council recommends that COAG: consider the following
    issues for improving the performance reporting framework for the National           Agreed
    Indigenous Reform Agreement:
            reframing the targets and indicators that rely on survey data to a three
    yearly reporting cycle and/or investigating the use of output measures that could
    serve as proxy indicators of progress towards these targets
   Recommendation 1(b):
                                                                                        Agreed
    The COAG Reform Council recommends that COAG consider the following
    issues for improving the performance reporting framework for the National
    Indigenous Reform Agreement:
            prioritising work on methods of calculating variability bands for
    administrative data to allow the council to determine the significance of year to
    year change.
   Recommendation 2:

   The COAG Reform Council recommends that COAG refer these issues (see
   Recommendation 1), along with the recommendations made in the council’s              Agreed in part
   2008–09 baseline report, to the review of the National Indigenous Reform
   Agreement announced by COAG on 13 February 2011.                                     Recommendations for the
                                                                                        2008-2009 baseline report have
                                                                                        already been incorporated into
                                                                                        the Terms of Reference of the
                                                                                        review of NIRA

                                                                                        The NIRA working group is
                                                                                        working towards finalising the
                                                                                        review of the National
                                                                                        Indigenous Reform announced
                                                                                        by COAG on 13 February 2011
                                                                                        by July 2011. It is not possible
                                                                                        for the review to consider the
                                                                                        recommendations of the 2009-
                                                                                        10 report as COAG will likely
                                                                                        meet very close to the end of
                                                                                        the review, if not after the
                                                                                        review is completed




                                                                                                    28 | P a g e
Attachment B - Terminology

  National Agreement
  An agreement defining the objectives, outcomes, outputs and performance indicators, and
  clarifying the roles and responsibilities, that will guide the Commonwealth and the States and
  Territories in the delivery of services across a particular sector.

  National Partnership Agreement
  An agreement defining the objectives, outputs and performance benchmarks related to the
  delivery of specified projects, to facilitate reforms or to reward those jurisdictions that deliver
  on national reforms or achieve service delivery improvements.

  •     Reform-based NPs: national priority reforms and improvements in service delivery (see IGA
        FFR E19(b) and (c)) [For example : Early Childhood Education National Partnership]; and

  •     Project-based NPs: specified projects that support national objectives (e.g. infrastructure or
        time limited projects to deliver a specific new or improved service; see IGA FFR E19(a)) [For
        example : Social Housing National Partnership]. Type and number of indicators under Project
        NPs subject to strict assessment of appropriateness and proportionality.

  Objectives
  Describe the mutually-agreed, overarching aspirations.

  Outcomes
  Describe the impact a government activity is expected to have on community wellbeing.
  Outcomes should be strategic, high level and observable goals expressed in clear, measurable
  and achievable terms.

  Outputs
  Describe the services being delivered by governments to achieve outcomes. Alternatively
  they may be used as a proxy for outcomes where outcomes are not readily observable.
  Outputs can also help to define roles and responsibilities. Outputs should be high level, as
  detailed outputs run the risk of constraining States’ responses to changing demand, cost
  drivers, priorities and service delivery models.

  Performance Benchmarks
  A quantifiable change in a performance indicator, usually expressed in respect of a period of
  time – for example, an X per cent increase in X by 20XX. Where necessary to inform the
  community, performance benchmarks should be few in number, high-level and reflect the
  highest order, most challenging goals toward attainment of outcomes.

  Purpose
  The purpose of performance benchmarks is to articulate the objective(s) of the agreement in a
  measurable manner and to set out a timeframe for their achievement. Ideally they should also
  have a trajectory to provide the indicative rate of progress necessary to achieve this
  timeframe.

                                                                                           29 | P a g e
Additional information
Performance benchmarks are distinguishable from performance indicators as they articulate
the desired level of achievement and when this should be achieved by.

Baseline and endpoint years should be selected with reference to data availability and
timelines relating the implementation of policies.

Performance benchmarks can be tailored to individual jurisdictions to ensure they take into
account characteristics and conditions. However, it is critical that they are defined and
measured in a uniform manner to preserve the integrity of the performance benchmark.

There are numerous ways to present performance benchmarks. To ensure the agreement is
transparent it is critical that the measure description, numerical goal and timeframe are all
publicly available.

Trajectories should only be developed where they are supported by data is of a sufficient
quality. They should be subject to review where new information becomes available
impacting on underlying assumptions.

Performance indicators
Data to inform the community about how governments are progressing towards the
achievement the objectives, outcomes and outputs.

Purpose
Performance indicators are a point in time measure which demonstrates how performance
under the agreement is tracking. Over successive measures the indicators should demonstrate
progress towards achieving the objective of the agreement.

Additional information
Performance indicators should be separate from performance benchmarks, however they can
be related.

Performance indicators should demonstrate achievement preferably in terms of outcomes, not
outputs.




                                                                                     30 | P a g e
Attachment C – List of the 6 NIRA Targets and 27 performance indicators1

 Close the life           Halve the gap in        Ensure all             Halve the gap for      Halve the gap for          Halve the gap in
 expectancy gap within    mortality rates for     Indigenous four year   Indigenous             Indigenous 20-24 year      employment outcomes




                                                                                                                                                       Targets
 a generation             Indigenous children     olds in remote         children in reading,   olds in Year 12            between Indigenous and
                          under five within a     communities have       writing and            attainment or              non-Indigenous
                          decade                  access to early        numeracy within a      equivalent attainment      Australians within a
                                                  childhood education    decade                 rates by 2020              decade
                                                  within five years

 Estimated life           Child under 5           The proportion of      Percentage of          Proportion of 20 to 24     Employment to
 expectancy at birth      mortality rate (and     Indigenous children    students at or         year olds having           population ratio, for 15
                          excess deaths)          (by geographic         above the national     attained at least a Year   to 64 year olds
                                                  location) who are      minimum standard       12 or equivalent or
 Mortality rate (and                              enrolled in (and       in reading, writing    Australian
 excess deaths) by        Mortality rates (and                                                                             Unemployment rate
                                                  attending) a           and numeracy for       Qualification
 leading causes           excess deaths) by
                                                  preschool program in   years 3, 5, 7 and 9    Framework
                          leading causes:
                                                  the year before                               Certificate II
                          • Perinatal                                                                                      Labour force
                                                  formal schooling       Rates of
 Hospitalisation rates                                                                                                     participation rate
                          • Infant                                       participation in
 by principal diagnosis                                                                         Apparent retention
                          • 1 to 4 years                                 NAPLAN reading,        rates from year 7/8 to
                          • 0 to 4 years                                 writing and            year 10 and year 12        CDEP participants and
 Rates of current daily                                                  numeracy tests –                                  off CDEP job placement
 smokers                                                                 years 3, 5, 7 and 9




                                                                                                                                                       Indicators
                                                                                                Attendance rates year
                          Child under 5                                                         1 to year 10               3 month employment
 Average daily alcohol    hospitalisation rates                                                                            outcomes (post
 consumption and          by principal                                                                                     program monitoring)
 associated risk level    diagnosis

                                                                                                                           Indigenous 18 to 24 year
 Levels of obesity –      Proportion of babies                                                                             olds engaged in full time
 body mass Index          born of low                                                                                      employment, education
                          birthweight                                                                                      or training at or above a
                                                                                                                           Certificate III
 Level of physical
 activity                 Tobacco smoking
                          during pregnancy                                                                                 Indigenous 20 to 64 year
                                                                                                                           olds with or working
 Access to health care                                                                                                     towards post school
 compared to need         Antenatal care                                                                                   qualification in
                                                                                                                           Certificate III or above


  1
      Source: COAG (2008c) National Indigenous Reform Agreement (as revised 13 February 2011)

                                                                                                                                                                    31 | P a g e
Attachment D - Write up from the CRC Recommendations

CRC Findings - NIRA Performance Report for 2008-09

The COAG Reform Council recommends that COAG:
    1. confirm that administrative data provided for each report should relate to the relevant reporting
       year;
    2. agree to revise the indicator of the proportion of Indigenous 20 to 64 year olds with, or working
       towards, a post-school qualification in Certificate III, IV, Diploma or Advanced Diploma to ‘the
       proportion of Indigenous 20 to 64 year olds with or working towards a post-school qualification
       in Certificate III or above’;
    3. note the issues and suggestions for improvements to the performance reporting framework
       listed below and refer them to the Heads of Treasuries Committee on Federal Financial Relations
       for further consideration and prioritisation, in consultation with the NIRA Performance
       Information Management Group (PIMG).

    Overall data improvements
    4. Give high priority to finalising the national and jurisdictional trajectories during 2010.
    5. Give priority to improving Indigenous identification in the Census and administrative data
       collections.
    6. Develop methods to allow mortality and child mortality data to be reported for jurisdictions with
       small Indigenous populations (where issues of Indigenous identification are also improved).
    7. Improve the disaggregation of performance indicators by geo-location (including survey
       coverage in remote areas) and investigate the desirability of providing desegregations by geo-
       location for other indicators in the National Indigenous Reform Agreement.
    8. Investigate the feasibility of developing a model for an Indigenous index of socio – economic
       disadvantage.
    9. Consider the appropriateness of the frequency of the data collections for indicator data and the
       impact on the council’s ability to analyse the year-to-year performance of jurisdictions, taking
       into account both the benefits and costs of more frequent reporting.

    Target: Life Expectancy

    10. Support the work being undertaken by the Australian Institute of Health and Welfare (AIHW) to
        develop a better measure of access to health services compared to need.

    Target: Child Mortality

    11. Support the work currently undertaken by the AIHW to:
               a. Improve the collection of data on access to antenatal care and tobacco smoking
                   during pregnancy to allow jurisdictional comparisons;
               b. Collect data on the Indigenous status of the child, as well as the mother, in the
                   National Perinatal Data Collection.

                                                                                            32 | P a g e
    Target: Early Childhood education

    12. Develop, as a matter of priority, a data source that can be used to report against this indicator.

    Target: Literacy and Numeracy

    13. Better understand non-participation in NAPLAN testing by separately reporting absent and
        withdrawn students, and by disaggregating students by geo-location

    Target: Year 12 attainment
    14. Consider inclusion of a timely and accurate measure of Indigenous Year 10 completion rates for
        use in both the National Indigenous Reform Agreement and the National Education Agreement.
    15. Continue to develop alternative measures of secondary school engagement to apparent
        retention rates for use in both the National Indigenous Reform Agreement and the national
        Education Agreement.
    16. Continue to investigate a more timely and informative measure of Year 12 attainment based on
        the actual number of Year 12 completions through certificate identification.

    Target: Employment
    17. Develop the relevant data sources to be able to report on engagement with study by level of
        study, and report separately on higher education and vocational qualifications.

CRC Findings - NIRA Performance Report for 2009-10

The COAG Reform Council recommends that COAG consider the following issues for improving the
performance reporting framework for the National Indigenous Reform Agreement:
    18. reframing the targets and indicators that rely on survey data to a three yearly reporting cycle
        and/or investigating the use of output measures that could service as proxy indicators of
        progress towards these targets;
    19. prioritising work on methods of calculating variability of bands for administrative data to allow
        the council to determine the significance of year to year change; and
    20. refer these issues, along with the recommendations made in the council’s 2008-09 baseline
        report, to the review of the National Indigenous Reform Agreement announced by COAG on 13
        February 2011.




                                                                                               33 | P a g e
Glossary
ABS          Australian Bureau of Statistics

AIHW         Australian Institute of Health and Welfare

COAG         Council of Australian Governments

CRC          Council of Australian Governments Reform Council

DEEWR        Department of Education, Employment and Workplace Relations

DoHA         Department of Health and Ageing

FaHCSIA      Department of Families, Housing, Community Services and Indigenous Affairs

HPF          Health Performance Framework

HoTs         Heads of Treasuries

IGA FFR      Intergovernmental Agreement on Federal Financial Relations

NAs          National Agreements

NAGATSIHID   National Advisory Group on Aboriginal and Torres Strait Islander Health
             Information and Data

NASWD        National Agreement for Skills and Workforce Development

NATSIHS      National Aboriginal and Torres Strait Islander Health Survey

NATSISS      National Aboriginal and Torres Strait Islander Social Survey

NEA          National Education Agreement

NIRA         National Indigenous Reform Agreement

NIRA PIMG    NIRA Performance Information Management Group

NPs          National Partnerships

SCFFR        Standing Council for Federal Financial Relations

SCRGSP       Steering Committee for the Review of Government Service Provision

WGIR         Working Group on Indigenous Reform




                                                                                34 | P a g e

								
To top