12. Monitoring and improving performance
There are a large number of possible indicators (output, input, process, outcome etc) of
possible relevance to measuring and assessing performance. Some of these can be
expressed as relationships (output/input=efficiency; outcome/cost=cost effectiveness).
One set of problems revolves around finding relevant information and ensuring data
integrity. Another centres on using the data to reach conclusions and as a basis for
action. Absolute measures are of limited usefulness. Relative measures are more useful
(for instance targets set in the budget, the entity's previous performance or the
performance of other similar entities doing the same job).
Comparing what was authorized with what was spent is a narrow and inadequate basis for
judging the success of budget execution. It emphasizes what was spent, but not what was
achieved. It focuses on line items and votes and examines expenditures in relation to
authorizations. However, funds were made available to achieve results (reduce disease;
increase literacy; improve crop yields etc.). The effectiveness of budget execution or of
government programs is judged against such objectives .
But effectiveness is an elusive quality. How do we know whether a program is effective
(achieves the impacts necessary to fulfil its objectives and what degree of effectiveness is
adequate? ). A number of conceptual and practical constraints make it difficult to answer
these questions satisfactorily:
Were the program and its objectives adequately defined, and were the
objectives explicit and feasible in terms of the inputs provided (resources,
staff, materials, logistical support)?
Are the means of measuring the achievement of the objectives appropriate,
and are the effectiveness measures themselves reliable and free from
What portion of the objectives were to be achieved within a given time period
(e.g. during the budget year)?
Is there a one to one relationship between the program and the objectives or
could the achievement of the objectives be influenced by factors outside the
program (e.g. other programs, the weather, the state of the economy, the
nature of the target clientele etc.)?
Do targets relate effectiveness to costs or is the objective a desired impact
irrespective of cost?
And if a program has several objectives, what level of effectiveness is
expected from each objective, bearing in mind that resources may be shifted
between objectives and that achieving more of one objective may mean
achieving less of another?
Effectiveness is correctly identified as the key criterion for assessing the success of
government programs. However judging effectiveness in individual cases depends on
finding satisfactory answers to the sorts of question indicated above. For this reason the
practical use of effectiveness is limited to scenarios of the following type:
In policy choice and program design where effectiveness is a key question but
where actual measurement of effectiveness is not involved
Where a superficial assessment of effectiveness is all that is needed (e.g.
where significant cases of ineffectiveness can be identified as “self-evident”)
Where there is a choice between competing means of achieving the same
result (e.g. roughly equal effectiveness but very different costs)
Where effectiveness can be subject to in-depth research and subjected to
adequate analysis of causes and effects (sometimes referred top as program
Efficiency is a relationship between inputs to outputs. Greater output for the same input is
an example of greater efficiency. Inputs can be measured in terms of cost. But cost
comparisons over time will be vitiated by inflation. So it is conventional in studying
long-term efficiency changes to identify a significant input which can be expressed in
physical terms (person/years). This can be compared with an output indicator, also in
physical terms. In the public sector the main output is services (items delivered,
operations performed, cases completed, permissions given, applications approved etc.).
The services supplied can be regarded as a stream of outputs. Thus outputs and inputs
can be compared and changes in efficiency identified.
We have to distinguish between intermediate and final outputs. A final output provides
value outside the government sector (to a person or other entity). It is like a good in
microeconomics, that has crossed the production boundary. Intermediate outputs are
produced inside the entity. If intermediate outputs are handled efficiency, the chances of
producing final outputs efficiently will be improved. But intermediate outputs could be
misleading as an indicator of overall efficiency as they do not serve an external
consumer. So government entities are assessed according to their final outputs.
Obviously we need “valid” output indicators. That is to say the outputs should represent
the transfer of something valuable to the consumer/client. We also need a reliable means
of measuring and reporting such outputs. Some of the problems encountered in this area
are as follows:
Government entities which do not have reliable output measurement systems;
absence of reliable output measures
Managers or politicians who wish to report performance in so far as it puts
them in a favorable light
Outputs which are lumpy (one unit of output cannot be compared with another
because each is fundamentally different in size, quality, cost etc.)
Outputs which are meaningful only when they meet certain quality criteria
(e.g. timeliness, freedom from error, availability)
Absence of performance targets (e.g. in budgets) and the related problem of
lack of knowledge of an appropriate standard for judging whether efficiency
gains are "adequate"
Performance targets which result in maximization of the wrong outputs and
All of the above make performance measuring and monitoring into complex tasks.
12.4 Performance budgeting
Performance budgeting (definition in annex) links outputs to program costs (or this is its
stated objective). It has been introduced in many developing countries, but with little real
success. In addition to the problems related immediately above, other issues have been
It is difficult to implement a budget reform without the necessary recording
and reporting systems. These depend upon management: its accountability,
orientation and capacity.
It is easy enough to identify output indicators. Often many indicators are
reported each with a different meaning, so that it is very difficult to know if
output is increasing/falling and indeed which output is to be maximized
There are few examples in which outputs are directly related to costs so that
the emphasis cannot be on changes in efficiency. The reason for this is
difficulty of cost attribution when a program has multiple outputs
Generally, meaningful figures of output can only be found at below program
Governments with performance budget systems publish more information on
outputs but their budget systems remain fundamentally unaffected by the
Clearly a budget system should be orientated towards performance. However, achieving
this in the developing world remains a serious challenge. Comprehensive performance
budgeting systems rarely work. But some selective approaches may make sense:
concentrating on performance measurement in discrete parts of government
(e.g. particular ministries or parts of ministries)
setting up formal performance measurement systems which are properly
documented and capable of verification
improving the management of these entities in parallel
ensuring that established performance measurement systems are appropriate
(e.g. actually used to manage the entity)
applying performance measurement only to activities where performance
measurement is feasible (e.g. activities with a stream of relatively
homogeneous stream of outputs); not attempting to measure the unmeasurable
supplementing output measures with formal means of measuring quality of
outputs and interpreting output measures in relation to quality of service
adapting the accounting system so that program costs can be linked to
establishing systems which report on cost, output and the quality of services
Using this approach it may be possible to assess some aspects of budget implementation
on a performance basis. The process would have to start at the budget formulation stage
(identification of objectives, outputs and quality factors; setting of targets). Then during
implementation (e.g. at mid-term review and for annual accountability) outputs, costs
and quality of service delivery can provide a viable focus for monitoring. However
internalization of these systems is a necessary condition for their successful external use.
12.5 Advanced performance-based systems
The introduction of more advanced systems assumes that problems of the sort already
discussed have been resolved/addressed. Some countries (in particular OECD countries)
have graduated to more complex models for improving performance.
Bench-marking provides a standard against which actual performance can be judged.
The standard is that of another entity which carries out the same function. Best practice
bench-marking is a standard of excellence. The entity compares each of its processes
with those judged to be best in the outside world. It then adapts each process so that it
can achieve the same or better performance than its most able competitors.
Bench-marking in the developing world, must take account of local conditions and
opportunities. Basing targets on the performance of a well-resourced entity in a wealthy
country is likely to be unrealistic. Choosing that of a neighbouring country or one at a
similar stage of development may make more sense. Also of relevance are the country's
own institutions. For instance where there are many institutions carrying out similar
functions (e.g. hospitals, schools) performance can be bench-marked against those that
perform best. But adjustments must be made for factors such as funding, nature of
clientele, etc. Without this, such schemes can become deeply controversial.
In a way, a government budget is just such a contract. It provides funds in exchange for
performance. But its terms are unclear, performance is only one of many considerations
and the consequences of good or bad performance are not always explicit. Moreover the
incentives of those who manage spending programs (See paper two) sometimes conflict
with performance considerations. Of course having a contract in which there is no
competition amongst contractors and where the contractor remains essentially the same
irrespective of performance (management of the relevant ministry) lacks realism. A
performance contract is an attempt to correct such tendencies.
A contract is normally between two parties. In principle its terms are legally
enforceable. A performance contract is usually between government (for instance a
ministry) and the managers of a sub-entity. It establishes the resources, targets, quality
constraints, time-scale and other conditions relevant to performance. It also stipulates the
rewards for fulfilling the contract (additional remuneration for management and/or staff;
improved access to funds etc.). Management accepts the contract and bases its objectives
on fulfilling the contract. Clearly there are pitfalls (e. g. the contract does not refer to
certain government objectives, which are naturally ignored by management; the quality
of expected performance is not sufficiently defined; management performs badly but
cannot be replaced until the contract expires etc.).
This is a type of performance contract in which parts of government are defined as
purchasers of services (they are also providers of funds) and other parts as providers of
services. Purchasers set contract terms for the provision of discrete services. Service
providers bid for the available funds. Those outside the government sector may also
compete for contracts. The best known example is in New Zealand. It has the effect of
holding down the cost of providing government services but is thought to have
compensating weaknesses (Schick, 1998).
Alternatively some services traditionally supplied by government agencies may be
identified for contracting out (private sector provision). Again contracts are let for the
provision of services. Experience suggests that certain peripheral and support services
can be easily and effectively contracted out (e.g. cleaning and refuse collection services).
Services where personal and state security are important issues (e.g. airport security)
may be less suitable. Nevertheless contracting out is extensively employed for the
provision of health services, prisons, transport infrastructure (rail systems, motorways)
In its approach to performance improvement the Bank attaches great importance to the
experience of developed countries, in particular USA, Australia, New Zealand and United
Kingdom. While reforms in these countries are relevant to developing and transitional
countries, the extent of relevance varies from case to case. Countries with strong
capacity and strong management-orientation can learn from these reforms and may
introduce similar systems. Others (and this must apply to the majority) have deep-rooted
problems which these reforms cannot touch. Therefore the challenge for Bank staff is in
correctly diagnosing a country's problems in this area and in backing reforms that are
within the country's capacity, and likely to be effective in addressing their problems.
Asian Development Bank (1999), Public Expenditure Management Handbook, Manila.
Organisation for Economic Co-operation and Development (1997), Modern budgeting,
Organization for Economic Cooperation and Development (2001) Managing Public
Expenditures: A Reference Book for Transition Countries. Richard Allen and D.
Tommasi, editors. (Chapter 15)
World Bank (1998), Managing government expenditure, PREM, Washington.
World Bank (February 1998), Why most developing countries should not try New
Zealand’s reforms, Allen Schick, World Bank Research Observer, volume 13, number 1,
Brief notes on budget reforms
Many reforms have been suggested and tried over the years:
Program budget: a budget made up programs as groupings of activities intended to
contribute to identifiable government objectives (e.g. poverty alleviation, literacy, control
of contagious disease.). In practice it is difficult to identify satisfactory programs
because they are often made up of activities controlled by several different ministries.
Moreover, the presentation of a program budget may help some users of information but
hinder others. Few governments have useful program budgets; most follow the existing
organizational structure of ministries.
Performance budget: a program budget which also presents measures of performance and
service delivery (e.g. students graduating, surgical operations performed, tons of cargo
unloaded). The concept is excellent; examples of successful adoption are limited due to
problems of defining performance and relating it to programs and their cost.
Zero-base budget: a budget which is justified from zero. Each agency has to justify its
whole budget as if it were applying for funding for the first time. The concept is
sometimes used selectively.
Biennial budget: a budget that provides funds for two years instead of one. Budget
allocations do not lapse until the end of the second year. It is an attempt to compensate
for an artificial assumption of traditional budgeting: that it is sensible to budget for short
periods when many decisions are implemented over longer periods.
Multi-year budget: a budget that takes into account not just the budget year, but two or
more subsequent years. Usually lapse of funds occurs at the end of the budget year.
Figures for “out years” are indicative. The aim is similar to that of biennial budgeting.
Multi-year budgeting has been replaced by the MTEF.
Medium-term expenditure framework (MTEF): a process for improving government
expenditure programs which assists decision-makers to guage what is affordable in
aggregate over the medium-term and to reconcile this with spending policies and their
costs over the same period. The aim is similar to that of multi-year budgeting. It
incorporates a performance budgeting approach. It is employed extensively in developed
countries but is yet to be proved in practice in developing and transitional countries.