Cost Benefit Analysis Tool by maureenshubert

VIEWS: 0 PAGES: 5

									                                                                                                           1

BENEFIT ASSESSMENT TOOL – GUIDANCE ON USE

1       This note sets out guidance on how to complete the Bank of England’s benefit assessment tool.
The main aim of the tool was to make a rigorous, consistent and transparent assessment of the relative
benefits of the information collected by the Bank. This process is explained more fully in the CBA
Handbook: ‘Cost-benefit analysis of monetary and financial statistics, a practical guide’.


The process for using the benefits assessment tool

2       The tool has been developed to give non-monetary benefits scores for statistical data. These
scores provide a useful gauge of the relative importance of data, which can be set alongside the
information on the costs of providing the data. Those data which look relatively costly and are
providing only low benefits to users are the most obvious collections to question first.

3        The benefits assessment tool has been used in a number of ways within the statistics area of the
Bank. Some analysts have looked through the different uses of the data for an entire collection (form /
table); other analysts have looked at the benefits of each individual box on a data collection or by
sections of a data collection. Because of the nature of banks’ costs, it is only through cutting whole
sections of forms or entire collections that significant cost reductions for reporters can be achieved.
Therefore it may be best to concentrate on measuring the benefits on the same basis – ie at a whole
form/section level.

4      It is only the most important use of the data that is being assessed. Multiple uses are taken into
account within the tool, so there is no need to re-run the benefits for different users/uses unless there is
some uncertainty over the main user/use of the data.

5       The benefit assessment would normally be completed by the most experienced analyst
conducting the review of the form / table, taking account of the views of known major users of the data
– in some cases there may be widespread users and a public consultation may be required.

6       Where the analyst is very experienced in the uses and users of the data, a basic benefits
assessment could be made at an early stage of the review. The results could help to focus on different
options for whether to continue the data collection, or possibilities of lowering reporting costs. Of
course, initial views may change as new information comes to light during the course of the review, so
it is worth keeping an open mind on any initial assessments and revisit benefits scores where
appropriate.

7       There needs to be some mechanism for ensuring consistency across benefits assessments for
different data collections, which are also typically undertaken by different analysts. The Bank found
the most effective way to do this was through a meeting of all analysts involved in reviews. This group
                                                                                                          2
would examine the relative scores for each section of the benefits assessment and would revisit scores
that looked out of line with others.

8        The tool is a means to an end, and it should accurately reflect where data have high, medium or
low benefit. But it may also need to be adapted to particular circumstances. Relative scores of
benefits and costs may indicate where savings could be made, which can be particularly helpful when
there is pressure to lower reporting or compilation burdens. It will also inform priorities and help
general resource allocation.


How to fill in the form

9       There are two types of scores on the form: gold boxes, where one (and only one) option needs
to be chosen in each of the boxes by placing a 1 in the relevant box; and green boxes, where a 1 is
entered for each that applies. A score out of 100 will automatically appear at the end; the higher the
score the greater the benefits of the data collection. There are also blue boxes to capture qualitative
information about the data collection being assessed.


The blue boxes Fig 1

Data being reviewed

Form / section / table
(where applicable)
Section of form / table
(where applicable)

Description of any other uses
(include internal statistical area uses
e.g. cross-checks, sample selection)




10      The blue area at the top is to record which part of the form / table is being assessed and which
are the main data output(s) involved. Because the benefits tool can be used for a wide range of data
sources - from individual boxes, sections of forms, full tables / forms etc - it is important here to be
clear what is being assessed. The final blue panel is to record if and where the information is used
internally (within the compilation institution); for instance, if it is used for cross-checking information
on other sections / tables / forms. These uses can be important and should be noted.


Policy use and relevance (50% of total score) Fig 2
                                                                                                                                    3

Policy use                   Very high                      used in key policy decisions, or flagship publications and statistics
(choose 1 only)

                             High                           used in important policies, publications and statistics



                             Medium                         used in less important policies, publications and statistics



                             Low                            hardly used or in secondary publications



                             Very low                       Very rarely used

                             None


Policy relevance             Very high                      The key component of the activity / policy use
contribution of these data
(choose 1 only)              High                           A major input to the activity / policy use

                             Medium                         A generally important component of the activity / policy use

                             Low                            A sometimes important component of the activity / policy use

                             Very low                       A rarely important component of the activity / policy use

                             None




11       These two categories assess the importance of the activities that the data are used for (policy
use), and then the significance of the data for those activities (policy relevance). There two separate
scores to allow one to differentiate, for example, between very important activities where the data
plays either a major or minor role, and less important activities where data are either very important, or
less so.

12      Policy use is rated between very high and none, with accompanying descriptions to help guide
the assessor. Although this is subjective, there should be some evidence to base the decision on.

13      Policy relevance is similarly rated and captures how important the data are for the various
policy uses listed. Policy relevance can be important even if the policy use is not and vice versa.
Relevance should reflect how much importance the main users place on the data for decision making.
                                                                                                                                          4
Standards and regulation (15% of total score) Fig 3


Standards and regulation   Legal obligation                   No choice but to implement (excluding ESA95)
(choose 1 only)
                           Meets agreed standard              For full consistency with international standard (ESA, SNA, BPM)


                           Helps international                Other countries also produce the data but not comparably across standards
                           comparisons

                           None




14      In this section record if data are required by law (excluding ESA95), to meet an agreed
international standard (eg ESA95, SNA95 or BPM5), or if they data help international comparisons.


Additional uses (10% of total score) Fig 4


Additional uses            Helps outside research
(choose all that apply)
                           Helps inform general
                           public or media

                           Helps other economic
                           policies

                           Published, eg Stats
                           Release

                           Consistency check with
                           Other data / counterperties etc.




For each that applies, enter 1 in the associated green box.

15      This section awards a score for data that we know have other uses outside of the main user and
use (as measured in the policy use and relevance section). This can apply to outside research (eg
commentators, academics); to the general public or the media; or to other economic policies (eg where
government economists take interest in the data).


Value added (15% of total score) Fig 5


Value added                High                               Main source of high level data (i.e. aggregate boxes)
for internal analysis
(choose 1 only)            Medium                             Further breakdowns of main aggregate data


                           Low                                Similar data available elsewhere, or data could be estimated


                           None                               No value added




16      This section captures how much we can gain from these data, over and above what is available
from other sources. A high rating should mean that both the aggregate data and any sectoral or
industrial breakdowns are only available from this source, or it is the main source of a high level
aggregate. A medium score means that it is additional information to that available elsewhere such as
                                                                                                         5
a further breakdown of a main aggregate captured elsewhere. A low score means that similar data are
available elsewhere, or that the data could be estimated.


Quality (10% of total score) Fig 6


Quality                    High                             Census
(choose 1 only)
                           Medium                           Large sample


                           Low                              Poor quality - small sample / low coverage




17      This section looks at the underlying statistical quality of the data – how representative is the
sample, are there frequent significant revisions, do the data coherently fit in with what users want, or
are they a close second? A simple proxy here is to look at the size of the panel as a proportion of the
census in terms of data coverage. High would be full census; medium less than census but capturing a
representative panel; low is poor quality, where the panel is small and probably not representative.
More advanced measures are also worth investigating, depending on how the data are compiled and
what the sources are.


New Data Requests

18      The tool can be also used for new data requests. The principle is the same but the score will be
determined by the expected benefits from the expected policy use and relevance. Again costs here can
be weighed up against benefits for different options, for instance for different levels of data quality. A
new data request is likely to involve fixed set up costs and therefore the hurdle in terms of potential
benefits is likely to be high. It is important to be transparent about this with users and data suppliers.


Finally

19      The benefits tool gives a benefits score at the end. This is a percentage (max 100, min = 0),
and the higher the score, the more beneficial the data. After having filled out the form, it is a good
idea to check the score fits with what you would have broadly expected. It is also be helpful to
benchmark the score against other forms, and discuss how other people have used the tool to rate their
own tables / forms. The distribution of scores can be helpful to assess the important and less important
data, but only where there is consistency in the way the tool has been used. It is worth archiving
benefits scores, and ideally the thinking that they are based on, for future reference.

20     For data with no or very low benefit scores, an immediate question might be: why it is still
being collected? Similarly, where new data requests score low on benefits, it may not be necessary to
follow a full cost-benefit analysis.

								
To top