RIA Guidelines Economic Indicator by mikeholy


                                 Support for Improvement in Governance and Management
                             A joint initiative of the OECD and the European Union, principally financed by the EU

                                CONTRIBUTION TO GENERAL DISCUSSIONS
                                        PARTS I, II, III, AND IV

                                                   MR. MARIO MARTELLI

                                                          CONFERENCE :

                                      REGULATORY IMPACT ASSESSMENT


                                                           19-21 April 2006

This document has been produced with the financial assistance of the European Union. The views expressed herein are those of the author, and can
in no way be taken to reflect the official opinion of the European Union, and do not necessarily reflect the views of the OECD and its member
countries or of the beneficiary countries participating in the SIGMA Programme.
                                      TABLE OF CONTENTS

INTRODUCTION ....................................................................................... 1

INTERVENTION........................................................................................ 1

2. OBJECTIVES OF THE REGULATORY ACTION ........................ 2
  2.1.     THE REGULATORY AGENDA............................................................... 2
  2.2.     INDICATORS ...................................................................................... 3
  2.3.     MONITORING AND EVALUATION IN RIA ........................................... 3
3. INFORMATION NEEDS FOR RIA .................................................. 3
  3.1.     DIRECT OR IN DIRECT DATA COLLECTION.......................................... 4
  3.2.     DATA COLLECTION EFFECTS ON RIA ................................................ 5
  3.3.     DATA COLLECTION – A FEW REMARKS .............................................. 6
4. RIA CONSULTATION TECHNIQUES ........................................... 7
  4.1.     FOCUS GROUPS ................................................................................. 8
  4.2.     PANEL............................................................................................... 8
  4.3.     SEMI-STRUCTURED INTERVIEWS ....................................................... 8
  4.4.     SAMPLE SURVEYS............................................................................. 9
  4.5.     NOTICE AND COMMENT ...................................................................... 9


Good data collection strategies are to be considered crucial and at least as important as
other component of the analysis. The analyst should then plan in advance and develop
a reliable inventory of data sources, taking into consideration what explained in the this

This section is aimed at introducing the existing data collection mechanism for RIA and
data needs, vis-à-vis with the relevant government‟s current activities.

The regulatory intervention has always to be motivated by specific social, economic and
juridical needs. Therefore, the proposing Administration will have to carry out an
analysis to provide a clear and precise picture of the circumstances leading it to
propose a legislation revision or an ex-novo creation of one. In this phase, the
Administration has to carry out a CONTEXT ANALYSIS strongly connected to a
quantitative analysis.

In the respect of transparency, which has to characterize RIA, in presenting the needs
scenario, the subject expressing the need in question will always have too be clearly
specified; the same applies to the information sources (self identification, suggestions
from Administrations and/or different subjects, public events, outcome of consulting

                           EXAMPLE REGARDING THE NEEDS
     The description of a need in the following terms: “According to the Trade Union X,
     there are too many pathologies caused by the massive use of information
     technologies” is too generic. The proposing Administration should be able to report
     further details on it, for example as follows: “The Trade Union X reported that, on
     the basis of a research conducted by the enterprises Y, Y1, Y2 and Y3 using the
     methodology M, in the a, b and c productive sectors, on a total number of k workers
     using computers for at least z hours a day, the pathologies d, e and f, with r
     frequency, are observed”.

The final objective is to obtain a regulation which, in the respect of efficiency and
effectiveness criteria, is able to modify behaviours in order to accurately meet the
needs of the reference population.

                                                                                       Page 1
In particular, with reference to the needs, the attention has to be focused on the
provision of the services connected with the regulation under examination and on their
capability to meet the demand both in quantitative and qualitative terms. The real
innovation introduced by RIA is the attempt to include in a rigorous quantitative
analysis, variables that reflect the frame of mind that the situation object of the study
arouses in the stakeholders. The effort of a RIA analyst is indeed to quantify the
qualitative data collected and approximating them with the quantitative variables which
are the best approximation to them or the influence of which turns out to be especially

     ‘Have a head for FIGURES.’ That is to say, we must attend to the quantitative
     aspect of a situation or problem and make a basic quantitative analysis. Every
     quality manifests itself in a certain quantity, and without quantity there can be no
     quality. To this day many of our comrades still do not understand that they must
     attend to the quantitative aspect of things -- the basic statistics, the main
     percentages and the quantitative limits that determine the qualities of things. They
     have no ‘figures’ in their heads and as a result cannot help making mistakes.
     Methods of work of party committees. This was part of Mao Tse-tung's concluding speech
     at the Second Plenary Session of the Seventh Central Committee of the Communist Party of
     China (March 13, 1949)

The implementation of regulatory interventions to achieve Government policy
objectives should be the object of a scrupulous programming, articulated in long,
medium and short term actions. The regulation has a continuously growing crucial role
in the implementation of the Government programmes. RIA is a methodology for a
preliminary assessment of the regulations to be introduced, and of its connected to it
effects; therefore, it is becoming an indispensable functional tool for the identification
of “good quality” norms to reach predetermined objectives.

The programming of regulatory interventions should use a very useful operational tool,
the REGULATORY AGENDA, which shows in a chorological order the measures to be
adopted, in relation to the identified objectives. The adoption of the Agenda is
functional to RIA programming in a double way:

-    It clearly shows the regulatory interventions which are selected for an impact
     analysis over time: this allows a “timely” allocation time and resource by the
     Administrations involved in the analysis;
-    If widely published, it allows the recipients of the interventions to get ready on
     time to participate knowingly to the consultations associated with the exercise.

                                                                                            Page 2
This term defines the specific condition, and/or modification of the initial situation
which, when it occurs, is supposed to guarantee the achievement of the associated
objective. Indicators must be quantifiable as regards time and space, that is, they can be
expressed through a numerical value, thus enabling objective verification of their
accomplishment. Indicators can be considered as a quantitative representation of
the objective intended to be achieved, and, as such, they provide information
regarding the extent of the expected impact of the regulation application. The
identification of appropriate indicators of the objectives is also useful for the assessment
phases following RIA (intermediate and ex-post evaluation). Such assessment can be
more easily carried out if the indicators have already been selected in the preparation
phase of the regulatory measure.

Policymakers need systems capable to verify whether implementation is performing,
and to what extent the regulation is achieving its expected objectives. Then, when the
regulation is not achieving its objectives, they also need to know whether this is due to
flawed policy design or poor implementation in terms of i) identification of problems;
ii) link problems-objectives; iii) sufficient capability in the implementation phase;
iv) arising of unknown assumptions/criticalities.

Monitoring and evaluation arrangements (M&E), including generating data on the basis
of carefully designed indicators provides valuable information in this regard and help in
defining how to optimise the intervention. In this respect, already when RIA is under
way, the analysts have to define some indicators for the main policy objectives and to
outline the monitoring and evaluation arrangements envisaged. Then, it is fair to assume
that these general objectives are reasonably stable across the various alternative policy
options envisaged in RIA. These indicators must be checked against the purpose they
are supposed to serve, i.e. measuring to what extent a policy has been properly
implemented and its objectives achieved. Consequently, their credibility and clarity are
very important. Another important factor in the choice of indicators is the ease with
which relevant data can be collected; collecting data on an indicator should not be more
costly than the value of the information they provide.

For a correct implementation of RIA, one basic element is represented by the collection
of a great variety of specific data in connection with various aspects: regulatory
measures, needs of regulatees (both expressed or unexpressed), field of action of new
measures, availability of human and financial resources etc…

                                                                                     Page 3
Years spent on testing RIA‟s efficiency, have highlighted the difficult experiences that
the concerned Administrations have gone through during the phase of collection of
necessary data. The problem is not so much the scarcity of data, but the fact that
Administrations are not used to think in quantitative term and, therefore, they are not
totally acquainted with data collection. This problem reflects the poor attitude of
Administrations towards managing their internal data in a systematic and functional
way; the result is a serious loss of information and a recurrent duplication of actions.
However, quite a number of experts concerned with the RIA for various reasons, think
that, in the long term, the progressive application of this technique will persuade/induce
Administrations‟ personnel to acquire a quantitative attitude enabling them to make
their data collection and organization operations (not only for regulation) faster, and
above all, effective.

This section includes a list of the various data sources made available for the
Administrations, showing pros and cons and outlining, as much as possible, a way of
approach for data collection that the Administrations may adopt to streamline their
research work, in both the short and the long term.

In any case, always keep in mind that:

    The analyst will usually need much highly specific data that is tailored to the
     questions raised by the specific regulation.
    RIA must start as early as possible to leave time for identification and
     satisfaction of data needs. This can be a time-consuming part of the RIA process.
    Much data needed is held by the regulated community. Data collection is to be
     considered even a public-private task. In this respect appropriate working
     relations with private sectors partners have to be developed.

At first, a rough distinction must be made, as regards sources, concerns the type of
collection: direct or indirect. Information is DIRECT when results arise from a specific
survey designed and implemented ad hoc to attain a precise objective. In contrast,
information is INDIRECT when derives from data previously collected with other
objectives, different from the current one, and possibly re-processed.

1.   If a DIRECT SURVEY is carried out in compliance with appropriate statistical
     collection/retrieval rules, it will result in a precise data collection which will be,
     therefore, more pertinent for the survey. However, in most cases, collection is
     very expensive, especially if - as it is the case of local and national level
     regulations – the reality to be surveyed is particularly complex. Moreover, in
     order to produce sound data, survey must be carried out by competently trained
     personnel, acquainted with the appropriate statistical methodologies and their
     implementation techniques. In most cases, external personnel is to be called to
     assist, and RIA costs will raise.

                                                                                    Page 4
       That is why, for RIA, Administrations generally adopt, indirect data, a fact that is
       often detrimental to the completeness and/or relevance of information. Recourse
       to consultation is an attempt to, somehow, balance these disadvantages especially
       as regards the regulatory fields of action where indirect survey is particularly

2.     INDIRECT data  can be extracted from either administrative and statistical archives
       or from other different types of archives. Administrative archives keep data
       collected by public Administrations for the current implementation of their
       activity. After some appropriate processing, these data can be also used for
       measurement objectives, therefore, also for the implementation of RIA.
       Some authors distinguish administrative sources into internal sources (data
       collected by Administrations on previous occasions) and external sources
       (information collected by other public bodies). Data kept in these archives present
       certain characteristics: in general they can be easily obtained and at low cost, their
       degree of relevance is high (otherwise they would not be taken into consideration)
       and they are, generally, reliable. Collection of these data is not ruled by any
       comprehensive methodology, therefore, comparison of data produced from
       different archives is often difficult. In contrast, statistical archives, gather
       together all data that officially authorized bodies have collected in compliance
       with methodologies and techniques of strictly statistical nature.
       One additional data source for Administration is the group of archives made
       available by universities, private research centres and sectoral organisations
       (enterprises and unions). When identified, these would provide particularly
       relevant and quite reliable data. Access however, is quite difficult and collection
       not so easy. Similarly to what happens with data kept in the administrative
       archives, their comparability is often insufficient.

From the above-described situation one should assume that Administrations can rely
upon quite a wide range of possible solutions for data collection. It seems that selecting
one source rather than another would generate certain consequences on results, in terms
of quality and costs. Therefore, Administrations might profit from a rigorous and
systematic organization of data.

A preliminary action for the identification of the data necessary for a correct
implementation of RIA is recommended, and for each datum deemed necessary, the
following issues should be highlighted:

i)     in which phase should the data be used;
ii)    which is the possible source for its retrieval;
iii)   how long would it take to retrieve it.

                                                                                      Page 5
With this kind of data retrieval organization, Administrations will clearly and
immediately have the notion of their information needs and, indicatively, of
relevant costs. Administrations should then be capable of addressing their choice
towards the solutions that would more efficiently apply in the specific context.

According to what previously mentioned, it would be useful for the Administration in
charge of RIA to start the systematization of data retrieved on various occasions as well
as a preliminary organization of appropriate techniques (according to a generalized
structure permitting to share them with every other Administration) for data retrieval
and systematization. In this way, in a few years, it should be possible to obtain a number
of archives containing homogenous and, presumably, relevant data.

Finally, one should notice that in the period of source selection, Administrations should
carefully consider the goals of their survey. In fact, information retrieval must provide
quantitative support to the analysis of the situation in which regulation intervention is to
be performed. In case of information concerning mandatory standards for the activity of
both individuals and community as a whole, there might be plenty of data available,
therefore, the choice of one source in place of another might distort analysis results (i.e.
with socio-economic indicators like poverty, education, access to services etc.). As
regards the present evolution of RIA, no specific provisions have been produced on this
subject. The choice is, therefore, made relying upon the capability to analyse and
understand the specific goals that RIA is to reach for the Administration.

Consider how to compensate if you cannot find enough data.

     In the absence of adequate valid data, clearly identified assumptions are necessary
      for conducting the RIA. Transparency is even more essential to test the
      assumptions. Analysis of the risks, benefits, and costs associated with regulation
      must be guided by the principles of full disclosure and transparency.
     Shift data costs to the stakeholders by asking questions in the consultation period
      (see Section 4). Those affected by regulations have the incentives to provide the
      data necessary to complete RIA.

Consider the risk of „data capture‟.

     Stakeholders will provide much of the needed data, with a high risk of biased
      RIA. This risk can be managed by diversifying data sources, a check and balance
     Data biases can also be detected by being completely transparent. When data are
      weak, the more external review that RIA receives, the better it is likely to be. The
      latter is also true for RIA as a whole. The more the process is open the more it is
      likely to be accurate. Furthermore, a transparent process enhances RIA

                                                                                     Page 6
Consider the organisational aspects for the RIA group.

    Stakeholders are probably not accustomed to support their requests with data and
     figures. It is necessary to define as soon as possible clear rule of the game.
    The analysis of data influences the composition of the RIA team: the role of
     statisticians is essential in finding, selecting, and validating data and data
     sources. This could require organizational consequences (the role of the statistical
     office during RIA is fundamental).

The consultation of regulatees and competent entities on regulatory issues, mostly aims
at identifying regulatory measures‟ direct costs. Nevertheless, it might also produce
information, data and opinions concerning both direct benefits and indirect benefits
and costs.

The use of appropriate consultation techniques allows RIA to propose itself as one of
the instruments capable of laying down and reinforcing regulatory proposals. The direct
involvement of stakeholders can, as previously indicated, give the basis of information
necessary for the formulation of public policies capable of more appropriately
responding to the needs of individuals and enterprises. In particular, the
Administration which adopts consultation can facilitate the acquisition of three types of
significant information: i) assessment and in-depth analysis of needs; ii) option
formulation and identification of the relevant pre-requisites; iii) identification and
subsequent qualification and quantification of the cost/benefit categories of potential
regulatees of intervention.

It must be clear that the consultation is not intended to establish with the concerned
regulatees any representation-negotiation relationship. This instrument is intended for
the assessment of the perceptions and evaluations of both regulatees of intervention
(not necessarily through their representations), and privileged witnesses, reducing the
information asymmetry of public decision-makers, who keep their role of sole actors in
charge of regulatory activity.

The adoption of a consultation technique must be evaluated on the basis of three
criteria: i) burdensomeness of implementation; ii) reliability and validity of expectable
results and finally iii) usefulness and exhaustiveness of results (for RIA‟s goals). The
most popular consultation techniques to collect data, elements and opinions are the
following (in progressive ascending order of significance, importance, complexity and
costs): i) Focus groups, ii) Panels, iii) Semi-structured interviews, and iv) Sample
investigations. Then, it‟s worth mentioning the so-called „notice and comment‟.

                                                                                  Page 7
The focus groups technique is carried out through targeted interviews, designed for
small groups. In particular, these interviews can be addressed, at the same time or in
diversified situations, to the categories of entities involved in the new regulatory
intervention: direct regulatees, experts, representatives of categories of citizens directly
or indirectly involved, competent public Administrations. The ad hoc trained staff shall
prepare, conduct, register and analyse each focus group. For each regulatory
intervention there might be from a minimum of 2 to a maximum of 6 focus groups,
depending on the diversification and number of positions to be represented.

This technique presents a variety of advantages: i) relatively low cost (design of focus
and analysis of data, moderators, attendance allowance for participants); ii) rapid
implementation; iii) in-depth analysis of issues (closed answer questionnaires would not
produce the same); iv) possibility to raise different issues and viewpoints and let them
interact. Under certain circumstances, this would permit to carry out an analysis more
accurate than what would be possible with a semi-structured interview, described here
below. Provided that focus groups are appropriately organised and interpreted, all
assessment criteria of consultation techniques seem to have been fully met. To ensure
good quality of results, the concerned Administrations could recruit external experts to
assist their personnel, who would thus acquire the necessary experience and
subsequently be in a position to prepare, conduct and autonomously analyse

4.2. PANEL
The panel technique consists in building up a group of individuals or enterprises who,
after a preliminary assessment of their availability, are recurrently consulted during a
well-defined period of time, presented with closed questions questionnaires (that is,
with fixed number and type of answers), or semi-structured questionnaires (that is, with
closed questions and many open questions, used when the number of panel components
is small). The multipurpose panel (that is, opinion of components is required on a
variety of issues) with closed questions has a relatively low cost, still its presents
considerable weaknesses as regards validity and reliability, exhaustiveness and
usefulness of results. On the other hand, the panels targeted for each case reduce (even
if only a bit) weaknesses in terms of validity and reliability, but are rather expensive.

With the semi-structured interviews technique, interviewers have a more or less
detailed layout for interviews, with prepared fixed contents. Depending on the
circumstances, interviews may be conducted with the “basic” list of issues to be
addressed and the formulation of a number of general questions. With RIA, semi-
structured interviews should be carried out both with potential direct and indirect
regulatees and with privileged observers. Semi-structured interviews have a balanced
burdensomeness and rapidity and can produce reliable, useful, reasonably exhaustive
and valid results, though not necessarily and strictly representative or in other words,
capable of accurately reproducing characteristics and attitudes of reference populations.

                                                                                     Page 8
The sample survey technique is based on structured interviews, addressed to
representative samples of the population(s) being analysed, with closed question
questionnaires (that is, fixed number and type of answers), mailed and self-completed.
Where necessary and/or more appropriate, questionnaires can be submitted by ad hoc
trained interviewers through direct or telephone interviews. High costs and long times
required for implementation make this technique unfit for the time usually required for
RIA and unfit in most cases: therefore, the use of sample investigation technique must
be very carefully evaluated.

It is a consultation technique addressed to those probably affected by the proposal. It
provides for the publication of a consultation document asking for written answers.

The advantages are the following: i) it could allow to know different points of view in
detail; ii) it is relatively cheap, especially if conducted through the Internet (possibility
to reach many persons). The disadvantages are the following: i) Risk to implicitly select
respondents (only those who have the required tools to answer); ii) the analysts needs
time for elaborating the answers (especially when open-ended questions are proposed);
iii) there is no interaction among respondents and between respondents and public

                                                                                      Page 9

To top