Document Sample
34 Powered By Docstoc
					                        OECD WORKSHOP
                Evaluating Agri-environmental Policies

                       Paris, 6-8 December 2004

                   Session 4. Evaluation of Payments

Evaluation of agri-environmental measures
          in Flanders (Belgium)

       Discussion of the RDPs Mid-Term evaluation results (2003)

                                                                        Koen Carels
                                                                Dirk Van Gijseghem
                                   Division for Agricultural Policy Analysis (DAPA)
                                                Flemish Agriculture Administration
                                                                   Brussels, Belgium

1        Agri-environmental measures (AEM) in Belgium are implemented trough the Rural
Development Programs (RDPs) of the Second Pillar of the Common Agricultural Policy of the European
Union. There are two rural development programs in the country’s two regions: Flanders and Wallonia.
Flanders was late in implementing AEM. Even regulation 2078/92 was not put into practice in Flanders.
With the Rural Development Plan 2000-2006 Flanders tried to catch up on AEM with the other European
countries. Despite little experience with AEM, about 12 AEM have been implemented since the start of
the program. A mid-term evaluation by an independent evaluator in 2003 considered the 2000-2002
period. The evaluation was done in the framework of the EU common monitoring and evaluation
approach. For this purpose, the European Commission (EC) had defined common evaluation questions,
criteria and indicators to be used by program evaluators. Most of the evaluation approach went to
fulfilling the EC demands, which aimed mostly at assessing the environmental effects of the AEM. The
evaluator was full-time assisted by a staff member of the Division for Agricultural Policy Analysis
(DAPA). This set-up proved to be very successful. Too early to measure results and impacts, the
evaluator however made useful recommendations to program management. The importance of an
integrated monitoring and evaluation framework was shown. Also a survey among farmers
implementing AEM was held. The analysis delivered useful information about what’s happening on the


Belgium, Flanders, mid-term evaluation, agri-environmental, RDP, survey


2       This paper is to be presented at the OECD Workshop on Evaluating Agri-environmental Policies
in Paris, December 6-8, 2004. It will be part of a session focusing on the evaluations of agri-
environmental payments that have taken place as part of the mid-term review of EU member countries’
rural development plans.
3       For the purpose of the workshop, the Secretariat developed a series of questions to guide authors.
The Secretariat argued that ‘addressing these questions would greatly assist the comparability of the
papers presented, facilitate better discussion and enable conclusions regarding the workshop objectives to
be drawn more easily’. It even suggested using the questions as section headings. This paper follows
these directions. This means that the paragraphs hereafter will discuss the questions. Finally some
overall conclusions will be drawn.


4       Agri-environmental measures (AEM) in Belgium are implemented trough the Rural
Development Programs (RDPs) of the Second Pillar of the Common Agricultural Policy of the European
Union. There are two rural development programs in the country’s two regions: Flanders and Wallonia.
Because of political decisions to dismantle the existing Federal Ministry of Agriculture in 2001, there is
no longer coordination on agriculture policies on a Belgian level. The same decision had been made in
the early 1990s about environmental policies. Agricultural and environmental policies are under the full
authority of both regional governments.

5       This means that concerning the RDPs and more specific the AEM that are part of it, the two
regions developed their own policy without any consultation in a Belgian context. The federal stage was
omitted and a direct consultation between regional authorities and the EC took place. This paper will
discuss the Flemish case, which is the region in the north of the country (Figure 1).

Figure 1. Two (independent) RDPs in the Belgium’s two regions Flanders (north) and Wallonia


6        The Rural Development Program for Flanders 2000-2006 (RDP) aims primarily at improving the
economic viability of the agricultural sector. Special attention however is given to the enlargement of
activities in the agricultural sector and the increased care for the environment. Policy instruments
addressing the latter are the Agro Environmental Measures (AEM), a major component of the program.

7       In 2003, a mid-term evaluation of the program was outsourced. The commissioning authority
was the Division for Agricultural Policy Analysis (DAPA), part of the Flemish Agriculture
Administration. The study was carried out by a consortium of five organizations: the Department for
Agricultural Economics and the Laboratory for Forestry (both are divisions of the University of Gent),
Belconsulting, the Centre for Agricultural Economics and IDEA Consult. Among those five, one
specialized in environment and another in agricultural economics. In charge of the consortium was Idea
Consult, a Belgian based consulting company and part of Ecorys, a multidisciplinary group of European
research- and consulting companies.

8       But the above-mentioned DAPA did more than just outsourcing. The terms of reference set forth
the role of DAPA in the whole process. In this, DAPA was the counterpart of the consortium. To this
purpose it made one member of staff almost full-time available. The set-up proved to be most useful in
guiding the consortium trough the muddle of responsible administrations, this to find the required
contacts and to obtain the proper indicators without losing track of the evaluation objectives. Because
DAPA’s other task concerning the RDP, the collection of relevant output indicators and the annual
progress report, its experience was highly appreciated by the consortium.

9        The 2003 study was the first in Flanders analyzing the existing AEM as a whole. Because the
existing measures were far from being managed coherently, analysis proved difficult. The reason for this
divided management was twofold: On the one hand a complex and continuous changing institutional
situation caused by the gradual shift of agricultural and environmental authority to the regional
governments. On the other hand a rather reluctant attitude towards implementing agri-environmental
measures in Flanders in the 1990s.

10       Flanders was late in implementing AEM. It did not implement article 19 of Regulation 797/85,
not even when European support was attributed to these measures as described in Regulation 1760/87.
Even regulation 2078/92 was not put into practice in Flanders. The European Commission approved the
Flemish environmental program already in 1994 but it was only in autumn 1999 that this program was
translated into applicable AEM.

11       With the Rural Development Plan 2000-2006 Flanders tried to catch up on AEM with the other
European countries. The preparatory work of the 1990s became the core of the agri environmental
chapter in the RDP 2000-2006. By the end of 2000 the program had been approved by the EC so
implementation of the policy could finally begin. As shown in Figure 2, the ecological expenses (mostly
AEM) represent 28,4% of the total RDP expenses in 2003. This figure doesn’t differ much of the share
of total ecological expenses in the total expenses for the whole program period 2000-2006.

Figure 2. Percentage of ecological, social and economical expenses in the Flemish RDP in 2003



Source: Mira T Environmental Report, 2004

12       The assessment of the AEM in 2003 took place in the context of a mid-term evaluation of the
whole program. This evaluation considered the 2000-2002 implementation period of the AEM. The
measures subject to evaluation (Box 1) can be divided into two types: (1) measures directed at farming
practices and (2) measures directed at nature and landscape management. Although most of the measures
had been running since 2000, that wasn’t the case for the integrated program management. At the start
up, there was little awareness for the need to collect monitoring and evaluation indicators. This lack of
awareness combined with the early stage of this evaluation (barely three years after implementation of
the measures and only two years after official EU approval of the program) made it for the evaluator a
difficult task to measure the first results.

Box 1. AEM applied trough the Flemish RDP 2000-2006

    (1) AEM directed at farming practices
    x    Soil cover
    x    Mechanical weeding
    x    Reduction of fertilizers and pesticides in ornamental plant cultivation
    x    Conservation of genetic diversity of local species threatened by extinction
    x    Organic farming
    x    Integrated fruit production
    x    Reduced fertilizers compared to the standard ‘Vulnerable area water’ (Water Management)
    x    Conversion of traditional to organic pig farms

    (2) AEM directed at nature and landscape management
    x    Management of meadow birds
    x    Management of field edges
    x    Restoring, planting and maintaining of small landscape elements
   x Botanical Management (Nature Management)
Source: RDP Flanders, 2000


13        In the RDP, among the ten strategic goals of the program, 4 are related to environment:
     x    Promotion of high quality production methods that are environmentally friendly, animal friendly
          and socially acceptable;
     x    Fast development of multifunctionality, organic farming and direct marketing;
     x    Involving farmers and horticulturalists in nature development within agricultural areas and
          beyond in delimited areas;
     x    Conversion to a sustainable water management.

14      But the relation between these strategic goals and the AEM chapter is not transparent. No
specific objectives were defined for the measures. In fact, for the 12 AEM of the RDP, only hazy
expected outcomes were defined (Table 1).

Table 1. Expected outcome of the AEM in the Flemish RDP 2000-2006

              Agri-environmental measure                      Expected outcome
Soil cover                                                    - Prevention of soil erosion
                                                              - Water quality improvement
                                                              - Increase humification
Mechanical weeding                                            - Use-reduction of plant protection products (PPPs)
Reduction of fertilizers and pesticides in ornamental plant   - Use-reduction of plant protection products (PPPs) and
cultivation                                                     fertilizers
Conservation of genetic diversity of local species            - Preservation of endangered breeds
threatened by extinction                                      - Increasing biodiversity
Management of meadow birds                                    - Preservation and protection of meadow bird populations
                                                              - Increasing biodiversity
Management of field edges                                     - Water quality improvement
                                                              - Prevention of soil erosion (if edges along rivers)
Restoring, planting and maintaining of small landscape        - Increasing biodiversity
elements                                                      - Landscape development
                                                              - Increasing biodiversity
Botanical Management (Nature Management)
                                                              - Water quality improvement
Reduced fertilizers compared to the standard in               - Water quality improvement (lower nitrate concentrations
‘Vulnerable area water’ (Water Management)                      on bottom and surface water)
                                                              - Increasing organic farming
Conversion of traditional to organic pig farms
                                                              - Decreasing manure problem
                                                              - Countering pollution
                                                              - Increasing biodiversity
Organic farming
                                                              - Improving soil quality
                                                              - Improving food safety
                                                              - Countering pollution
Integrated fruit production
                                                              - Improving food safety
Source: RDP Flanders, 2000

15      There were about 90.000 contracted hectares in Flanders in 2003. However, several AEM can be
applied to the same parcel, so the real surface under one or more AEM is estimated to be about 60.000
hectares, or 10% of the Flemish farmland. No ranking of importance of the different measures is given
in the RDP. However, total number of hectares per measure and expenditure data can give an indication
(Table 2).

Table 2. Number of hectares and expenditure for AEMs in the Flemish RDP in 2003

                                                                     contracted              expenditure
                              Measure                                                %                       %
                                                                    surface (ha)             (1000 EUR)

Soil cover                                                            39.582       43,26%      1.978       12,15%
Mechanical weeding                                                     2.627       2,87%        346        2,13%
Reduction of fertilizers and pesticides in ornamental plant
cultivation                                                             767        0,84%         72        0,44%
Conservation of genetic diversity of local species threatened by
extinction *                                                           3.704       4,05%        200        1,23%
Management of meadow birds                                              499        0,55%        227        1,40%
Management of field edges                                               308        0,34%        225        1,38%
Restoring, planting and maintaining of small landscape elements        5.602       6,12%        186        1,14%
Botanical Management (Nature Management)                               1.934       2,11%        612        3,76%
Reduced fertilizers compared to the standard ‘Vulnerable area
water’ (Water Management)                                             30.709       33,56%      10.540      64,77%
Conversion of traditional to organic pig farms                           6         0,01%         5         0,03%
Organic farming                                                        2.861       3,13%        667        4,10%
Integrated fruit production                                            6.613       7,23%       1.214       7,46%

TOTAL BUDGET                                                          91.508       100,00%     16.272      100,00%
Source: Annual Progress Report RDP Flanders, 2003

16     As shown in the table, the foremost important measure is the one addressing the nitrate problem.
The measure is called ‘Reduced fertilizers compared to the standard ‘Vulnerable area water’ (Water
Management)’ and consumes almost two thirds of the budget of the AEM chapter in the RDP. Second
most import measure in budget terms is ‘Soil cover’ while ‘Integrated fruit production’ ranks third. The
most economical measure, ‘Conversion of traditional to organic pig farms’, costs about 5000¼ ,W
concerns only two farms and can hardly be called a success. Using the classification of Box 1, 92% of
the AEM budget is directed at farming practices. Only 8% is directed at nature and landscape


17      Different aspects of AEM were analyzed. Because of the early stage of implementation, most of
the evaluator’s attention was directed at how public authorities were implementing and monitoring the
measures and what the first results and the possible environmental effects were. Several types of criteria
were used at this.

Progress of implementation

18      The most easily verifiable criterion is the ‘progress of program implementation in relation to the
targeted output’. For instance, a measure aims at a surface of X hectares in 2006. The criterion is then
‘X hectares’. In fact, these kinds of criteria should be an integral part of any program. However, not all
targets were well-considered in the Flemish RDP and some had to be adjusted or re-interpreted (by the
evaluator and in consultation with the administration) in order to give a realistic judgment of the progress
of implementation.

Accuracy of the support

19      Compliance with article 24 of Regulation 1257/99 was also verified. This article firstly states
that support in respect of an agri-environmental commitment shall be calculated on the basis of the
income foregone, the additional costs resulting from the commitment and the need to provide an
incentive. So the criterion was ‘accuracy of the support’. Interesting is that next to a recalculation of the
costs of each measure, a survey was done to ask the involved farmers whether they found the support to
be sufficient.

Soft effects

20      The survey examined the ‘soft effect of the measure’. Changes in attitude of the farmers were
measured, next to awareness with respect to the environment. These kinds of questions are very relevant
to keep authorities informed of what’s happening on the field.

Efficiency of implementation

21      Although implementing costs, both for public authorities and private sector have not been subject
to evaluation, ‘efficiency of implementation’ was also a criterion. Because of the long procedures, the
many organizations involved, the rather decentralized organization and the many rules to comply with,
the evaluator pointed at the high governmental costs of implementing AEM in Flanders. The same
applies to the high (transactional) costs to be made by the private sector. The criterion was qualitative,
so no calculations of costs were made.

Environmental effectiveness

22      The last and most difficult type of criteria in the evaluation was about ‘environmental
effectiveness’. These criteria were all part of the evaluation framework set up by the European
Commission. The criteria were about soil, surface and ground water quality, biodiversity and landscape.
Because of the very early stage of the program, most of the answers to these common evaluation
questions could not be quantified and were most difficult to underpin.


23      The methodology of the evaluator consisted first of all in collecting relevant data and knowledge
in order to assess the progress of program implementation. For this purpose, actors involved in the
public authorities were consulted frequently. This was necessary because the different AEM are
managed by three different governmental administrations, each with its own corporate culture, approach
and management of the measures. To say it this way, the only thing the different measures had in
common, was that they were (accidentally) part of the same program.

24      Very useful in the organization of the evaluation process was the assistance offered by the
Division for Agricultural Policy Analysis (DAPA), the principal of the evaluation assignment. As
already mentioned above, one member of staff was almost full-time occupied in assisting the consortium.

25      Interviews were held with experts of different environmental issues, like soil, birds, biodiversity,
water quality etc. An important part of the data collection consisted of holding a survey among farmers
applying AEM. This was done to collect the necessary data not available in the administration’s
monitoring system. The survey consisted of about 200 farmers, implementing and not implementing
AEM in the whole region. They were questioned about their attitude, whether they found the support
sufficient and what they thought the impact of the measure was on their farmland and on biodiversity in
general. The sample of 200 farmers was taken out of the Flemish Farm Accountancy Data Network
(FADN), so also financial data could be analyzed.

26      The analysis consisted further of making some cross tables and setting out the frequency of
different types of answers. Concerning one AEM called ‘Soil cover’, for one specific case (a
municipality) a GIS-overlay with maps indicating soil sensitivity for erosion was made.


27      Two methodologies for evaluating the measures can be distinguished: ‘the no policy situation’
and ‘farmers not affected by the policy’.

No policy situation

Example: survey among 109 farmers receiving AEM support
28      In the survey, 109 farmers receiving AEM support were asked ‘would you have started
implementing AEM without support?’. Table 3 gives the answers per measure. A surprisingly 51% of
the farmers stated that they also would have started with the AEM without support. Only 28% answered
that they never would have started without support. According to these figures, support was very
relevant for ‘Management of meadow birds’ (75%) and ‘Mechanical Weeding’ (44%) and almost not
relevant for ‘Reduction of fertilizers and pesticides in ornamental plant cultivation’ (100%) and
‘Integrated fruit production’ (75%).

Table 3. The influence of support on the decision to start with an AEM according to survey results

                                                              would have     would have
                                                                                               have       wouldn’t
                                                                started     started but on
                          Measure                                                             started    have started   number
                                                                without        a smaller
                                                                                             but later      (%)
                                                              support (%)     scale (%)
All measures with support                                          51            19              2           28          104
Soil cover                                                         50            21             3            26          80
Mechanical weeding                                                 56             0             0            44           9
Reduction of fertilizers and pesticides in ornamental plant
                                                                   100            0             0             0           3
Management of meadow birds                                          0            25             0            75           4
Management of field edges                                          25            50             0            25           4
Restoring, planting and maintaining of small landscape
                                                                   40            20             0            40           5
Botanical Management (Nature Management)                           67            17             0            17           6
Reduced fertilizers compared to the standard ‘Vulnerable
                                                                   35            17             4            43           23
area water’ (Water Management)
Integrated fruit production                                        75            25             0             0           8
Source: Mid-Term Evaluation RDP Flanders, 2003

Example: case at municipal level
29      The effect of the measure ‘Soil cover’ was calculated. For one specific community (case study),
the tons of avoided erosion were calculated. For this purpose, an overlay of the parcels with the AEM
‘Soil cover’ and a soil map indicating sensitivity for erosion was made. In literature it was found that
soil cover results in a reduction of erosion of 50%. Compared to the ‘no policy situation’ however, one
needs to take into account the results of Table 3. Concerning ‘Soil cover’, 50% would have started
without support, so the real net effect of that measure is half of the calculated reduction.

Example: interviews with experts
30      Experts were interviewed and literature was reviewed in order to get an answer to the question of
‘no policy’. For instance concerning one measure called ‘Management of meadow birds’ experts found
the parcels on which the AEM were applied too fragmented. It was recommended that the measures,
although they are only eligible in designated areas, should be more part of an integrated territorial
approach. The measure should also be accompanied by other measures and enforced for all parcels in the
area. Without the condition of no fragmentation, the situation is not expected to make much of a
difference with the ‘no policy situation’.

Farmers not affected by the policy

Example: survey among 68 farmers implementing AEM without support
31      In the survey, 68 farmers indicated that although they were implementing AEM, they were not
receiving any AEM support. Table 4 gives the survey results per measure. The reason for not receiving
support is mostly the administrative burden (25%) and not being informed of support possibilities (25%).
The administrative burden seems to be very high in the measure ‘Reduction of fertilizers and pesticides
in ornamental plant cultivation’(50%).

32       Of course these last figures are not that straightforward. Most of the farmers think that the
efforts they are doing are sufficient to satisfy the requirements of the AEM. However, the terms of
reference of the AEM contracts are quite demanding. As stated in the Rural Development Regulation,
‘Agri-environmental commitments shall involve more than the application of usual good farming
practice’. Probably the farmers are strongly overestimating their own agri-environmental efforts and are
in most of the cases not eligible for support.

Table 4. Reasons for not receiving support for AEM by farmers in 2003

                                                                                                         too large
                                                   not eligible not interested support too              administrat   number
                     Measure                                                                 aware
                                                       (%)           (%)        small (%)               ive burden      (%)
 Without support                                       21               9          15         15             25         68
 Soil cover                                            18               10         16         5              30         50
 Mechanical weeding                                    24               0          18         24             24         17
 Reduction of fertilizers and pesticides in
                                                       17               17         17         0              50         6
 ornamental plant cultivation
 Management of meadow birds                                                                                             0
 Management of field edges                             13               0          0          25             0          8
 Restoring, planting and maintaining of small
                                                       6                0          18         12             29         17
 landscape elements
 Botanical Management (Nature Management)              23               8          0          15             31         13
 Reduced fertilizers compared to the standard
                                                       22               11         6          11             22         18
 ‘Vulnerable area water’ (Water Management)
 Integrated fruit production                           0                0          0          0              0          1
Source: Mid-Term Evaluation RDP Flanders, 2003

Example: survey about future income
33      In the survey, all farmers were questioned about their future income expectations. Three groups
can be compared (Table 5): farmers implementing AEM with support, without support and not
implementing AEM. Most of the farmers don’t seem very positive about their future but those who are
most positive are those not implementing AEM. In fact, least positive are the farmers implementing
AEM with support. A possible explanation for this could be that the farmers implementing AEM
according to the terms of reference are most aware of the issue of agriculture and environment.

Table 5. Survey on expected future income in three groups of farmers: implementing AEM with
support, without support and not implementing AEM

                                                           Moderately         moderately
    Future income (%)              very positive                                                   very negative        total
                                                            positive           negative
AEM with support                         1                    28                  53                    17              103
AEM without support                      2                    32                  48                    18              66
No AEM                                   4                    48                  44                    4               54
Source: Mid-Term Evaluation RDP Flanders, 2003

Example: diary cattle farmers
34      Because support for investments is the largest expenditure of the RDP, most attention went to the
analysis of this chapter. Investment support in the RDP is available for almost all kind of farming types,
as long as the criteria for support are fulfilled. To answer the evaluation questions on investments, the
evaluator chose to focus only on one type of farming, the specialized diary farmers. Out of the Flemish
FADN, 87 farms were selected of which data was available for the period 1995-2001. In that period, 32
farms made investments and therefore received investment support, 26 farms made investments but
without investment support and 29 farms made no investments at all.

35      Investment support is a measure already existing since 1994. Because too limited data was
available for the period 2000-2002 (no data for 2002 was available) and because the measure now
included in the RDP does not differ much of the one before 2000, data from 1995 on was taken into
account to answer the RDP evaluation questions.

36     Because of the large amount of data in the FADN, almost all economical indicators at farm level
concerning the three groups could be calculated. Table 6 gives an example of a calculation of revenues
from milk and meat production per worked hour. All figures are statistically significant and show the
economical advantages of investment support.

Table 6. Revenues from milk and meat per worked hour (euro/hour) in three groups of farmers

  Revenues of milk and meat per
                                  1995            1998           1999            2000           2001
     worked hour (euro/hour)
Investments with support          26,95          30,37           30,84          32,23           34,53
Investments without support       24,10          29,25           28,61          30,52           30,09
No investments                    22,85          25,04           24,79          24,96           26,43
Source: Mid-Term Evaluation RDP Flanders, 2003

37       Because these farmers were also part of the survey, they were asked how their environmental
situation had changed since 2000 (Figure 3). Although not all investments are directed at environment,
the figures give an indication of the positive impact of investment policy of the RDP on the environment
according to the farmer’s perception.

Figure 3. Perception of environmental improvement in three groups of farmers

                                               61% 60%
   40%                                                                       33% 31%
   30%                                                                                    26%
   10%           6%

             strong improvement                 improvement                     no change

       no investments         investments without support            investments with support
Source: Mid-Term Evaluation RDP Flanders, 2003


38       No methods for this purpose were used. All of the attention of the evaluator went to the
collection and interpretation of relevant data completed with expert analysis.

39     Problems with ‘disentangling’ are that the same objectives are pursued by different measures,
both part of the RDP as outside the RDP. So what is the contribution of that specific measure?
Moreover in Flanders, measuring effects of single AEM would be very difficult because the surface
would become very small and disintegrated for most of the measures. About 10% of the farmland is
under 12 AEM, so little impact of single measures is to be expected.


40       The survey is definitely the stronghold of the evaluation of the AEM. Because the farmers are
the target group of the AEM, a survey measuring their attitude and incentives is key relevant information
for policy makers.

41      Farmers affected and not affected by the policy were studied in the analysis of the investment
chapter of the RDP. Among those not affected by the policy, it was important to distinguish two groups:
farmers implementing the measure (or a kind of) but not eligible for support and farmers not
implementing any kind of measure at all. An analysis of the arguments of those not eligible can give
useful information.

42      Next to attitude, economical performance indicators were calculated for three groups of farmers:
investments with RDP support, investments without RDP support and no investment. The last two
categories are farmers not affected by the policy. Data from the FADN was used for the period 1995-
2001. Only the farming type ‘specialized diary’ was taken into account. Because of this focus on one
type, indicators were easier to compare and so conclusions could be drawn.

43      The EU common evaluation framework was difficult to implement. Not all questions were
relevant for the Flemish program and moreover, proposed criteria and indicators were too detailed.
Almost all the questions were aiming at an assessment of the environmental impact of the measures. But
expecting an in depth impact analysis of a mid-term evaluation is not realistic.

Progress of implementation

44      Not all AEM were on schedule. For some measures, no or no realistic targets had been defined.
But altogether, taking the difficult start of the RDP into account, the evaluator concluded that the AEM
chapter was gathering momentum.

Accuracy of the support

45     Not all support meets the needs. For some measures the support should be raised in order to
comply with article 24 of Regulation 1257/99.

Soft effects

46     There is a positive influence on the mentality of the farmers to be more environmentally aware.
Moreover, support for AEM at an (actual) small part of the total population could influence farmers to
convince other (new) farmers.

Efficiency of implementation

47      The managing authority was made aware of the high transaction costs involved in program
implementation. These costs can be reduced through a more integrated approach to the program.
Fundamental to performing an evaluation is the existence of a comprehensive monitoring system.
Although separated monitoring systems were available, more integration is necessary. In fact, a fine-
tuning of monitoring at all levels needs to be done. At present, there are still four types of data collection
in RDPs (monitoring indicators of the EC, evaluation indicators of the EC, financial data and data
collected in the administrative divisions).

48       The integrated approach assists in the set-up of different targets of the program at different levels
(strategic, output, result, outcome). This allows program management to focus more on results and
targets to be achieved.

Environmental effectiveness

49      Looking at the soil, one can definitely say that the measures are reducing soil erosion and
chemical contamination of soils. The reduction in sediments and chemicals can easily be calculated by
comparing the good agricultural practice with the agricultural practices under the AEM. Of course, the
percentage of farmers declaring in the survey that they would have implemented the measures without
support, need to be subtracted to calculate the net policy effect.

50      Water quality is more difficult to measure because not all observations of a certain (bad) water
quality are due to agriculture and not all improvements can be attributed to AEM. Furthermore, because
AEM are not restricted to certain areas but are generally applicable, one parcel can be under an AEM
while the adjacent parcel may not. So only with a sufficient and concentrated area under AEM one may
measure water quality improvements in a specific area. At present, no measurable effects can be shown
for Flanders.
51      The area of farmland under agreements targeting particular wildlife species was calculated but
evidence for a positive relationship between the supported actions and the abundance of the targeted
species could not be given.

52     Generally spoken, all AEM contribute in one way or the other to a better environment.
Moreover, most of the AEM contribute both to the soil and water quality and to biodiversity and
landscape. Multiple objective policy measures are difficult to evaluate.

Ex-ante versus ex post

53       Concerning the results and the expected outcome, the ex-ante noticed a lack of criteria and target
levels for all the AEM in the RDP. Only at output level of the measures (number of hectares, number of
farmers), quantitative targets had been specified. The ex-ante however didn’t elaborate the lacking
targets and indicators itself.


54       By the European Commission’s common monitoring and evaluation framework, the managing
authority of the Flemish RDP was obliged to perform a mid-term evaluation of the program. At the start
of the program, little action was taken to underpin the program with indicators and criteria. In fact, little
was done to prepare a mid-term and ex-post evaluation. The only monitoring foreseen at program level
consisted of the financial data from the paying agency and the indicators of the annual progress report
that has to be sent to the EC each year. Despite these problems caused by the difficult start of the
program, the evaluator together with the DAPA realized a useful report. It contained recommendations
for the different administrative units on program implementation. It brought insight in how an integrated
monitoring and evaluation framework should be organized. The survey was very useful to keep
authorities informed of what’s happening on the field. For instance, the attitude of farmers implementing
the policy could be compared with farmers not affected by the policy. Moreover, the use of the FADN
made it possible to calculate financial result indicators at farm level.

55     More difficult however was linking the measures with the environmental results on the field.
Because of the early stage of the program implementation and the lack of data, little results on this part
were achieved.


Annual Progress Report RDP Flanders. (2003). Brussels, 37p.
Mid-Term Evaluation RDP Flanders. (2003).           Evaluatie van het Vlaamse Programma voor
Plattelandsontwikkeling. Consortium van de afdeling Landbouweconomie en het Laboratorium voor
Bosbouw van de Universiteit Gent, Belconsulting, het Centrum voor Landbouweconomie en IDEA
Consult, Brussel. IDEA Consult, 536 p. (with english summary)
Mira T Environmental Report. (2004). Brussel, 488p.
RDP Flanders. (2000). Programmeringdocument voor Plattelandsontwikkeling in Vlaanderen van 2000-
2006. Ministerie van de Vlaamse Gemeenschap, Dept. EWBL, Administratie Land en tuinbouw, ALT,
Brussel, 351p.


Shared By: