Contracts Outline Cardi by cyh20065

VIEWS: 23 PAGES: 41

More Info
									   The Royal Netherlands Embassy of Indonesia’s
                Assessment of the
UNDP North Maluku and Maluku Recovery Programme
                  (INS/01/A29)




                   Final Report
                  19 March 2004

                 Consultant Team

                   Ivan Nasution
                 Dea Widyastuty
                     Ida Siregar
            Robert Griffin, Team Leader
                              Table of Contents


                                                  Page

List of Abbreviations                             2

Executive Summary                                 3

1. Introduction                                   10
    Background of the programme                  10
    Background of the assessment                 11
          o Mid-term Assessment                   12
          o DFID Review                           12
          o Netherlands assessment mission        13
    Assessment methodology                       13
    Organization of the report                   14
    Subprojects in the assessment sample         15

2. Programme and Subproject Design                17
       Programme objectives                      17
       Subproject identification                 17
       Subproject design                         20
       Implementation arrangements               23
       Subproject review and approval            25

3. Monitoring and Reporting                       27
      Monitoring at the programme level          27
      Monitoring at the subproject level         28
      Progress reporting                         29
      Conclusions and recommendations            30

4. Subproject Implementation                      31
       Contracting processes                     32
       Reconciliation approaches                 32
       Conclusions and recommendations           32

5. Subproject Results                             33
       Results table                             34


Annex: List of Persons Interviewed                37




                                     1
                   LIST OF ABBREVIATIONS

CARDI       = Consortium for Assistance to Refugees & the Displaced in Indonesia
CBO         = Community Based Organization
CoBILD      = Community-Based Initiatives for Housing and Local Development
CPB         = Community Programming Board
CPRU        = Crisis Prevention Recovery Unit
CRP         = Community Recovery Programme
DEX         = Direct Execution
DFID        = Department for International Development
DIKDASMEN   = Pendidikan Dasar dan Menengah (Primary and Secondary School)
ICMC        = International Catholic Migration Commission
IDP         = Internally Displaced People
INGO        = International Non-governmental Organization
MCI         = Mercy Corps International
MPMBS       = Manajemen Pendidikan Menengah Berbasis Sekolah
             (School-based Secondary Education Management)
NEX         = National Execution
NGO         = Non Governmental Organization
NMMRP       = North Maluku & Maluku Recovery Programme
PADU        = Pendidikan Anak Dibawah Umur (Early Childhood Education)
PBHI        = Perhimpunan Bantuan Hukum & Hak Asasi Manusia Indonesia
            (Indonesian Legal Aid and Human Rights Association)
PLN         = Perusahaan Listrik Negara (State-own Electricity Company)
RAPBS       = Rencana Anggaran Pendapatan dan Belanja Sekolah (School Budget Plan)
SMU         = Sekolah Menengah Umum (Senior High School)
UN          = United Nations
UNDP        = United Nations Development Programme
UNOPS       = United Nations Office for Project Services
VSAT        = Very Small Aperture Terminal




                                     2
                                  Executive Summary

Background

The Royal Netherlands Embassy of Indonesia’s Assessment of the North Maluku and
Maluku Recovery Programme (NMMRP) managed by UNDP is the third in a series of
three reviews of the programme. The first of the reviews, an internal assessment
conducted by UNDP, emphasized the substance of the programme, particularly future
programme directions. The second, conducted by DFID, focused on the UNDP
management structure for crisis prevention and recovery in Indonesia. This assessment
focuses on operations and implementation issues.

The three main objectives of this assessment are:

   1. To review and assess the quality of the monitoring process in theory and in
      practice;
   2. To review the (sub) project identification, approval and contracting processes;
   3. To review the relationship between the activities undertaken and the original
      subproject proposal, and the extent to which subproject objectives have been
      achieved both qualitatively and quantitatively.

The NMMRP is too large for a comprehensive assessment in the time available so the
team selected a sample of 11 subprojects (total value $1.7 million) for intensive review.
The criteria used in sample selection were: inclusive of all three of the programme’s
main geographical areas; a variety of different types of partners; a variety of different
types of activities; and various sizes of subprojects. The report of the assessment has
been organized in the order of the processes in the project cycle.

General Comments

      The programme started slowly. The programming process implemented in the
       first area of work (Tobelo-Galela), designed to enhance participation, ended up
       taking nine months. In other areas, potential INGO partners were already in the
       field, but UNDP contracting requirements delayed their engagement.
      Many subprojects in the sample are still ongoing. Therefore, not all results have
       been produced.
      Observed and reported results are satisfactory for most infrastructure and INGO
       community-based work, but weak in the area of livelihood promotion activities.
      In a post conflict situation, programme personnel are new, there is high turnover,
       working conditions are difficult, and there is pressure to respond quickly. In this
       context, operating systems and procedures should be simple, be easy to
       understand and facilitate action. However, virtually all of the programme’s
       operational processes are over-engineered and cumbersome. A major effort at
       simplification is in order.




                                            3
Main Conclusions and Recommendations

1. Programme Objectives

The immediate objectives of the programme are as follows:

1) The return of internally displaced people (IDPs) into key communities in an
atmosphere of reconciliation;
2) The rehabilitation of community-level physical infrastructure and provision of
effective public services in key areas;
3) The initiation and growth of social and economic activity with a particular focus on the
most vulnerable; and
4) The strengthening of good governance practices and local governance capacities.

Conclusions. The programme objectives were never operationalized into statements of
results to be achieved. No indicators, targets or benchmarks were established. The
programme objectives appear to have been treated as guiding principles or areas of
emphasis for programme activities, not goals to be achieved by programme operations.

In the absence of indicators and targets, there is no practical guidance for the formulation
of the subprojects as to how they should be designed so that they can contribute to the
achievement of the objectives of the programme. Furthermore, monitoring or assessing
progress towards the achievement of programme objectives is not possible. In other
words, measurement of achievement of programme objectives can only be subjective.

Recommendations.
    Establish indicators and targets for achievement at the programme level for any
     follow-on programme to enable detailed programme design, monitoring, and
     results-based management of implementation.

2. Subproject Identification and Selection

Conclusions. The programme tried two methods for identification of activities: a
Community Programming Board (CPB) in Tobelo-Galela and an informal request for
proposals from INGOs, Government and civil society organizations in Ambon and
Ternate. The CPB process took too long and its integrity came into doubt. The INGOs
wrote good proposals and offered local experience and participatory approaches to
community development and reconciliation.

Recommendations. The next round of programme development should follow a more or
less normal programming process:
      Conduct needs assessments of intended beneficiaries to establish baseline data
        for future programming.
      Identify and assess potential partners in government and civil society that can and
        should play roles in meeting identified needs.




                                             4
       Establish a programme framework based on the foregoing analyses. Flesh out
        the detailed activities of the framework using appropriate consultative and
        participatory strategies. Cluster related activities for efficient management.

3. Subproject Design

Conclusions.
    The programme’s subproject design procedures allowed the creation of too many
      small, unrelated subprojects increasing the management burden and diminishing
      impact.
    The subprojects were created with two many objectives and inflated expectations.
    The subproject format and procedures are too complicated and conceptually
      fuzzy.
    The inclusion of capacity building activities was premature in the programme
      context.
    INGO use of their own proposal formats has limited the comparability of their
      subproject models and their results.
    INGOs play a key role in the participatory design and implementation of
      community level activities. The INGO’s successful participatory strategies
      enabled a higher degree of impact for their small projects on local communities.
      UNDP effectively outsourced to these INGOs the application of participatory
      processes with intended beneficiaries.

Recommendations.
    Simplify and standardize the subproject format and related procedures for
      subproject design; require the same procedures and formats for all subprojects.
    In the next programme, organize related subprojects into programme components
      in order to facilitate management and create synergies.
    Determine whether the INGO model for community participation remains
      relevant and appropriate for follow-on programme activities.

4. Implementation Arrangements

Conclusions. The nature and effectiveness of implementation arrangements varied a
great deal. Some subprojects had appropriate implementation arrangements with
Government departments. However, attention was focused on the selection of one key
implementing partner in most cases.           Arrangements that involved government
departments in fiduciary roles were typically the least effective.

Recommendations.

       In designing any new programme, emphasis should shift from the selection of
        implementing partners to the design of appropriate implementation arrangements
        that encompass the entire, relevant institutional set. Memoranda of Agreement
        (MOAs) need to be negotiated with all participants regardless of whether or not
        they receive UNDP funding.


                                           5
      Subprojects should be seen as often having several significant participants or
       partners. Their roles and responsibilities in subproject implementation should be
       matched with each participant’s mandates and competencies on the one hand and
       the requirements of the subproject on the other. In other words, these roles and
       responsibilities should be determined pragmatically, not in accordance with a
       rigid or arbitrary model.

      Government agencies should increasingly be in leadership roles particularly in
       relation to policy-making or interpretation, advocacy and coordination of
       stakeholder decision-making processes.

      In cases where a government agency is a recipient of direct support or capacity
       building assistance, its roles in leadership and decision-making remains
       important. However, these roles do not automatically qualify the government
       agency to carry out the fiduciary role of managing UNDP’s funds. Indeed, given
       the high propensity for rent-seeking within the government, this role for
       government agencies should be avoided.

      In accordance with UNDP’s global emphasis on results-based management,
       UNDP should identify and select the agency best qualified to manage and deliver
       UNDP-funded assistance and to undertake the fiduciary role for UNDP resources.
       These may include firms, individuals, NGOs, UN agencies and in some cases,
       government agencies. If no competent agency is available, UNDP take on these
       responsibilities itself.

      In regard to INGO’s, the CPRU should take into account UNDP headquarters
       planned issuance of new guidelines on working with NGOs and take advantage of
       the opportunities for flexibility in the creation of implementation arrangements for
       future activities.

      However, in circumstances where requests for proposals are appropriate and
       several INGO candidates are interested, competitive bidding should continue.
       Implementing agencies that have been selected by competitive bidding should be
       treated as contractors, not partners.

Note that the kind of flexible structuring of implementation arrangements suggested
above is facilitated under direct execution. Under national execution, there is less
flexibility.

5. Subproject Review and Approval

Conclusions. Subproject approval and activation involves up to three approval processes:
subproject, memorandum of agreement (MOA), and contract. There is a need to
coordinate and simplify subproject “legal” documentation and related processes.



                                            6
The assessment team found some confusion in regard to the required documentation, the
order it which documentation is prepared and approved, and authority to sign the various
documents.

INGOs subprojects were treated differently. As all arrangements with INGOs were
contracts, they had to be submitted to the UNDP contracts committee (CAP/C) for
consideration and endorsement. To complicate matters further, the INGO contracts
required waivers of competitive bidding. Such contracts had to be submitted to the
Assets, Contracts and Procurement Committee (ACP) at UNDP headquarters in New
York for approval. These cumbersome arrangements were often slow.

Recommendations.
    Subproject documents and MOAs should be prepared in a coordinated and
     integrated manner for processing and approval at the same time. The resident
     representative should delegate authority to sign MOAs to the head of the CPRU.
    If possible, terms of reference or specifications documents should be prepared in
     parallel with subproject documents so that upon approval of the subproject, these
     documents can be submitted to commence the contracting process.
    Budgets proposed by potential implementing partners should be carefully
     reviewed in relation to local costs and prices.

6. Monitoring and Reporting

Conclusions.

Programme Level Monitoring. As noted above, the programme’s objectives were never
fully operationalized by the specification of indicators of achievement for each objective.
As a result, programme monitoring cannot provide data on progress towards the
achievement of programme objectives. Progress reporting on the programme is reduced
to description of results produced at the subproject level. While this information is
interesting, it does not and cannot indicate whether the programme is achieving its
objectives.

Subproject Level Monitoring. At the sub-project level, UNDP field personnel
responsible for monitoring are performing effectively on the front lines. They know the
status of their subproject activities and usually report accordingly. Nevertheless, the
NMMRP arrangements for subproject monitoring are complex, labor-intensive and
conceptually fuzzy. When monitoring procedures are followed to the letter, more
information is collected than can be analyzed.

Less formal means of reporting and sharing monitoring data appear more effective than
more formal written reports. The procedure established by the Ambon Project Office of
regular information-sharing meetings with all implementing partners seems to be an
effective method for information dissemination, brain-storming, and problem solving.




                                            7
Progress Reporting. The progress reporting by implementing partners is linked to the
subproject payment schedule. This connection provides a strong incentive to report in a
timely manner. Progress reporting for infrastructure projects is clearly linked to
construction benchmarks. Progress on non-infrastructure projects is often not linked to
completion of a mutually–agreed set of activities however. In these cases, the progress
report is of little value in ensuring that payments are related to performance.

Progress reports of all subprojects are supposed to be assessed at the concerned Project
Office. However, INGO progress reports are submitted directly to UNDP Jakarta
through the INGO’s Jakarta offices. The assessment team found some confusion about
whether the responsibilities for reviewing INGO progress reports rested with UNDP
Jakarta or with the concerned project office. In Ambon, this ambiguity may have been
the result of the unfilled vacancy in the Programme Manager’s position for several
months.

Recommendations.
      Establish a programme level monitoring system for the next programme based
        on programme success indicators and interim targets.
      Revise subproject monitoring procedures and report formats to greatly
        simplify data collection and analysis. The checklist approach used by CARDI
        should be considered for emulation.
      Establish activity-based payments schedules for non-infrastructure
        subprojects.
      Institutionalize informal information sharing and problem solving meetings
        among all implementing partners at all project offices. Ensure organizational
        learning by documentation of the issues discussed at these meetings and
        reporting to programme management.
      Clarify responsibilities and simplify steps for monitoring and follow up
        actions.     Clarify responsibilities and simplify steps for progress report
        assessment, particularly for INGOs, focusing responsibilities at the level of
        the concerned project office.

7. Subproject Implementation

Conclusions

Contracting Processes. Contracting processes carried out by implementing partners were
the most problematic in the assessment sample. Rent-seeking, manipulation of awards,
and collusion among bidders were all reported. Most of these issues were successfully
managed, however.

Contracting processes carried out by UNDP ran into difficulties, particularly early in the
programme in regard to sole sourcing to INGOs.       The INGO contracts necessitated
waivers of competitive bidding. Waivers must be approved by UNDP headquarters,
which questioned the need for sole sourcing in these cases. Back and forth
communications created long delays in processing these contracts, though they were


                                            8
ultimately approved. Aside from the slowness, the assessment team did not find any
evidence of major irregularities in UNDP procurement and contracting.

Reconciliation Approaches. Should subprojects mainstream reconciliation and IDP re-
integration? There is a need to balance reconciliation needs with successful subproject
delivery. The establishment of the fishermen’s cooperative gave priority to IDPs. As a
result, 40% of the cooperative membership consists of men who were never fishermen.
Whether or not the coop will be viable remains to be seen.

The contractor re-building the Tobelo Harbor Workers Association office could have
used IDPs at sub-market wages as construction laborers, but chose not to do so.

Muhammadiyah showed good faith by selecting a Christian contractor to rebuild the
former Muslim school, now planned to become a mixed religion school to encourage
reconciliation. The Christian contractor did not rely on only Christian labor, but
employed a large number of Muslim laborers.

Recommendations.

      Subprojects should not, as a general rule, allow implementing partners to carry
       out procurement and contracting, except for low value, locally available supplies
       and services.
      The trade-offs between subproject efficiency and reconciliation need to be taken
       into account in subproject design.




                                           9
                 The Royal Netherlands Embassy of Indonesia’s
                              Assessment of the
              UNDP North Maluku and Maluku Recovery Programme
                                (INS/01/A29)


1. Introduction

Background of the Programme. UNDP has been supporting activities in response to the
conflict in the provinces of Maluku and North Maluku since 1999. The Netherlands
Government has provided most of the financial support for these programmes. These
activities have been packaged as a number of projects and support services:

   In the initial stages of the conflict, the Ambon Emergency Operations Project
    provided assistance from 1999 to 2002. This project delivered emergency food aid,
    shelter, non-food items and support for micro-credit and revolving funds to over
    119,000 conflict-affected persons. Funding for this programme totaled USD 952,548
    from the Governments of the Netherlands and Norway.
   Beginning in 2001, additional humanitarian support was provided through the UNDP-
    supported Community Recovery Programme (CRP) for internally displaced persons
    in and from the Maluku Provinces, with funds of USD 1.23 million from the
    Netherlands Government. This programme was in addition to CRP’s regular
    programme of civil society grants addressing issues of food security, basic social
    services, job creation and income generation, which provided grants to 110 non-
    governmental organizations (NGOs) and community-based organizations (CBOs) in
    Maluku and North Maluku provinces.
   Two other longer-term developmental and recovery projects were supported in
    Maluku: the Community-Based Initiatives for Housing and Local Development
    (CoBILD) project (funded by the Netherlands Government) and the Kei Islands Peace
    Building Programme (executed by CRP, with USD 2.2 million from the Netherlands
    Government). The Kei Islands Programme, launched in June 2001, provided grants
    to village and inter-village committees to support projects in such areas as education,
    water supply, sanitation, village communications, electricity, drainage, transportation
    infrastructure, market construction and income generation.
   UNDP also supported the establishment of UN Resource Centres in both North
    Maluku and Maluku to provide common administrative, logistical and security
    services for UN agencies operating in these areas. The Netherlands Government has
    provided USD 888,128 for safety support services to UN agencies, and by extension
    to the community of international aid agencies, through the UN Resource Centres in
    Maluku and North Maluku.

The North Maluku and Maluku Recovery Programme (INS/01/A29) was launched in
September 2001 to provide multi-sectoral support in concert with the efforts of the
Government and civil society for post-conflict recovery, longer-term peace and
sustainable development.




                                            10
The four immediate objectives of the programme address: 1) the return of internally
displaced people (IDPs); 2) the rehabilitation of community-level physical infrastructure
and provision of effective public services; 3) the initiation and growth of social and
economic activity; and, 4) the strengthening of good governance practices and local
governance capacities.

The strategy of the programme has focused on area-based programming in priority
geographic areas complemented with programming at the provincial level. The
programme commenced operations in the sub-districts of Galela, Tobelo and Tobelo
Selatan in North Maluku and has moved forward with new area-based programmes on
Bacan Island in North Maluku Province, and in Ambon, Maluku Tenggara and Maluku
Tengah in Maluku Province. The programme activities in Maluku Tenggara is building
on the results of the Kei Islands Peace Building Programme, which ended in June 2003.

To date, the programme has approved or planned 69 subprojects totaling more than USD
7.5 million in value. These subprojects – some ongoing, some already completed –
support:
        the rehabilitation of community-level infrastructure in the areas of housing,
           electricity, water supply, drainage, transportation;
        public service provision including the rehabilitation of facilities (schools,
           health facilities) and training for service providers;
        the promotion of economic activity including construction of markets, support
           to livelihoods and employment, labour intensive public works to inject money
           into local economies;
        the enhancement of social interaction such as cultural celebrations and sports
           competitions;
        improvements in local governance.

UNDP manages the programme directly with the Ministry of Human Settlement and
Regional Infrastructure (Kimpraswil) serving as Government counterpart. The
programme’s subprojects are implemented through partnership arrangements with
Government at the provincial and district levels, UN agencies, international NGOs,
Indonesian NGOs and other civil society organizations.

The programme has received a strong response from donors. The Governments of
Australia, Italy, Japan, the Netherlands, New Zealand, Norway, Sweden and the United
Kingdom have committed nearly USD 12.0 million to the Trust Fund established for the
programme. The Netherlands Government is the largest donor with a commitment of
USD 8.0 million.

Background of the Assessment.

This assessment mission is the third in a series of reviews that will provide input to the
next round of programming for the NMMRP. The two reviews were a mid-term internal
assessment of the North Maluku and Maluku Recovery Programme carried out by a
consultant team at the request of UNDP and a “Strategic Review of UNDP Indonesia’s



                                           11
Conflict Prevention and Recovery Programme” conducted by DFID. Both reviews took
place in the fourth quarter of 2003.

Mid-Term Assessment.        The NMMRP mid-term assessment exercise had three
objectives:

          To assess strategic directions towards achievement of programme objectives;
          To analyze the current situation to determine whether the current objectives
           remain relevant; and
          To make recommendations and present options for strategic directions.

The assessment identified activities related to the four NMMRP objectives and reached
the following general conclusions:

          That the programme had helped heal economic and structural impacts of the
           conflict, but that now more focus was needed on systemic conflict risks;
          That some programme activities lacked a peace-building focus;
          That programme emphasis was already shifting from IDPs and infrastructure
           to governance and economic development, a move which is critical to the
           future structure of the programme.

The main recommendations of the assessment focused on future programme activities
Others touched on knowledge management and learning, partnerships and management
systems. Of interest to the current assessment were the following recommendations:

          That the programme shift to national execution;
          That peace and conflict indicators be established on the basis of conflict
           analysis;
          That more technical assistance be provided for subproject design and
           implementation;
          That programme management changes should enhance communications and
           efficiency and clarify roles and responsibilities;

DFID Review. During 2002 and 2003, DFID had provided over 1.0 million pounds to
UNDP Indonesia to strengthen the management capacity of the country office’s Crisis
Prevention and Recovery Unit (CPRU). DFID provided an additional 3.0 million pounds
for programming through the CPRU, a portion of which was devoted to the NMMRP.
The DFID review was concerned with the success of its capacity building activities at the
CPRU and the relevance of the overall approach of the unit to conflict in Indonesia.
Recommendations were to provide advice to both the CPRU and DFID on future
directions.

DFID’s main conclusions and recommendations target the CPRU’s overall portfolio of
conflict-related interventions, but as the NMMRP is by far the largest of these, they are
clearly relevant. The main conclusions of the review were:




                                           12
          that, given the “constant evolution” of the context, the CPRU needs a much
           higher level of strategic analysis to focus its programme activities on the long-
           term process of prevention action;
          that internal capacities and staff skills in the CPRU are limited and need to be
           strengthened;
          that the CPRU lacks policies on critical issues;
          that the CPRU lacks basic systems for organizational learning such as
           monitoring and evaluation.

The review made recommendations for future programme priorities that emphasized
strategic analysis, organizational learning, gender mainstreaming, and national staff
development.

Netherlands Assessment Mission.

Purpose. The current assessment completes the triangulation of overlapping perspectives
on the NMMRP. To the DFID perspective on UNDP’s overall structure and management
of conflict-related programmes and the mid-term assessment perspective on programme
substance, this assessment adds a perspective predominantly on processes. The mission’s
terms of reference emphasize “the need for ongoing review of operational and
programmatic experience in field-based programming towards… continuing to strengthen
operating procedures and mechanisms at both the Jakarta and field office levels.”

The three main objectives of this assessment are:

   1. To review and assess the quality of the monitoring process in theory and in
      practice;
   2. To review the (sub) project identification, approval and contracting processes;
   3. To review the relationship between the activities undertaken and the original
      subproject proposal, and the extent to which subproject objectives have been
      achieved both qualitatively and quantitatively.

In the course of the assessment, the mission is to, inter alia:
       Review and assess the programme’s monitoring cycle (theoretical);
       Review and assess the quality of the programme’s implementation of monitoring
        and follow-up procedures (in practice);
       Review and assess the programme’s project identification, approval, and
        contracting processes; and
       Review and comment on division of monitoring and evaluation responsibilities
        between Project Field Offices and Jakarta UNDP/CPRU.

Assessment Methodology. In order to fulfill its terms of reference, the assessment team
selected a sample of 11 subprojects from the total of 69 subprojects in the NMMRP for
intensive review. The total budget of the 11 sample subprojects is nearly $1.7 million or
nearly 23% of the current NNMRP subproject budget of $7.5 million. The sample was
selected deliberately to include subprojects that were representative of the following



                                            13
criteria: different sizes of subproject by budget amount; various thematic areas;
geographical distribution among the three areas covered by project field offices; and
various types of implementing partners. The assessment team also reviewed relevant
documents and interviewed UNDP staff at headquarters and project staff in the
programme area. The sample subprojects are briefly described on Table 1 below.

The proposed activities for the assessment mission place heavy emphasis on monitoring.
The basic purpose of monitoring is to collect and analyze information about project
progress and problems for decision-making, in other words, to improve project
performance. Monitoring also plays an important role in accountability by ascertaining
whether or not resources are being utilized as intended. The view of the assessment team
is that while critically important, monitoring is only one element in the project cycle that
contributes to project performance and accountability. The team has therefore taken a
broader view and considered factors at all stages of the project cycle that affect
performance and accountability. In the case of the NMMRP, the team feels that two
other variables merit special attention: 1) the design of the programme and its
subprojects and 2) programme and operations support from UNDP Jakarta.

Organization of the Report. The chapters in the remainder of the report follow the order
of the main elements in the project cycle: programme and subproject design with a
special emphasis on implementation arrangements; monitoring; subproject
implementation; and subproject results. Within each section, observations are presented
regarding subprojects in each of the three areas visited by the team – Tobelo, Ternate,
and Ambon.        Each chapter also includes conclusions and recommendations.




                                            14
                                                           Table 1
                                           Subprojects in the Assessment Sample
                                                     (North Maluku)

 No.       Short Title                              Description                           Implementing Partners       Budget $
016    Rehabilitation of    Rehabilitation of 75 km of electric power lines in Tobelo and PLN (National Electricity    231,205
       Electric Power       Galela sub-districts                                          Company), CV Gaya
       Lines                                                                              Teknik
034    Community            Establishment of community-based organizations to supervise CARDI                          499,917
       Rehabilitation       implementation of community rehabilitation activities
046    Tobelo Harbor        Re-construction of Tobelo Harbor Workers” Association Tobelo Harbor Workers’                29,690
       Workers’             building                                                      Association, UNDP
       Association
048    Legal Literacy       Education about legal rights for IDPs                      N. Halmahera District            51,251
                                                                                       Court, PBHI
064     Livestock          Training, supplies and equipment for the District Livestock N. Halmahera District             6,597
                           Dept.                                                       Livestock Dept.
068     Pajeko Fishermen’s Organization of a fishermen’s cooperative, procurement and N. Halmahera District             30,137
        Cooperative        installment sale of fishing boat and equipment, training on Fisheries Dept.
                           coop management, fish marketing, and fishing technology
069     District Planning  Strategic planning workshop for N. Halmahera District       N. Halmahera District            11,776
                                                                                       office, UNDP
Total for North Maluku Sample Subprojects                                                                              860,573




                                                              15
                                                     Table 1 (continued)
                                            Subprojects in the Assessment Sample
                                                          (Maluku)

 No.       Short Title                                  Description                             Implementing Partners   Budget $
001    Supporting            Humanitarian assistance to meet the immediate needs of IDPs Mercy Corps                     203,538
       Stability, Return     and activities to support creation or reactivation of livelihoods International
       and Economic
       Empowerment in
       Maluku (Phase 1)
003    Promoting             Facilitation of peace-building and documentation of peace- ICMC                             204,310
       Community             building processes
       Recovery and
       Sustainable Returns
       (Phase 1)
009    Reconciliation      Re-construction of a high school, teacher training, and Muhammadiyah                          201,373
       through Education   integration of Muslim and Christian students and teachers       Foundation
010    Reconciliation      Re-habilitation of seven schools; development of a Ambon Municipality                         224,211
       through Education   reconciliation   process     and     educational     materials;
                           improvements in the quality of education.
Total for Maluku Sample Subprojects                                                                                       833,432
Total for Assessment Sample Subprojects                                                                                 1,694,005




                                                              16
2.      Programme and Subproject Design

A.      Programme Objectives

The immediate objectives of the programme are as follows:

     1. The return of internally displaced people (IDPs) into key communities in an
        atmosphere of reconciliation;
     2. The rehabilitation of community-level physical infrastructure and provision of
        effective public services in key areas;
     3. The initiation and growth of social and economic activity with a particular focus
        on the most vulnerable; and
     4. The strengthening of good governance practices and local governance capacities.

These objectives are firmly grounded in the global experience of UNDP and other
international agencies in post conflict situations. However, these objectives were never
fully operationalized into statements of results to be achieved. No indicators, targets or
benchmarks were established. In practice, the programme objectives appear to have been
treated as guiding principles or areas of emphasis for programme activities, not goals to
be achieved by programme operations.

Conclusions. In the absence of indicators and targets, there is no practical guidance for
the formulation of the subprojects as to how they should be designed so that they can
contribute to the achievement of the objectives of the programme. Furthermore,
monitoring or assessing progress towards the achievement of programme objectives is
not possible. In other words, measurement of achievement of programme objectives can
only be subjective.

Recommendations. The establishment of indicators and targets for achievement at the
programme level for any follow-on programme essential for detailed programme design,
monitoring, and results-based management of implementation.

B.      Subproject Identification

From its inception, the programme has been committed to participatory processes in the
identification and selection of its activities. This policy was set out as follows in the
programme document:

“..the programme will take a highly participative approach with actual decisions on
the activities to be undertaken being made by communities. UNDP in cooperation
with the government and other stakeholders will facilitate this process maintaining
neutrality while emphasizing the importance of equity and inclusive decision-
making.”

The position was further elaborated in the first quarterly report of the programme:




                                            17
“Through facilitating participatory decision-making processes, the programme seeks
to enhance communication and coordination among key stakeholders in planning and
implementing area-based recovery programmes. In supporting the development of
such decision-making processes, emphasis will be placed upon ensuring that former
conflicting communities are represented in a balanced manner, and also that
minority groups in each community are appropriately represented. The programme
will place emphasis upon the participation of women in these decision-making
processes.1”

Tobelo. While an initial set of activities were carried out to meet emergency needs, a
major effort was made to establish a participatory decision-making process for the
Tobelo-Galela area, the first subdistricts in which the programme started to work. This
process was built around the establishment of a Community Programming Board (CPB)
which would review local needs and set priorities for assistance. The principles
underlying this approach were adapted from successful experience in UNDP activities in
Aceh and Southeast Maluku.

Existing community groups and organizations were asked to nominate members for the
CPB. Eventually 16 of these leaders were chosen for the Board. The members included
community and Government leaders, representatives of both major religions, youth,
women, and vulnerable groups.

Over a nine-month period (April-December 2002), the CPB reviewed needs, solicited and
assessed proposals for assistance and ultimately recommended a list of 44 proposals to
UNDP for implementation. The criteria that were supposed to have been applied in the
selection of proposals were: greatest immediate needs; vulnerability of the target
beneficiaries; and support for reconciliation. However, a series of planned needs
assessments that would have enabled a cross check on the validity of proposals
recommended by the CPB were, for unexplained reasons, never carried out.

The CPB experience in Tobelo had mixed results. CPB membership appeared to be
representative. However, candidates put forward by target groups for CPB membership
did not undergo any kind of validation process. The board eventually succeeding in
reaching a consensus on a set of priorities, but the members were reported to have been
pre-occupied with the dollar value of funding available. Finally, the board president and
another officer supported a false claim for payment against UNDP resulting in the board
being disbanded in March 2003. This unfortunate situation tainted to entire results of the
CPB process.

Ternate. The UNDP project office in Ternate oversees only two subprojects at present,
but both are large. The subproject in the mission’s sample was North Maluku
Community Rehabilitation Initiative implemented by CARDI, an international NGO.
CARDI prepared and submitted its proposal to UNDP for consideration. Under the
proposal’s methodology, CARDI would identify and select its own implementing
partners, based on a clear set of selection criteria, from among local NGOs and
community-based organizations (CBOs). In the absence of representative CBOs, CARDI
1
    North Malulu and Maluku Recovery Programme, 1 st Quarterly Report, October-December 2001, page 4


                                                   18
planned to create new ones. These implementing partners would oversee the
implementation of sectoral activities in the case of local NGOs and community-based
programmes in the case of CBOs.

Ambon. The UNDP office in Ambon has supported the implementation of 15
subprojects to date, six of which were very small in financial terms (less than $5,000
each). The assessment mission sample was drawn from the remaining nine and consists
of two implemented by INGOs, one by a national NGO and one by Government. The
office also has a large pipeline of subprojects scheduled for implementation beginning in
April 2004.

In Ambon, UNDP made an informal request for proposals from potential partners. All
four of the sample subprojects were identified by the implementing partners themselves.
The local officials of the national religious NGO Muhammadiyah conceived and
formulated its subproject proposal in Ambon. Both Mercy Corps and ICMC were
working on their activities before the UNDP programme started in Ambon. UNDP
funding extended and expanded these INGO activities. The mayor of Ambon conceived
the municipality’s reconciliation project and requested UNDP support for its
implementation.

Conclusions. UNDP invited participation in various ways in the identification and
selection of the subprojects that have made up the programme. The CPB process
attempted to work with the leadership of various government and civil society
constituencies. This process took far too long. CPB membership and the integrity of its
process ultimately came into question. UNDP does not appear to have brought sufficient
expertise in participatory processes to this exercise. This approach should not be
replicated in future.

The identification of potential activities by INGOs and local organizations in Ternate and
Ambon produced better results. This approach was pragmatic and in some cases was
able to start activities quickly. The INGOs had a high level of formulation skills. In
addition, the three INGOs in the sample were already working in the area before the
NMMRP began. Their proposals capitalized on their local knowledge and experience.
Each INGO also designed their proposals to provide economies of scale by consolidating
a large number of community level activities under one subproject umbrella.

The INGOs all incorporated participatory processes into their operational approaches.
These processes focused on mobilization of activities at the community level. This type
of participatory strategy may not be relevant in the future if programme emphasis shifts
away from community-based activities.

Though there are a number of questions about the results of the CPB process in Tobelo,
the approaches to identification and selection of programme activities appear to have
identified activities worthy of programme sponsorship. These activities were responsive
to local needs However, they did not lead to an integrated and comprehensive




                                           19
programme. The result was a collection of subprojects, some of which may have been
related. In Tobelo, there were too many small subprojects.

Recommendations.
    Conduct needs assessments of intended beneficiaries to establish baseline data
      for future programming.
    Identify and assess potential partners in government and civil society that can
      and should play roles in meeting identified needs.
    Establish a programme framework based on the foregoing analyses. Flesh out
      the detailed activities of the framework using appropriate consultative and
      participatory strategies. Cluster related activities for efficient management.

C.     Subproject Design.

Tobelo. The proposals of the Tobelo-Galela CPB were identified in a participatory
manner but they were still only proposals. They had to be converted into the appropriate
formats for review and approval by UNDP. This formulation process commenced in
early 2003 and has continued until the present date. At this point, formulation of these
subprojects is virtually completed.

Forty-four subprojects were eventually prepared. Many of these subprojects were
relatively small: in the range of $5,000 to $30,000. The large number of subprojects
meant that a large number of subproject documents had to be drafted and negotiated at
the local level. Each in turn had to be reviewed, approved and administered at the central
level.

The sample subprojects illustrated strengths and weaknesses in their design. A number of
these design topics concerned the definition of implementation arrangements, a subject
which will be discussed in detail below.

One design issue that appeared in the sample subprojects was an overly ambitious
approach to capacity building. Indeed, one could question whether capacity building
should have ever been considered as an appropriate area of work at this stage, given the
needs for rehabilitation and the lack of staff in many Government offices. In any event,
what was characterized as capacity building was in fact direct support for the reactivation
of government services (in the case of the Fisheries and Livestock subprojects) or the
rebuilding of facilities (Harbor Workers Association).

The scope of project activities was also exaggerated in the design. Many subprojects
attempted to address more than one programme objective, creating overly complex and
ambitious undertakings. Some small-scale activities had two or three objectives and
several outcomes that greatly exceeded the capacity of the subproject’s activities to
achieve them. An illustrative example was the inclusion of an outcome on reduction of
dynamite fishing to be achieved by a brief training course, mainly on other topics, for
members of a newly created, 50-person fishermen’s cooperative.




                                            20
Another design issue involved the creation of a revolving fund. This planned fund is to
be created by the sale of a large boat and equipment on credit to the fishermen’s coop.
The proceeds from the sale of this UNDP procured asset, valued at nearly Rps 250
million, would endow a fund to be managed by the district fisheries office in Tabelo.
The fund is supposed to be used for future boat construction and sale. Aside from the
workability of this scheme, the use of these proceeds to create a fund controlled by the
District Fisheries Department creates policy and procedure questions for UNDP that have
not yet been addressed.

A more successful subproject design is illustrated in UNDP’s response to N. Halmahera
District’s request for assistance to provide input into the District’s development planning
process. UNDP’s responsiveness on short notice was commendable and the design of the
workshop seems to have directly corresponded with the district’s needs at the time.

Ternate. CARDI’s proposal became the basis for the UNDP-supported subproject. The
subsequent contract with CARDI provided an umbrella for the INGO to identify and
implement its own programme of activities in two categories: 1) area-based activities in
specific sectors to be carried out by local NGOs; and 2) community-level activities that
were identified, designed and implemented through community-based organizations.

The assessment focused on CARDI’s work with the community-based organizations
(CBOs). CARDI’s approach to participatory community rehabilitation is essentially to
create a community based structure to oversee the process of community identification,
design and management of their own rehabilitation activities. A conscious decision was
made to create new, genuinely participatory and representative mechanisms based on the
assessment that existing governmental or traditional structures had either broken down or
become corrupted as a result of the conflict.

CARDI’s approach to the creation of CBOs is deliberate and consultative. Candidate
villages are selected on the basis of the severity of damage sustained during the conflict,
the extent of IDP-related problems and the needs for strengthening social cohesion
(particularly in villages with mixed religious affiliations). The creation of a CBO follows
these steps: CARDI field staff assess community readiness to work together to address
community needs, identify leaders representing 8-10 community groups, validate the
acceptability of leaders in community meetings, and form community leaders into a
community committee that will oversee the process of identification, selection, design
and implementation of specific rehabilitation activities. CARDI staff train community
leaders on the preparation of proposals and explain the criteria for selection of proposals
for funding. These criteria include: benefit to the community as a whole, transparency in
decision-making, community participation, community capacity to implement, and
availability of funding. This process takes place over a period of 2-3 months.

Communities prepare proposals locally and carry out their own prioritization. CARDI
does technical analysis of the proposal ranked first by the community and negotiates any
suggested changes. CARDI agrees to fund proposals that meet the selection criteria.




                                            21
Ambon. All four implementing partners in the sample designed their own subprojects.
In the case of Muhammadiyah and the Ambon municipality, their proposals were
negotiated and re-written in a UNDP format. Some changes were made in proposed
budgets and, in the case of the municipality, some changes were made in proposed
activities as well. In the cases of the two INGOs, their proposals became the project
documents.

Discussion. The complexity of subproject design was at least in part the result of a
complicated project format and procedures that includes objectives, outputs, outcomes,
and outcome indicators. This complexity contributed to the creation of inflated
expectations for what were in fact limited and relatively small interventions.

Unfortunately for the programme, it bridged a transition in UNDP logical frameworks
and terminology while at the same time inventing its own hybrid logframe. These are
illustrated on the table below:

             UNDP Logical Frameworks and Terminology for Projects

            Old                       Current                NMMRP Subprojects
Development Objective        Outcomes with indicators      Objectives
Immediate Objectives with    Outputs with targets          Outputs
Success Criteria
Outputs                      Activities                    Activities
Activities                   Inputs                        Outcomes with indicators
Inputs                                                     Inputs


In an additional complication, INGOs used their own formats for their project proposals
and these were not converted in the programme’s subproject document format.

Conclusions.
    The programme’s subproject design procedures allowed the creation of too many
      small, unrelated subprojects increasing the management burden and diminishing
      impact and any potential synergies.
    The subprojects were created with two many objectives and inflated expectations.
    The subproject format and procedures are too complicated and conceptually
      fuzzy.
    The inclusion of capacity building as subproject activities was premature in the
      programme context.
    INGO use of their own proposal formats has limited the comparability of their
      subproject models and their results.
    INGOs play a key role in the participatory design and implementation of
      community level activities. UNDP effectively outsourced to these INGOs the
      application of participatory processes with intended beneficiaries. However, this
      model may not be appropriate for future programme activities.



                                          22
Recommendations.
    Simplify and standardize the subproject format and related procedures for
      subproject design; require the same procedures and formats for all subprojects.
    In the next programme, organize related subprojects into programme components
      in order to facilitate management and create synergies.
    Determine whether to INGO model for community participation remains relevant
      and appropriate for follow-on programme activities.

D.     Implementation Arrangements

A key element in subproject design is the determination of implementation arrangements
for the activities.

Tobelo. The subprojects in the sample in Tobelo included a variety of implementing
partners and arrangements. The categories of these arrangements can be described as
follows: 1) government agency as implementing partner; government agency as
implementing partner with private firm as subcontractor; government agency as
implementing partner with NGO as subcontractor; NGO as implementing partner.

The sample projects reveal that partnership can have many forms and that some forms are
more likely to be efficacious than others. The cases in which government agencies were
fully “empowered” to act implementing partners were the most problematic. The
creation of such arrangements indeed appears contrary to the direct execution modality of
the overall programme. In UNDP jargon, this approach has “NEXed DEX”. In other
words, national execution (government management) has replaced the programme’s
direct execution modality (UNDP management). In UNDP’s global approach to national
execution, there is a pronounced tendency to automatically include management of
UNDP resources as an element of government responsibility. In the circumstances of the
post conflict environment and given the propensity for rent-seeking that is pervasive in
the government, the choice of such arrangements for two subprojects in the sample
(Livestock and Fisheries) was inappropriate.

A more appropriate model was set up in the cases where government agencies have been
given key roles in leadership, decision-making, and/or supervision, but an NGO or
private firm has been contracted by UNDP to carry out project activities. In these cases
(Legal Literacy, Rehabilitation of Electric Power Lines), the management of project
resources was entrusted to the NGO or private contractor who was directly contracted to
UNDP. While this model cannot eliminate rent-seeking, it does lower the risks.

The general problem that arises is the need for more specific idea of roles and
relationships in partnerships that are created in subprojects. Are NGOs and private firms
“partners” or contractors? Does the status of partner automatically confer management of
project resources as one of the responsibilities? How can the role of government be
defined or clarified in subprojects?




                                           23
Ternate. In the case of CARDI, UNDP has essentially subcontracted the management of
a large number of small projects to an INGO. CARDI is fully responsible for the design
and implementation of its entire portfolio. CARDI was managing community-based
activities in 27 communities at the time of the assessment. CARDI’s management model
for community-based activities emphasizes a standardized approach to the activation of
community participation and transparency in the management of funds. Any transfers of
funds to communities for activities are supposed to be made publicly at community
meetings.

Ambon. Various implementation arrangements are illustrated by the four sample
subprojects in Ambon. In the case of Muhammadiyah, the model is full NGO
implementation. The bulk of the funding for the subproject was for reconstruction of a
school. Under the arrangements that were established, Muhammadiyah was given
responsibility to carry out the contracting process and supervise the construction from
start to finish. Similarly, the reconciliation subproject of the municipality was fully
implemented by a government entity. The subprojects managed by ICMC and MCI
followed the model of INGO implementation. UNDP did not directly manage any part of
these subprojects.

Discussion: Re-thinking “Implementation Arrangements”. UNDP tends to regard all
participants in project implementation as “partners”. However, participants in project
implementation may plays a number of important roles not all of which is synonymous
with the idea of partnership. To further complicate matters, an “implementing partner” is
selected for each subproject. In UNDP parlance, an “implementing partner” is usually
the entity that UNDP will fund to implement subproject activities. The decision about
implementation arrangements has thus been reduced to identifying the implementing
partner. The choice has been over-simplified and can often yield poor results,
particularly when government agencies are involved. Consider two examples:

1. Legal Literacy. The District Court in N. Halmahera District identified the need to
   educate IDPs in the district about their legal rights. An active judge conceived a
   proposal, acted as advocate for the proposal to the court and UNDP, provided
   leadership and decision-making in the design of the subproject and the selection of an
   NGO to carry out the subproject’s educational activities, and provided overall policy
   guidance. The NGO was selected as subproject contractor in accordance with UNDP
   contracting procedures and holds a contract directly with UNDP. It will carry out
   surveys, educational activities and media campaign.

   In this example, the government agency’s role and comparative advantage are
   maximized: its role is in leadership, policy, advocacy and decision-making on
   activities. UNDP has established a substantive partnership with government and a
   contractual relationship with an NGO. The NGO carries out activities in the area of
   its expertise as a contractor.

2. Fishermen’s Cooperative. The District Fisheries Department was appointed
   implementing partner of a subproject to organize a fishermen’s cooperative, procure



                                           24
   and equip a boat for the coop, arrange the credit sale of the boat and equipment to the
   coop, and establish a revolving fund with the loan repayments for future boat
   procurement and install sale to fishermen’s groups. The department is managing
   these activities and the related UNDP resources.

   In this example, the government agency has been given all “implementation”
   responsibilities. The department is deeply into the temptations of managing other
   people’s money and working in areas that greatly exceed its technical skill set: group
   mobilization; procurement and contracting (an attempt to extort funds from the boat
   builder has already been reported; who holds title to the boat is unclear.); creating a
   revolving fund (banking arrangements are already confused and the question of
   whether UNDP can allow an asset procured under a subproject to be sold on credit
   has never been addressed.). UNDP is trying to control its partner through monitoring
   and the new fishermen’s coop is on the sidelines waiting to be outfitted for fishing.
   While there are obvious question about the design of this subproject, it is nevertheless
   highly unlikely to succeed under these implementation arrangements.

Recommendations.

      In designing any new programme, emphasis should shift from the selection of
       implementing partners to the design of appropriate implementation arrangements
       that encompass the entire, relevant institutional set. Memoranda of Agreement
       (MOAs) need to be negotiated with all participants regardless of whether or not
       they receive UNDP funding.

      Subprojects should be seen as often having several significant participants or
       partners. Their roles and responsibilities in subproject implementation should be
       matched with each participant’s mandates and competencies on the one hand and
       the requirements of the subproject on the other. In other words, these roles and
       responsibilities should be determined pragmatically, not in accordance with a
       rigid or arbitrary model.

      Government agencies should increasingly be in leadership roles particularly in
       relation to policy-making or interpretation, advocacy and coordination of
       stakeholder decision-making processes.

      In cases where government agencies are recipients of direct support or capacity
       building assistance, their role in leadership and decision-making remains
       important. However, these roles do not automatically qualify the government
       agency to carry out the fiduciary role of managing UNDP’s funds. Indeed, given
       the high propensity for rent-seeking within the government, this role for
       government agencies should be avoided.

      In accordance with UNDP’s global emphasis on results-based management,
       UNDP should identify and select the agency best qualified to manage and deliver
       UNDP-funded assistance and to undertake the fiduciary role for UNDP resources.


                                           25
         These may include firms, individuals, NGOs, UN agencies and in some cases,
         government agencies. If none is available, UNDP take on these responsibilities
         itself. Implementing agencies that have been selected by competitive bidding
         should be treated as contractors, not partners.

        In regard to INGO’s, UNDP headquarters is currently working on a new
         procedure that would allow the selection of an INGO to implement UNDP-funded
         activities on a negotiated basis. However, in circumstances where requests for
         proposals are appropriate and several INGO candidates are interested, competitive
         bidding should continue.

Note that the kind of flexible structuring of implementation arrangements suggested
above is possible under direct execution. Under national execution, there is less
flexibility.

E.       Subproject Review and Approval.

Subproject approval and activation involves up to three approval processes: subproject,
memorandum of agreement (MOA), and contract.

Subproject approval authority is vested with the head of the CPRU. Subprojects, initially
from Tobelo-Galela, were prepared in the standard format and sent to Jakarta for review.
If there were comments, the document was modified accordingly, signed by the
implementing partner and sent to Jakarta for approval. Later, subprojects from Tobelo
were reviewed and endorsed by the Ternate project office before being sent to Jakarta for
approval. In Jakarta, the Programme Manager reviewed the documents before
forwarding them to the CPRU head for approval.

In a few cases in which subprojects were based on proposals from implementing partners,
there were no apparent efforts to compare budget requests to prevailing local prices. The
budget estimates provided by the Ambon municipality, for example, appeared to be
excessive to the team.

The approval of a subproject does not activate it, however. An MOA and/or a contract
will be necessary depending on the implementation arrangements that have been
established. The resident representative approves an MOA. The resident representative
or the DRR operations signs contracts. Over a certain threshold, the Contracts, Assets
and Procurement Committee (CAP/C) must be recommended the contract for approval.
The assessment team found some confusion in regard to the required documentation, the
order it which documentation is prepared and approved, and authority to sign the various
documents.

INGO proposals in the sample were submitted directly to the CPRU in Jakarta. The
CPRU then carried out conceptual and technical reviews, negotiated revisions and
approved the subproject. However, as all arrangements with INGOs were contracts, they
had to be submitted to the CAP/C for consideration and endorsement. To complicate



                                            26
matters further, the INGO contracts required waivers of competitive bidding. Such
contracts had to be submitted to the Assets, Contracts and Procurement Committee
(ACP) at UNDP headquarters in New York for approval.

These cumbersome arrangements were often slow, particularly for the INGO contracts
that required waivers. After a nine-month process of project selection, the Tobelo office
was particularly sensitive to the delays in approvals. Despite the fact that the Tobelo
subprojects represented only around 10% of the total value of the NMMRP, that office
had the largest number of subprojects (44 of the total 69 subprojects in the programme) to
be processed.

Conclusion. There is a need to coordinate and simplify subproject “legal” documentation
and related processes.

Recommendations.
    Subproject documents and MOAs should be prepared in a coordinated and
     integrated manner for processing and approval at the same time. The resident
     representative should delegate authority to sign MOAs to the head of the CPRU.
    If possible, terms of reference or specifications documents should be prepared in
     parallel with subproject documents so that upon approval of the subproject, these
     documents can be submitted to commence the contracting process.
    Budgets proposed by potential implementing partners should be carefully
     reviewed in relation to local costs and prices.
    The CPRU should take into account UNDP headquarters issuance of new
     guidelines on working with NGOs – when they are finalized - and take advantage
     of the opportunities for flexibility in the creation of implementation arrangements.

3. Monitoring and Reporting

Two levels of monitoring are relevant to this assessment: monitoring at the programme
level and monitoring at the subproject level.

Monitoring at the Programme Level. As noted above, the programme’s objectives were
never fully operationalized by the specification of indicators of achievement for each
objective. As a result, programme monitoring cannot provide data on progress towards
the achievement of objectives. Progress reporting on the programme is reduced to
description of results produced at the subproject level. While this information is
interesting, it does not and cannot indicate whether the programme is achieving its
objectives.

The NMMRP “Note on Monitoring Processes” lists no less than nine tools and
mechanisms for programme monitoring. Five of these are above the level of the
programme, at the level of the CPRU and the Country Office. Some of these reports are
prepared to meet corporate requirements, such as, audits and the Results-Oriented Annual
Report. Other tools on the list are for in-depth assessment exercises, not monitoring.
Only the quarterly Progress Reports have as their primary purpose regular reporting on


                                           27
the programme’s progress, but due to the reasons cited above, they provide only
descriptive information.

Monitoring at the Subproject Level. The NMMRP Operational Guidelines set out
formats for implementing partners for progress and financial reports; a procedure for
UNDP assessment of these reports; formats for reports for monitoring site visits and
technical monitoring of infrastructure projects, and for weekly and monthly summaries of
monitoring findings. The design of this system is typical of the kind found in
development cooperation. On paper, the system is excessively complicated, prescribes
the collection of too much information for meaningful analysis, and requires collection of
information that is either already documented elsewhere or subjective. The forms
themselves are not user friendly. These forms and procedures are not uniformly applied
across the programme. They are most fully attempted in Tobelo, partially implemented
in Tenate, and used with modifications in Ambon.

Tobelo. The Project Office in Tobelo has 9 personnel supporting and monitoring the 44
subprojects being implemented in the area. In addition, two national consultancies were
hired specifically to monitor the electrification subproject. Monitoring is carried out
effectively, if not excessively, at the subproject level. At least in regard to the
assessment’s sample subprojects, these personnel have made frequent visits to subproject
sites and have prepared regular reports. They are intimately familiar with subproject
activities. In short, the assessment mission had the distinct impression that Tobelo
Project Office staff know what’s going on in the subprojects that they monitor.
Nevertheless, staff complained of that paperwork was excessive.

Monitoring activities in the sample projects were able to uncover an attempt to extort a
kickback from the boat-building contractor on the Fishermen’s coop subproject. The
Project Manager has raised the problem with the Head of the District and the Head of the
District Fisheries Department, but the matter remains unresolved. Technical monitoring
also improved the specifications of the Tobelo Harbor Workers’ Association building by
recommending a flexible wooden floor instead of a cement floor for the structure which
is in earthquake zone.

Ternate. The UNDP project office in Ternate has only one staff member monitoring the
CARDI subproject. (Other activities of that office are managed by UNOPS.) This
monitor prepares site visit and weekly reports but no monthly reports. This staff member
regularly accompanies CARDI staff on field trips to community activities. His
monitoring findings are reported and discussed orally in regular meetings between UNDP
and CARDI staff. His monitoring uncovered community allegations of non-transparent
decision-making by the implementation teams of responsible for implementing
community activities. These findings are being addressed by CARDI.

CARDI staff implement community mobilization activities and also monitor the
implementation of the projects – mainly small infrastructure projects - selected by
communities. These staff members report to the CARDI Project Manager who




                                           28
incorporates their findings into his progress reports. CARDI progress reports are sent
directly to UNDP Jakarta with a copy to the Ternate office.

CARDI’s monitoring forms are almost entirely in checklist formats. A number of
checklists are available for different purposes, particularly for infrastructure, and for
different stages of their activity cycle, e.g., formulation, implementation, and completion.
Such forms are not only easy to use, they also establish regular behavior patterns for
personnel doing the monitoring.

Ambon. The Project Office has five staff members who monitor subprojects, but none
has technical expertise in construction. In the case of three of the subprojects in the
sample, these staff members visit subproject sites and discuss activities with
implementing partners. However, during most of 2003, monitoring of Mercy Corps
appears to have been light. This may have been due to the prolonged period when the
INGO’s funding was in limbo, pending a decision on a proposed amendment to its
contract. This period lasted from the submission of the request for amendment in March
until its approval in October. On instructions from UNDP Jakarta, the project office
launched an intensive review of a sample of 15 Mercy Corps activities in December. The
report of this review is currently being considered by Mercy Corps.

Monitoring reports on INGO activities are in a narrative format of local design. The
Ambon Project Office holds periodic meetings with all implementing partners. At these
meetings, implementing partners make presentations on the progress of their activities.
These meetings provide a venue for problem solving, brainstorming as well as
information sharing among partners and UNDP staff.

Monitoring activities in sample subprojects identified a major problem with the award of
a construction contract, which was quickly corrected by the implementing partner. In
another instance, a failure to meet construction specifications was identified. In this case,
the contractor was required to do additional compensatory construction.

In the case of ICMC, as in the case of CARDI in Ternate, the INGO has its own
implementing partners whom it monitors. ICMC implements its programme of
facilitation services for IDPs through seven local NGOs. ICMC staff monitor the work of
these organizations and report progress and problems to ICMC management. Similarly,
Mercy Corps has dedicated program and compliance staff to monitor the performance
and use of funds by its NGO partners.

Progress Reporting. The progress reporting by implementing partners is linked to the
subproject payment schedule. This connection provides a strong incentive to report in a
timely manner. Progress reporting for infrastructure projects is clearly linked to
construction benchmarks. Progress on non-infrastructure projects is not linked to
completion of activities however. In these cases, the progress report is of little value as a
pay-for-performance tool.




                                             29
Progress reports of all subprojects except those of INGOs are assessed at the concerned
Project Office. INGO progress reports are submitted directly to UNDP Jakarta through
the INGO’s Jakarta offices.         Theoretically, all progress reports, regardless of
implementing partner, are assessed by Programme Officers at UNDP Jakarta before
payments are approved and processed. The assessment team found some confusion about
whether the responsibilities for reviewing INGO progress reports rested with UNDP
Jakarta or with the concerned project office. For two of the three INGOs in the
assessment sample, their progress reports were reviewed only by UNDP Jakarta and not
by the concerned project offices; in the third case, the progress report was reviewed by
both the project office and UNDP Jakarta.

Conclusions. At the subproject level, UNDP field personnel responsible for monitoring
are performing effectively on the front lines. They know the status of their subproject
activities and usually report accordingly. Nevertheless, the subprojects are burdened by
complex and labor-intensive arrangements for monitoring. When all monitoring
procedures are followed, more information is collected than can be analyzed. The
NMMRP operational guidelines for monitoring are too complicated and conceptually
fuzzy. The checklist-based approach to monitoring applied by CARDI could be usefully
emulated by UNDP.

The assessment mission noted a few problems in the transmission and use of monitoring
findings. One Project Office Manager and one subproject monitor reported reluctance
among national monitors to report and document problems. A cultural tendency was
noted to “sit on” operational problems or to not raise them publicly, perhaps resulting
from worries that the messenger would be blamed for the problems. The Ambon
procedure of regular information-sharing meetings on subproject progress among all
implementing partners seems to be a good method for information dissemination and
problem solving.

Recommendations.
     a. Establish a programme level monitoring system for the next programme based
        on programme success indicators and interim targets.
     b. Revise subproject monitoring procedures and report formats to greatly simply
        data collection and analysis. The checklist approach used by CARDI should
        be considered for emulation.
     c. Establish activity-based payments schedules for non-infrastructure
        subprojects.
     d. Institutionalize information sharing and problem solving meetings among all
        implementing partners at all project offices. Document the issues discussed at
        these meetings and provide reports to programme management.
     e. Clarify responsibilities and simply steps for monitoring and follow up actions.
        Clarify responsibilities and simply steps for progress report assessment,
        particularly for INGOs, focusing responsibilities at the level of the concerned
        project office.




                                          30
4. Subproject Implementation

Tobelo. Four of the Tobelo sample subprojects did not appear to experience (or haven’t
yet experienced) any significant implementation problems. The four subprojects were:
the Rehabilitation of Electric Lines which was delivered on time, on budget and
according to specifications; Support to Kabupaten Planning; and Livestock. The Legal
Literacy subproject has successfully selected its subcontractor but is only now beginning
activities.

The Pajeko Fishermen’s Cooperative had the most problems, many of which should have
been foreseen: rent-seeking in the procurement process for boat construction conducted
by the district Fisheries Department; confusion regarding the ownership of the boat being
built under the subproject; how the boat and equipment can be transferred from UNDP
ownership to the coop or the department; confusion regarding financial arrangements for
handling repayments; dubious plans for a revolving fund for more boat-building to be
managed by the department; and a problematic fishermen’s coop that includes a large
number of IDPs with no fishing experience.

Though outside the assessment sample, the second Livestock subproject was brought to
the attention of the assessment team by the concerned government officials. They are
still awaiting approval to purchase 130 head cattle on neighboring Seram Island to
replace some of the 4,000 head that farmers lost during the conflict. Included in this
procurement are also chickens for distribution and veterinary medicines. The Livestock
Department had unadvisedly announced this programme to intended beneficiaries and has
lost a bit of credibility as a result. In Jakarta, the team was advised that the Livestock
Department could procure the cattle itself. However, neither the department nor the
Tobelo project office was aware of this decision and no funds had been transferred for
this purpose.

This subproject also includes plans for cost recovery in kind. Farmers receiving cattle are
supposed to return an offspring to the Livestock Department for further distribution. In
addition, medicines bought by the subproject may be sold to recover costs. The
arrangements for transfer of these subproject inputs and for the revolving fund have yet to
be finalized.

In the Tobelo Harbour Workers Association subproject, the contract for construction was
signed and work began before the subproject was approved and before the CAP/C had
acted. Though now almost completed, the work is behind schedule.

Ternate. In the one sample subproject reviewed by the assessment team, there were no
serious implementation issues. However, the UNDP project monitor did report
community complaints that implementation teams, selected by CARDI, were not
transparent in their management of the implementation of community projects.
Nevertheless, in general the CARDI approach does appear to be a useful methodology to
operationalize participatory decision-making at the community level.




                                            31
Ambon. One subproject in the Ambon sample had problems in the contract award and
contractor performance (Ambon municipality), both of which were quickly corrected.
ICMC had no major implementation issues. The review of 15 MCI activities by the
project office yielded a number of concerns, however. Fishing equipment purchased for
village fisheries was destroyed due to improper use. A Rps. 14 million micro-credit
scheme made grants to 34 clients only three of whom were able to sustain the business
activity they started. A village water pump was out of service because the village could
not pay the electricity bill.

Contracting Processes. Contracting processes carried out by implementing partners were
the most problematic. Rent-seeking, manipulation of awards, and collusion among
bidders were all reported. In most cases, these issues were successfully managed.

Contracting processes carried out by UNDP ran into difficulties, particularly early in the
programme, with sole sourcing to INGOs. These contracts had to be approved by UNDP
headquarters, which questioned the legitimacy of requested waivers. Back and forth
communications created long delays processing these contracts, though they were
ultimately approved. UNDP Jakarta has recognized that it was not geared up to support
the implementation of the NMMRP at the outset. The teething process and related
learning has occurred. Adjustments have been made, such as the weekly meeting of the
CAP/C, to expedite the processing of procurement and contracting transactions. Aside
from the slowness, the assessment team did not find any evidence of major irregularities
in UNDP procurement and contracting.

Reconciliation Approaches. Should subprojects mainstream reconciliation and IDP re-
integration? There is a need to balance reconciliation needs with successful subproject
delivery. The establishment of the fishermen’s cooperative gave priority to IDPs. As a
result, 40% of the cooperative membership consists of men who were never fishermen.
Whether or not the coop will be viable remains to be seen.

The contractor re-building the Tobelo Harbor Workers Association office could have
used IDPs at sub-market wages as construction labor, but chose not to do so.

Muhammadiyah showed good faith by selecting a Christian contractor to rebuild the
former Muslim school, now planned to become a mixed religion school to encourage
reconciliation.

Recommendations.

      Subprojects should not, as a general rule, allow implementing partners to carry
       out procurement and contracting, except for low value, locally available supplies
       and services.
      The trade-offs between subproject efficiency and reconciliation need to be taken
       into account in subproject design.




                                           32
5. Subproject Results

This chapter of the report presents planned and actual results to the extent that results
have been achieved in the table below.




                                           33
                                  Actual or Observed Results of Sample NMMRP Subprojects

   Subproject                     Objectives                         Actual/Observed Results                         Comments
001: Supporting   1. Increase the stability and enhance        1. Extensive service delivery for IDPs     1. Activities for first objective
Stability,        returns through providing for basic needs    in the areas of food, shelter, water and   completed before assessment
Return and        of conflict affected families                sanitation                                 2. Mixed results reported by
Economic          2. Empowering local communities              2. Several projects in micro-credit,       UNDP monitoring team
Empowerment       through economic development projects        fisheries and agriculture
in Maluku         that improve livelihoods
003: Supporting   1. Build community capacity to address       1. Extensive development of self-help      This subproject appears to
Sustainable       community needs                              facilitation model                         effectively support IDP return.
Community         2. Increase access to and participation in   2. Completed
Reintegration     district governance                          3. Activities in progress
                  3. Documentation of processes
009:              1. Rebuilding Muhammadiyah Ambon             1. Construction completed for new          Subproject to be featured in a
Reconciliation    high school                                  high school building (two-story, 11        BBC report
Through           2. Improved quality of teaching and          rooms)
Education         better student results                       2. Teachers trained, but training will
                  3. Muhammadiyah Ambon provides a             not be applied until school begins
                  facility where students and teachers from    3. Mixing of student body and
                  both communities interact in an              teachers scheduled to begin late
                  environment of improved quality of           March 04
                  education facilities and services
010:              1. To renovate school buildings and          1. Seven schools rebuilt; teachers and Reconciliation activities so far
Reconciliation    reestablish education services located in    school committees trained              appear to be successful
through           the reconciliation area                      2. Reconciliation activities to begin
Education in      2. To support reconciliation process         with new school year June 04
Ambon             through education




                                                                   34
  Subproject                     Objectives                       Actual/Observed Results                       Comments
016:              Rehabilitation of 75 km of main electric    Completed on time and on budget         Actual household connections
Rehabilitation    lines serving 45 villages                                                           overstated in project report.
of Electric                                                                                           Actual connections not known
Power Lines
034: North        1. To assist in the rehabilitation of 15    Expanded to 27 villages in lieu of      The subproject has developed a
Maluku            villages in W. Halmahera                    work on objective 2; activities         strong model for mobilizing
community         2. To increase the capacity of a            completed and ongoing                   community participation.
Rehabilitation    minimum of five local partners to deliver
Initiative        services to conflict affected communities
046:              1. Support social recovery and              Headquarters building 95%               No reconciliation or training
Construction of   reconciliation through Tobelo Harbor        completed; behind schedule.             activities planned or budgeted.
Headquarters      Workers Association Members
for Tobelo        2. Improve management of the Tobelo
Harbor Workers    Harbor Workers Association through the
Association       provision of adequate training
048: Legal        Enable the District Court of Tobelo to      NGO subcontractor recently began        Project just started
Literacy          prepare and obtain funding for support      work.                                   implementation.
                  for an effective legal literacy campaign
064: Livestock    1. Build Local Government capacities to     86 trained vaccinators of which 60      No work focusing on IDP
                  enable increased direct participation in    were Christian;                         returns.
                  local economic recovery, particularly       47 farmers trained on cattle business
                  through UNDP-executed projects              management;
                  2. Contribute to sustainability of IDP      Equipped animal health post;
                  returns through strengthening the Human     6 staff trained in basic computer
                  Resources of Local Government line          skills.
                  departments




                                                                 35
  Subproject                     Objectives                       Actual/Observed Results                    Comments
068: Fisheries   1. To provide sustainable livelihoods to   1. A fishermen’s cooperative          Cooperative members are not
                 vulnerable fishermen in Gamsungi           organized consisting of 51 men,       all fishermen. The coop is not
                 2. Build capacities of the Fishermen’s     Christians and Muslims and members    yet functioning, pending
                 cooperative to ensure organizational       of various vulnerable groups; One     delivery of boat.
                 viability and reconciliation among         Pakjeko (boat) constructed and
                 members                                    equipped (90%) for installment sale
                                                            to cooperative.
                                                            2. Cooperative personnel trained in
                                                            managing fisheries business;

069: District    1. Enable District Government to better    1. Strategic planning seminar
Development      access stakeholder inputs through a pre-   successfully completed with 70
Planning         Rakorbang workshop                         stakeholder participants to produce
                 2. Enable District Government’s and        draft development strategy for
                 other stakeholders understanding of        endorsement by district assembly;
                 UNDP’s present and potential roles         2. Awareness created of UNDP
                 3. Enable local government                 activities in district at District
                 implementing partners to better            Development Expo.
                 understand UNDP’s development
                 approaches and implementation
                 requirements




                                                               36
                                                                                                              Annex
                                  LIST OF PERSONS INTERVIEWED

    CITY           ORGANIZATION                    NAME                                TITLE

JAKARTA
           DUTCH EMBASSY           Robert Quarles van Ufford    Coordinator for Development Cooperation
                                   Caroline Smit                Second Secretary
                                   Maurice Powchez              Second Secretary



           UNDP                    Gwi-Yeop Son                 Sr. Deputy Resident Representative/Programme
                                   Shahrokh Mohammadi           Deputy Resident Representative/Operations
                                   Handoko                      Assistant Deputy Resident Representative/Operations
                                   Patrick Sweeting             Head of CPRU
                                   Phillip Cooper               Programme Operations Manager
                                   George Conway                Programme Specialist
                                   Kristanto Sinandang          Sr. Programme Officer
                                   Djasinur Sirait              Programme Officer for Maluku
                                   Maja Suhud                   Programme Officer for N. Maluku
                                   Yanawati Sinaga              Admin Finance Officer



TERNATE    UNDP                    Jopie Sinanu                 Area Programme Manager
                                   Roeswita Aboe                Admin. Assistant
                                   Enardson Layang (Boy)        Programme Officer

           CARDI                   John Holmes - Levers         Field Coordinator North Sulawesi - N. Maluku
                                   Yani                         Program Officer for Physical Construction
                                   Arabiyani                    Program Officer for Economy-Income Generating
                                   Helry Muraga                 Field Supervisor




                                              37
TOBELO   UNDP                              Karen Janjua             Area Programme Manager - Tobelo/Galela
                                           Thamrin Hanafi (Tommy)   Public Relations Officer
                                           Mohammad Sabar           NPO - Engineering
                                           Linda J. Sumarauw        Programme Assistant
                                           Ferry                    Translator
                                           Pandu Hartoyo            NPO-Livestock
                                           Kusnadi                  NPO-Fisheries
                                           Ahmad                    Technical Assistant
                                           Christian Budi P.U.      Technical Assistant

         Government                        Jidon Hangewa            Bupati North Maluku

         PLN (State-own Electricity Co.)   Simon                    Supervisor

         Tobelo Harbor Workers Association Jordan                   Head

         Dept. of Livestock                Ir. Johny Banne          Head of Livestock Dept.
                                           Ir. Redisson Hamangau    Division Head of Animal Health
                                           Suwedi                   Vaccinator - Cadres

         Dept. of Fisheries                Hein, Msp.               Head if Fisheries Dept.
                                           Fredo                    Head of Production Unit
                                           Ismail                   Member of Fisherman Cooperatives
                                           Yakobus                  Member of Fisherman Cooperatives

         District Court                    Arthur Haugewa           Judge



AMBON    UNDP                              Caroline Tupamahu        Maluku Programme Manager (a.i)




                                                        38
                     Madja Rumatiga                 Programme Officer
                     Wismanto P.Kristinov Simaela   Programme Officer
                     Yusran                         Programme Officer
                     Nuraini Soulisa                Research/Information Officer

MCI (Mercy Corps)    Roberta Henes                  Programme Manager
                     Stephen Ray L. Mathias         Head of Field Office (Tual)
                     Ine                            Compliance Manager
                     Sandy                          Head of Field Office (Masohil)
                     Sherly

Ambon Municipality   Jopie Papilaya                 Mayor
                     Odie Papilaya                  (Team - Assistant) Dept. of Education
                     Yanti Wakano                   (Team - Assistant) Dept. of Education
                     Dra. Y. Matitaputi             Dept. of Education
                     Moses Maisela                  (Assistant III of Mayor - Prog. Coordinator)
                     Usman Talib                    (Team-Assistant)
                     Bi Kalinami                    (Team - Assistant)
                     Albert Tabalessy               Education Coordinator
                     Enrico Matitaputi              Kimpraswil/PU - Project Manager
                     Dr. H. Huliselan               (Sekda) Secretary of the District

ICMC                 Carmen Lowvy                   Program Manager
                     Wien                           Lead Program Officer
                     Hindun                         Program Officer
                     Dedi                           Program Officer
                     Max                            Program Officer

Bappeda Kota         P.A. Titaley                   Head of Bappeda

Muhammadiyah         Ismail                         Head of Education Council Muhammadiyah Maluku




                                    39
40

								
To top