dipecho evaluation drr sc central asia f by r9maENR

VIEWS: 12 PAGES: 14

									Terms of Reference for External Evaluation


1. Summary


                     Evaluation to measure the a) relevance; b) sustainability; c)
                     efficiency and effectiveness; and d) impact of the regional Disaster
    Title            Management project titled Central Asia project DipECHO 5 on
                     Increasing Communities’ Capacity for Child-Focused Disaster
                     Preparation and Response(DIPECHO)
                     The project to be evaluated runs from 1 October 2008 to 31
    Duration
                     December 2009.
                     Kyrgyzstan: Alai and Karakylja regions of Osh oblast and Kadamjai
    Location                     region of Batken oblast,
                     Tajikistan: Khatlon province, Jomi and Khuroson districts
                     Tajikistan Red Crescent (RC), MOES in Kyrgyzstan and COE in
    Partners
                     Tajikistan, MoE in Kyrgyzstan and Tajikistan
                     The purpose of the evaluation is to measure the a) relevance; b)
                     sustainability; c) efficiency and effectiveness; and d) impact at the
    Purpose of       end of the project implementation, through assessing performance
    evaluation       and progress of the project, including strengths and weaknesses,
                     while making recommendations on strengthening programme
                     management and implementation to improve future projects.
                     To support strategies that enable local communities and institutions
    Project          to better prepare for, mitigate and respond adequately to natural
    objective        disasters by enhancing their capacities to cope and respond, thereby
                     increasing their resilience and reducing their vulnerability
                     1. Final evaluation plan and tools (methodology and questionnaires),
                        report formats and deadlines to be agreed on with SC team.
                     2. Briefing with SC team
    Chronological    3. Meetings with key stakeholders
    summary of       4. Field visits to the 2 countries
    activities       5. Debriefing with SC team
                     6. Full report with evaluation results in Tajik and English presented
                        within two weeks after completion of the field visits (draft to be
                        submitted within two weeks after completion of the field study)


2. Background Information


Tajikistan is prone to many types of hazards, particularly floods, mudflows, landslides,
epidemics, drought, earthquakes, avalanches, insect infestation, and windstorm.
Records of the deadliest earthquakes in Tajikistan include the Faizobod earthquake in
1943 and the Gissar (Sharora) earthquake of 1989. The country experiences over 5000
tremors and earthquakes every year1. Floods occur in spring following heavy rain, or in
summer as a result of melting snow. Mudflows threaten 85% of Tajikistan’s area, with
some 35% situated in mudflow high-risk zone.2 The World Bank estimates that there is

1
    UNDP, Central Asia Human Development Report, 2007, pg 115-116
2
    UNOCHA, UNDAC Mission, p. 6
Terms of Reference for External Evaluation                                                          2


a 5% chance that a natural disaster could cost nearly 32% of Gross Domestic Product in
Tajikistan3. The pervasiveness of these natural disasters combined with the continuing
economic vulnerability of many households and communities leaves a low level of
disaster resilience at the community level.
The World Bank has reported clear signs of recovery and a high level of economic growth
in the last few years, impacting positively on poverty reduction. The percentage of the
population living on less then $2.15 per day decreased from 81%(1999) to 64% (2003).
Gross Domestic Product has increased by almost 40% in the past few years. Yet, figures
indicate that 4.58 million people are left with very poor coping capacities and limited
means, if any, for undertaking risk reduction actions.
There is also, however, increasing food insecurity among households due to
environmental and economic shocks over the last year. As a result, many rural
households have begun to employ hazardous coping mechanisms in order to make ends
meet. For example, 85% of households surveyed by the UN Food and Agriculture
Organization have switched to cheaper and less nutritious food, 40% have taken to
skipping meals altogether, and 30% have resorted to consuming smaller portions. This
situation is linked to declining food production, related to the fact that 82.3% of all land
and 97.9% of agricultural land suffers from erosion.4 Inadequate household food supply
of standard quality, lack of sanitation facilities, an unsatisfactory health care system, and
limited access to basic social services (such as water and education) have restricted the
capacity of the communities to withstand and recover from disasters and to institute
measures to mitigate the impact of disasters.
As a means for dealing with recurrent disasters in the country, the Committee for
Emergency Situations has been given the responsibility for national management and
coordination of all disaster-related activities. The frontline entities responsible for
Disaster Risk Reduction and response are the local administrative units, i.e.
village/community. International Federation of Red Cross and Red Crescent Societies and
the Tajik Red Crescent are presently of working at the jamoat level to develop disaster
management plans. The district authorities are unable implement DRR activities because
of a lack
of financing.
The Kyrgyz Republic is also a highly disaster-prone country, consistently ranking among
the most vulnerable in risk indices. Between 1992 and 1999, there were 1,210 natural
disasters registered in Kyrgyzstan alone5. The Ministry of Emergency Situations
estimates that 3.5 million – or roughly 70% of population – are living in disaster-prone
areas, with 1.5 million residing in high-risk areas6. Moreover, according to the World
Bank, 40% of the population lives in the parts of the country that are vulnerable to
strong earthquakes (registering 8-9 on the Richter scale) and there are 5000 potential
active landslide sites, of which 3500 are in the southern part of the country 7. The
economic impact of these events is staggering. United Nation Economics Commission for
Europe (UNECE) estimates that natural disasters cost Kyrgyzstan over $20 million in a
single year8.
One of the biggest disasters in recent history, a landslide in 1994 in Osh oblast, killed 11
people, affected 60,000 people, and caused approximately $3.6 million in damage9. Save
the Children began programming response to this landslide, distributing non-food items,
oil, rice, and flour to affected populations. Because most of the earthquakes and
3
  Christoph Pusch, Preventable Losses: Saving Lives and Property Through Hazard Risk
Management: A Comprehensive Risk Management Framework for Europe and Central Asia, World
Bank, 2004, pg 72.
4
  FAO 2005
5
  UNDP, Central Asia Human Development Report, 2007, pg 115-116
6
  UNDP Press release, 15 May 2006.
7
  Christoph Pusch, Preventable Losses: Saving Lives and Property Through Hazard Risk
Management: A Comprehensive Risk Management Framework for Europe and Central Asia, World
Bank, 2004, pg 7.
8
  ibid, pg 115.
9
  EM-DAT, www.emdat.be


Netherlands Red Cross DIPECHO V project                              7 August 201226 October 2009
Terms of Reference for External Evaluation                                                               3


landslides occur in remote areas and are generally small, they have not been
documented or monitored well. However, the cumulative impact of disasters in
Kyrgyzstan is thought to be great, having devastating impacts on the economy, including
structural damage and human casualties and a significant and detrimental impact on the
country’s ability to develop10.
Kyrgyzstan can little afford the cumulative impact of these disasters. The country has
experienced a period of economic recovery since 1996, with annual growth rates of over
5% for every year except 2002 and 2006. Gross National Income (GNI) per capita has
progress from a low of $280 in 2000 to $490 in 2006 11. However, much of this growth
has benefited urban populations, while rural populations have not benefitted from the
current relative prosperity. Save the Children’s examination of childhood poverty found
that rural families often cannot afford to send their children to school regularly. The
reasons for this were often financial, but also included the distance to school, the
absence of quality and regular teachers, and lack of heating in schools.12          More
shockingly, 61% of families in rural areas have drastically reduced food consumption,
decreasing the caloric intake overall and particularly decreasing the amount of protein. 13
In the event of a disaster, these families will have few additional coping mechanisms
upon which to rely. After the earthquake in January 2008, many people were not willing
to leave their scarce assets and cattle and therefore families set up tents dangerously
near their to the damaged houses. The food reserves (wheat flour, rice, and potatoes)
that the population usually stores for the winter were partially lost as a result of the
earthquake, as people not were unable to enter damaged buildings to recover them, or
the stocks froze in the cold weather.14 Mitigation measures are needed to ensure that
villages, and particularly poor households, are more resilient the next time an
earthquake strikes.
On the national level, the Ministry of Emergency Situations is mandated to provide
overall response coordination and search and rescue services, within the goals of
protection, prevention, and elimination of emergency situations. In the National
Strategy, the Ministry is also responsible for preparing communities for disaster
response. Frontline response units, the rayons and/or ayil okmotu often, do not have
disaster preparedness plans in place, yet many have the skills to develop them.
However, there are limited resources available to the Ministry and little technical
knowledge of how to “prepare communities”. The Ministry has worked closely with
international non-governmental organizations and donor community to map out areas
for responsibility for Disaster Risk Reduction interventions.


Central Asia (Kyrgyzstan and Tajikistan)
The major goal of the first DipECHO 5 project “Increasing Communities’ Capacity for
Child-Focused Disaster Preparation and Response”, which is being implemented by the
International Humanitarian Organization “Save the Children” and funded by DIPECHO is
capacity improvement of the population in reacting and managing natural disasters. The
project is oriented to Tajikistan (Khatlon oblast, Jomi and Khuroson rayons) and
Kyrgyzstan (Osh oblast, Alay and Karakulja rayons and Batken oblast, Kadamjay rayon).
Overall, on the territory of both states project will involve 19,432 children of 69 schools
from 70 villages. Trainings on risk reduction and management on natural disasters will
be conducted within the framework of the project and will involve participation of 51,320
community members, 560 members of the village response teams in emergency

10
   OCHA situation report, 4 January, 2008.
11
   UNDP, Central Asia Human Development Report, 2007, pg 240.
12
   Mehrigiul Ablezova, CHIP Report 15: A Generation at Risk? Childhood Poverty in Kyrgyzstan ,
Save the Children, 2004, page 41.
13
   Mehrigiul Ablezova, CHIP Report 15: A Generation at Risk? Childhood Poverty in Kyrgyzstan ,
Save the Children, 2004, page 75.
14
   (http://www.reliefweb.int/rw/rwb.nsf/db900sid/EDIS-7APN8B?OpenDocument&rc=3&cc=kgz)


Netherlands Red Cross DIPECHO V project                                   7 August 201226 October 2009
Terms of Reference for External Evaluation                                                          4


situations, village school teachers, representatives of local authorities and trainers. Total
number of direct beneficiaries will reach over 71,000 residents of Tajikistan and
Kyrgyzstan.
The overwhelmingly important component of the project is training and management of
natural disasters. Local authorities, school teachers and population on the whole will
improve their knowledge and gain skills to respond in natural disasters. School Students
capacity will be built and strengthened through the trainings and events on awareness
raising conducted by teachers. As the part of DIPECHO project some small but effective
mitigation projects (initiatives) proposed and selected by each community will be
supported.       It will help to reduce the aftermath of natural disasters in
future.Introductory presentation has been done in REACT meeting on SC DRR activities
for DipECHO partners. Aim, objectives and all project related detail plan and activities
shared with the partners and further collaboration opportunities considered with Red
Crescent in conducting join trainings for community and schoolchildren. ToT for 30
national trainers and 12 SC staff has been conducted by international consultants in TJ
and KG. So during five day of intensive work participants have discussed the terms
associated with Disaster such as hazard, risk, risk management, vulnerability, capacity,
mitigation, disaster and etc; ways of how the adults learn and looked through the
experiential learning as the good example of adults learning, how to develop and submit
small project proposals for the mitigation activities on the community level, how to
ensure participation of community members and especially children in DRR activities,
how to develop comprehensive community preparedness and response plan, how to
establish CERT (Community Emergency Response team) and their responsibilities, how
to conduct training and what kind of training for CERTs and etc.Community selection
criteria’ developed in consultation with Local Committee of Emergency Situation. 43
villages in Tajikistan and 27 villages selected in KG in accordance to the developed and
approved criteria jointly with province Committee of Emergency Situation/MOES. 43
mahalla meetings in TJ and 27 village meeting in KG have been conducted for the
verification of selected villages by SC, province Emergency Committee and local
Hukumats. School selection criteria developed in consultation with District Education
Departments and 50 schools in Tajikistan and 23 schools in Kyrgyzstan were selected in
conformity with developed criteria.


3. Purpose


The purpose of the evaluation is to measure the a) relevance; b) sustainability; c)
efficiency and effectiveness; and d) impact at the end of the project implementation.


  3.1     Specific objectives

  The evaluation should contain conclusions and recommendations at both strategic and
  operational level to improve future disaster risk reduction projects of the Save the
  Children of Central Asia and other potential partners (including government and donor
  agencies).
  The objective of evaluation:
The evaluation will focus on SC project in identifying the impacts of the project on
targeted beneficiaries. The evaluation will place particular emphasis on the impact of the
project on children and community.

   Expected Result                           Indicators




Netherlands Red Cross DIPECHO V project                              7 August 201226 October 2009
Terms of Reference for External Evaluation                                                                   5


  Expected Result                            Indicators
  1. Strengthened                            70% of DRR meetings and training exercises include
     coordination mechanisms                 the participation of relevant local government
     at national and local                   stakeholders and Community Emergency Response
     levels to protect children              Team (CERT) representatives
     in disasters.                           30 relevant government stakeholders trained in the
                                             core principles of “Child Protection in Emergencies”
                                             and “Education in Emergencies”
  2. Increased capacity of                   80% members (448 community response team
     target communities and                  members) trained in promoted disaster risk
     local government                        reduction (DRR) strategies.
     stakeholders to cope with               78 hazard/risk/vulnerability maps developed
     disasters in a way that                 90% DRR plans (63 plans) that include at least one
     protects children.                      child protection mechanism.
                                             Stakeholder attendance at coordination and planning
                                             Consultation mechanisms developed between local
                                             government and communities.
  3. More protective                         80 simulation exercises in communities and schools
     environment created                     90 mitigation projects undertaken
     through risk mitigation                 75% of targeted adults (39,069 individuals) who
     interventions for children              trust the effectiveness of the DRR early warning
     and adults                              system (EWS)
  4. Increased capacities of                 100 disaster awareness campaigns carried out in
     school children and                     targeted communities by children’s groups
     teachers to prepare for                 80% targeted school children (15,545 children) that
     and respond to disasters                know at least one disaster risk reduction message
                                             promoted by the project.
                                             70% of children (13,602 individuals) in targeted
                                             schools who know preparedness measures.
  5. Improve quality of child                Development, publication and dissemination of
     protection strategies in                awareness raising and capacity building materials on
     DRR programming                         child protection strategies in DRR.
     nationally and locally                  100 participants in national and local level project
                                             monitoring and evaluation focused on identifying
                                             lessons learned, best practices and impact.


  3.2       Scope of work

  More specifically the focus of the evaluation should be on:

   a.       Relevance

   The evaluation will assess the relevance of the project, related to:
            Ithe appropriateness of training on activities for each target group
            the number of simulation exercises in communities and schools as well as their
             appropriateness and quality
            The content and quality of hazard/risk maps developed and involvement of community and
             children in the process of developing the maps
            Participation in project activities by local government and community emergency response
             teams
            the appropriateness of approaches to community mitigation planning and implementation




Netherlands Red Cross DIPECHO V project                                       7 August 201226 October 2009
Terms of Reference for External Evaluation                                                                    6


   b.       Sustainability

   The evaluation will assess the sustainability of the project, related to:
             the degree project contributed to strengthening coordination mechanisms at local and
             national level and the impact this had on children
            the DRR plans and their implementation by communities
            consultation mechanisms between communities and local authorities in relation to DRR
             activities
            improving DRR in communities and children’s participation in the design and
             implementation of mitigation projects
            the degree to which targeted adults feel that their communities are better prepared for
             emergencies and that early warning systems are effective
            Identify the impacts and reach of children’s groups disaster awareness campaigns
            the effectiveness of children’s mitigation projects and mitigation project outcomes
            teachers in targeted communities ability to implement INEE minimum standards for
             education in emergencies

   c.       Efficiency and Effectiveness

   The evaluation will assess the efficiency and effectiveness of the project, related to:
            the effectiveness of project strategies and approaches to raising awareness to the need to
             include child protection in DRR strategies at local and national levels
            Quantity, quality and timeliness of workshops, and training
            Community DRR disaster preparedness, mitigation and response
            Awareness raising at community level and for government partners
            Coordination with ECHO and partners and cooperation with other agencies
            Management, coordination and guidance at all levels
            Project and financial management and reporting
            Monitoring and Evaluation.

   d.       Impact

   The evaluation will assess the impact of the project, related to:
            building and strengthening child protection in national and local DRR strategies.
            Children’s participation in DRR activities at community level
            coordination mechanisms at national and local levels to protect children in
             disasters.


4. Work Plan


The final evaluation plan and tools will be determined following discussions between the
evaluator and the project team prior to the commencement of the evaluation study.
In working with partners, the evaluator is expected to be proficient in using a range of
participatory tools for data gathering and analysis, including quantitative and qualitative
evaluation techniques, conducting key informant interviews and leading small group
presentations.
The evaluation will also involve substantive documentation review, and report writing.
The evaluation is to be carried out in 3 stages:




Netherlands Red Cross DIPECHO V project                                        7 August 201226 October 2009
Terms of Reference for External Evaluation                                                              7


    1. Briefing and secondary data review in Dushanbe [4 days]

         -     A briefing will take place at Dushanbe office with responsible staff. All relevant
               documents and necessary clarifications will be provided. In this briefing, the
               detailed evaluation plan will be agreed.
         -     Secondary data review and analysis of relevant documents. This phase is to
               allow a careful planning of the activities/visits to be undertaken in the field.
         -     A formal debriefing will take place with preliminary findings before departure
               of any consultant from any country together with SC representative.


    2.       Field Study: [8 days in Kyrgyzstan, 8 days in Tajikistan including travel]

         -     The evaluator must work in close collaboration with the SC, local authorities
               and other partner organizations.
         -     Visit to participating community and school should include interviews with
               authorities, SC staff, and beneficiaries.
         -     Meetings with main stakeholders in the different countries, including with the
               donor.


    3. Debriefing [5 day] and submission of report:

         -     A debriefing with SC immediately after the field visit. Feedback should be
               included in the final report.
         -     A draft report needs to be submitted to SC within 2 weeks upon completion of
               the field study to give SC the opportunity to provide feedback. SC will provide
               its feedback to the consultant within 1 week upon receipt of the draft report.
         -     A Final Report in accordance with the format given in the next paragraph shall
               be submitted electronically and in 2 hardcopies (one for SC and DG ECHO) in
               both Tajik and English, within 1 week upon receipt of the SC comments on the
               draft report.


5. Reports


The evaluation results will be consolidated in a report, in English with a maximum length
of 75 pages (including annexes). The report format should include as follows:


    1. Cover page (title of the evaluation report, country, project title, sector, date,
       name of evaluators, indication that ‘the report has been produced at the request
       of SC and financed by the European Commission (DG ECHO). The comments
       contained herein reflect the opinions of the evaluator only’.


    2. Table of contents


    3. Executive Summary An Executive Summary of the main conclusions, lessons
       learned and recommendations of the evaluation should be no more than two
       pages with cross-references to the corresponding page or paragraph numbers in
       the main text. Proposed structure:

               -   Evaluated project
               -   Date of the evaluation
               -   Evaluator’s name
               -   Purpose and methodology
               -   Main conclusions related to the evaluation criteria and cross-cutting issues



Netherlands Red Cross DIPECHO V project                                  7 August 201226 October 2009
Terms of Reference for External Evaluation                                                             8


              -    Specific recommendations for a) implementing partners, b) SC c)
                   DIPECHO and d) others
              -    Lessons learned and good practices.


    4. Main body of the report The main body of the report shall elaborate on the
       points listed in the Executive Summary. It will include references to the
       methodology used for the evaluation and the context of the project. For each key
       conclusion, there should be a corresponding recommendation. Recommendations
       should be feasible and pragmatic and should take careful account of available
       resources.


    5. Annexes

              -    Terms of Reference
              -    List of persons interviewed and sites visited
              -    Map of the areas covered
              -    Abbreviations
              -    All confidential information shall be presented in a separate annex.


6. Team composition and skills of the evaluators


The evaluator will have extensive experience, knowledge, and skills in planning and
implementing project evaluations. The selected evaluator should have a solid experience
in community based disaster management and good knowledge of the Central Asian
culture. Knowledge of the English language is obligatory. The evaluator will work with an
assistant of staff who will provide translation of any documents and during field
interviews The evaluator will be competent in quantitative and qualitative research,
evaluation tools, data collection and analysis, and have sound report writing skills.
Ideally, the evaluator will have experience in review or evaluation of disaster
preparedness projects and familiarity with the rural development context in Central Asia.


7. Timetable


The evaluation will last a maximum of 25 days (16days for field visits, 4 for briefing and
de-briefing at Dushanbe Office; 5 days for preparation of a draft report, beginning with
the date of signature of the contract and ending with the acceptance of the final
evaluation report.


I. Payment and logistics

The evaluator will be paid a lump sum, from which all costs will be covered:
              -    Evaluator fee for the evaluator
              -    Transport/air tickets (international)
              -    accommodation


The evaluator is responsible for. The SC will arrange formal meetings with authorities,
interpretation and translation, transport, accommodation and other logistical support .




Netherlands Red Cross DIPECHO V project                                 7 August 201226 October 2009
Terms of Reference for External Evaluation                                                     9


8. Key Informants


The respondent profile should include all key players in the DIPECHO 5 project at all
levels as mentioned in the background of this TOR. A representative sample of all
beneficiary groups needs to be taken: school children, school teachers, CERTs members,
different groups within the community (e.g. elderly, women), local authorities, etc.
A strict division needs to be made between those CERTs and schools that have been
involved in DIPECHO project.


9. Ownership of data


All data collected during the evaluation will be handed over to and become property of
the SC in the database form agreed on at the beginning of the survey. Evaluators must
therefore treat findings in a confidential manner and require advance written approval
from the SC for sharing any information with third parties, in whichever form.




Netherlands Red Cross DIPECHO V project                         7 August 201226 October 2009
Terms of Reference for External Evaluation                                                         10




Annex 1 Criteria and sample questions


1. Is the Action relevant?

Relevance is concerned with assessing whether the Action correctly identified problems
and real needs and whether the projects funded under the Global Plan/Action were in
line with local needs and priorities as well as with donor policy. One should therefore
analyse in addition:
-   How well the reality of problems and needs, as well as target beneficiaries were
    identified and incorporated into the Action plan,
-   how local capacities for absorbing the aid were analysed;
-   How the Actions were prepared, noting particularly any obvious omissions. This
    should include policy assessments, sector analysis, planning workshops and how far
    they were incorporated into the Action.
-   Whether prior consultations were undertaken with appropriate people on the spot, i.e
    the Delegation, national and local authorities, intended beneficiaries and other donors
    and aid organisations (the latter being particularly important to ensure
    complementarity and avoid overlap).
-   How the programme complements and enhances, rather than duplicates and hinders,
    related activities carried out by other E.C. services, governments and donors.


2. Is the Action effective?

Effectiveness measures the extent to which the activities funded under the Action
achieve their purpose, or whether this can be expected to happen on the basis of the
results. Implicit within the effectiveness criterion is timeliness. There might be value in
using it more explicitly as one of the standard criteria because of its importance in the
assessments of emergency programmes.
Therefore, effectiveness should indicate the real difference made in practice by the
activities funded, how timely the intervention was; equally how far means were used to
their maximum effect, how far the intended beneficiaries really benefited from the
products or services it made available. Similarly, issues of resourcing and preparedness
may be addressed. Points to be taken into consideration:
-   whether the planned benefits have been delivered and received, as perceived mainly
    by the key beneficiaries, but also taking account of the views of donor management,
    the responsible national Government authorities, and other interested parties (NGOs,
    local organisations, etc);
-   if the assumptions and risk assessments at results level turned out to be inadequate
    or invalid, or external factors intervened, how flexibly the various levels of
    management adapted to ensure that the results would still achieve their purpose. To
    summarize, “were the right things done” to ensure that the potential beneficiaries
    actually benefited?;
-   Whether the balance of responsibilities between the various stakeholders was
    correct. What accompanying measures were or should have been taken by the
    partner authorities, and with what consequences?;
-   What unplanned effects of the Action, if any, have there been in the target areas?;
-   Whether any shortcomings were due to a failure to take account of cross-cutting or
    over-arching issues such as gender, environment and security during
    implementation.




Netherlands Red Cross DIPECHO V project                             7 August 201226 October 2009
Terms of Reference for External Evaluation                                                           11


10. How efficient were the various activities?


Efficiency measures how well the various activities transformed the available resources
into the intended results (outputs) maximising quality, quantity and timeliness.
This links with the question “were things done in the best way possible?” and thereby
also addresses the concept value-for-money, that is whether similar results could have
been achieved more by other means at lower cost in the same time. An analysis of
Efficiency will therefore focus on:
-   the operational capacities of the partners;
-   the systems of control and autoevaluation set up by the donors and partners,
    whether it was appropriate, accurate and followed up;
-   the quality of day-to-day management of the aid, for example in:
    o management of the budget (including whether an inadequate budget was a
        factor);
    o management of personnel, information, supplies, etc.
    o whether management of risk was adequate, i.e. whether flexibility was
        demonstrated in response to changing circumstances;
    o relations/co-ordination with local authorities, institutions, beneficiaries, other
        donors;
    o respect of deadlines.
-   costs and value-for-money: how far the costs of the activities were justified by the
    benefits     - whether or not expressed in monetary terms- compared, mutatis
    mutandis, with similar projects, activities or approaches elsewhere;
-   whether the chosen indicators of efficiency were suitable and, if not, whether
    management amended them;
-   did any unplanned results arise from the activities?


3. What was the impact of the Action?

Impact looks at the wider effects of the Action. Impact can be short or long-term,
intended or unintended, positive or negative, macro (sector) or micro (household).
This section should therefore show:
-   To what extent the planned overall objectives were achieved, and how far that was
    directly due to the Actions financed. This should take into account aspects such as
    contribution to the reduction of human suffering, effects on health and nutrition,
    effects of the humanitarian aid on the local economy, on local capacity-building, etc;
-   whether there were any unplanned impacts (e.g., creation of dependency on
    humanitarian aid), and how they affected the overall impact;
-   where appropriate, all gender-related, environmental, security and human rights -
    related impacts and any lack of overall impact resulting from neglecting these issues;
-   Whether the desired wider impact could have been better achieved otherwise.


4. What is the sustainability of the Action?

Sustainability is concerned with measuring whether an activity is likely to continue
after donor funding has been withdrawn and also whether its longer-term impact on the
wider development process can also be sustained at the level of the sector, region or
country.
In the case of strictly humanitarian Actions, connectedness might replace the concept of
sustainability. It is defined as the extent to which activities of a short-term emergency
nature, which are not in principle supposed to be sustainable, are carried out in a
context which takes longer-term and interconnected problems into account.
The LRRD (Linking Relief, Rehabilitation and Development) issue is of great importance
for the Commission in this context since it addresses the need to link relief activities with



Netherlands Red Cross DIPECHO V project                               7 August 201226 October 2009
Terms of Reference for External Evaluation                                                          12


longer-term development Actions, in order to reduce deficiencies resulting from the
different approaches and priorities.
An analysis of sustainability will focus on the aspects below. Their relative importance
will depend on the nature of the project. It is useful to examine how concern for, or
neglect of, one or other of these factors may have affected the sustainability of the
outcome:
-   ownership (agreed communality) of objectives and achievements, e.g. how far all
    stakeholders were consulted on the objectives from the outset, and whether they
    agreed with them and remained in agreement throughout the duration of the project;
-   policy support and the responsibility of the beneficiary institutions, e.g. how far donor
    policy and national policy corresponded, and the effects of any policy changes; how
    far the relevant national, sector and budgetary policies and priorities affected the
    project; what was the level of support from governmental, public, business and civil
    society organisations; whether national bodies provided resources;
-   institutional capacity, e.g. the degree of commitment of all parties involved, such as
    Government (e.g. through policy and budgetary support) and partner institutions;
    the extent to which the project is integrated in local institutional structures; whether
    counterparts were properly prepared for taking over, technically, financially and
    managerially;
-   the adequacy of the project budget for its purpose;
-   socio-cultural factors, e.g. whether the project is in tune with local perceptions of
    needs and of ways of producing and sharing benefits; whether it respects local
    power-structures, and beliefs; if it seeks to change any of them, how well-accepted
    are the changes both by the target group and by others; how well it was based on an
    analysis of such factors, including target group/ beneficiary participation in design
    and implementation; the quality of relations between the external project staff and
    local communities, notably their leaders;
-   financial sustainability, e.g. whether the products or services provided were
    affordable for the intended beneficiaries and remained so after funding ended;
    whether funds were available to cover all costs (including recurrent, operating and
    maintenance costs), and continue to be so after funding ended;
-   technical (technology) issues, e.g. whether (i) the technology, knowledge, process or
    service provided fitted in with existing needs, culture, traditions, skills or knowledge;
    (ii) alternative technologies were considered, where there was a choice; (iii) the
    intended beneficiaries were able to adapt to and maintain the technology acquired
    without further assistance; having minimal maintenance, operating and replacement
    costs; and using national resources (notably, in creating jobs) together with
    minimum waste;
-   wherever relevant, cross-cutting issues such as the already mentioned Linking Relief,
    Rehabilitation and Development (LRRD) question, gender, environmental impact,
    respect of human rights, etc.




Netherlands Red Cross DIPECHO V project                              7 August 201226 October 2009
Terms of Reference for External Evaluation                                                                  13


Annex 2: List of Abbreviations



CA                                    Central Asia
CARESI                                Central Asia Earthquake Safety Initiative
CERT                                  Community Emergency Response Team
CLO                                   Child led organization
COES                                  Committee of Emergency Situation and Civil Defence
DM                                    Disaster Management/ Disaster Mitigation
DP(P)                                 Disaster Preparedness (& Prevention)
DRR                                   Disaster Risk Reduction
ECHO                                  European Commission Humanitarian Office/Aid Department
FA                                    First Aid
HQ                                    Headquarters
IOM                                   International Organization for Migration
LDMC                                  Local Disaster Management Committee
M&E                                   Monitoring and Evaluation
MoES,                                 Ministry of Emergency Situations
MoU                                   Memorandum of Understanding
MoE                                   Ministry of Education
NGO                                   Non Governmental Organisation
PRA                                   Participatory Risk Assessment
RCS                                   Red Cross Society
RDRT                                  Regional Disaster Response Team
ToT                                   Training of Trainers




Netherlands Red Cross DIPECHO V project                                      7 August 201226 October 2009
Terms of Reference for External Evaluation                                                      14


Annex 3: List of useful documents and reports to conduct the evaluation




Netherlands Red Cross DIPECHO V project                          7 August 201226 October 2009

								
To top