Cyclone Nargis Emergency Response (CNER) Program

							Cyclone Nargis Emergency Response
         (CNER) Program




       End of Program (EOP)
         Evaluation Report

         Prepared by TANGO International
                  May 14, 2009
Executive Summary
By claiming the lives of 140,000 individuals, and destroying thousands of homes, schools,
roads, and water sources, Cyclone Nargis is easily classified as the worst natural disaster in
Myanmar’s history. One year later, storm victims continue to struggle with the
consequences of Cyclone Nargis in terms of lost family members, severe damage to
productive assets and widespread livelihood insecurity.

World Vision Myanmar joined an unprecedented effort on the part of international non-
government organizations (INGOs), the United Nations, community-based organizations,
religious institutions and the Government of Myanmar to meet the emergency needs of
affected households. WVM worked in Yangon Division and initiated large-scale relief
activities in a new program area, Ayeyarwady Division. With the overall goal of “reducing
the immediate vulnerability of households and communities affected by Cyclone Nargis in
Yangon and Ayeyarwady Divisions”, the Cyclone Nargis Emergency Response (CNER)
Program focused on providing emergency assistance in the form of food, shelter, household
items, water, sanitation, child protection and livelihood support.

This report presents the results of quantitative and qualitative research conducted by the
CNER End of Project (EOP) Evaluation Team in February/March 2009. The objective of the
study was to examine the relevance and effectiveness of World Vision Myanmar’s
emergency response, assess the application of Humanitarian and Emergency Affairs (HEA)
standards, identify linkages between the emergency activities and longer-term
rehabilitation efforts, and document lessons learned for the improvement of future
programming.

The summary below highlights the findings of the EOP Evaluation.

Impact of the Cyclone Nargis on Affected Households
While thousands of households in Yangon Division were severely affected by the disaster,
damage caused by Cyclone Nargis was much more extensive and the scale of emergency
response considerably larger in Ayeyarwady Division. Damage to, or complete destruction of
homes was identified by both Yangon and Ayeyarwady households as the most significant
problem caused by Cyclone Nargis, followed by limited access to food and loss of
livelihoods. Loss of productive assets and damage to agricultural land and water resources
has also had a severe impact on households, especially in Ayeyarwady Division.

The survey found that poor, landless households were among the most vulnerable to the
impacts of Cyclone Nargis. This was primarily due to their lack of food stocks and productive
assets, as well as their dependence on casual labor as the sole source of household income.
Eighty-five percent of all respondents claim that their household income has decreased
since Cyclone Nargis. Meanwhile, over half of all households surveyed reported that their
spending on loans has increased over the same period. Two-thirds of all households in the
sample report that they have made little progress in recovering from problems caused by
Cyclone Nargis. Each of these findings underscore the importance of continuing livelihood
support for beneficiary households, especially the landless. Special emphasis should be
placed on securing off-farm income generating opportunities.


Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   ii
Summary
Emergency Food Assistance
Emergency food assistance provided by the CNER Program was particularly effective given
the damage to crops and stored food stocks. Food assistance also alleviated the strain of
households dependent on market purchases of food for household consumption. Targeting
of food and other forms of assistance was made somewhat difficult in Yangon Division due
to the temporary relocation of thousands of households. WVM set up Food Management
Committees to aid the beneficiary selection and information process. The challenges faced
included villages that did not want to exclude anyone from receiving food aid, regardless of
need. Overall however, the vast majority of respondents reported that food assistance
provided by WVM was delivered in a timely manner the CNER Program was able to ensure
that food assistance was provided to the most vulnerable households. WVM has retargeted
food aid several times since the emergency to ensure that households continuing to receive
food aid are the most vulnerable.

Emergency Shelter
Among the entire sample, over two-thirds of respondents’ households were completely
destroyed. Accordingly, temporary shelter kits distributed by WVM were repeatedly cited as
perhaps the most important and useful forms of assistance provided under the CNER
Program. The majority of households were satisfied with the housing assistance they
received from WVM. The evaluation feedback did reveal some criticism of the targeting
criteria and delayed distribution of kits. FGD participants in Yangon Division in particular
claimed that many needy households were deemed ineligible for housing assistance, and in
some communities in the Delta, shelter kits did not arrive until two to three months after
the cyclone. Ten months after the cyclone, many households say that they have been
unable to re-establish ‘permanent’ housing due to continuing livelihood insecurity. Some
respondents report that they are living in overcrowded and sub-standard conditions and are
still using the emergency shelter materials, though these have deteriorated and are no
longer sufficient to protect household members from the elements.

Water, Sanitation, Health and Hygiene
Qualitative data confirms that damage to water sources for household consumption was
much more severe in Ayeyarwady Division than in Yangon Division due to the reliance of
rural households on ponds and other unprotected sources. The efforts of WVM to
rehabilitate ponds and provide storage and carrying containers is widely credited with
improving access to safe water, especially during the dry season. The CNER Program was
also instrumental in improving household access to sanitary latrines in the wake of Cyclone
Nargis though households in Ayeyarwady Division were much more likely to receive this
type of support than those in Yangon Division. In combination with training and materials
for improved household hygiene, the construction of latrines was credited by households
with increasing awareness of illness prevention and decreased rates of child diarrhoea.
Despite these successes, evidence suggests a continuing need for greater awareness of
improved sanitation and hygiene practices among beneficiary households.

Child Protection
Quantitative and qualitative findings provide a somewhat divergent perspective on issues
related to child protection. While quantitative data obtained through the household survey


Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   iii
Summary
demonstrate relatively high awareness of children’s rights, focus group discussions and key
informant interviews identified few current challenges faced by children or specific steps
that are being taken to ensure their protection at the community and household level.
Beneficiary households were highly complementary of the WVM’s provision of Child
Friendly Spaces (CFS) to communities affected by Cyclone Nargis. They credit these facilities
with easing the psychological trauma experienced by children as a result of the disaster. The
contributions made by beneficiary communities to CFS, mostly in the form of construction
and provision of volunteers bodes well for the sustainability of child protection activities
planned under the ongoing Cyclone Nargis Rehabilitation/Recovery Program (CNRP).

Disaster Risk Reduction
Among the entire sample, only one quarter (25.8%) of households claimed to have
experienced a natural disaster prior to Cyclone Nargis. Less than one-fifth (18.2%) reported
receiving information on how to best protect themselves from potential disaster since
Cyclone Nargis. The primary concern of households regarding vulnerability to future natural
disasters is the lack of storm-proof structures within the community. Together, these
findings suggest an urgent need for improved access to disaster risk reduction and
preparedness information and development of disaster preparedness plans at the
community and household levels. At the time of the evaluation, two months before the
onset of the rainy season, WVM disaster risk reduction activities were planned but not yet
underway.

Accountability
WVM posted information, established “feedback boxes” at distribution sites and initiated
post-distribution monitoring procedures for food assistance. These were useful measures to
support accountability.     However, evaluation findings suggest the need for WVM to
strengthen its use of participatory approaches to project monitoring. Among the entire
sample, just over half (55%) of all respondents claimed they were encouraged to give
feedback on CNER Program activities, and only 41 percent reported that WVM “changed its
activities or program in a positive manner as a result of feedback from the community”.
WVM should adopt a more proactive approach to project monitoring in order to ensure the
appropriateness of activities and benefit from lessons learned under the CNRP.

Program Management
• CNER Program Design
Activities carried out under the CNER Program were both appropriate and effective in
meeting the emergency needs of households affected by Cyclone Nargis. While WVM was
admittedly unprepared for the scale of the disaster, especially in terms of technical capacity,
it has won the praise of beneficiary communities and institutional partners for its
comprehensive and relatively swift response to the disaster. WVM staff responded well to
the challenge of adapting project activities in response to rapidly changing conditions at the
field level.
•Logistics and Coordination
WVM’s response received high praise from its two main institutional partners, WFP and the
Government of Myanmar Department of Social Welfare. In the days following Cyclone
Nargis, coordination of WVM’s emergency response was made especially challenging due to

Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   iv
Summary
disruptions in communication and transportation infrastructure. Prior to the arrival of WV’s
Global Rapid Response Team (GRRT), WVM staff were also challenged by their lack of
previous experience in emergency programming. While CNER Program staff participated in
UN cluster meetings to exchange information on selection criteria and beneficiary lists,
decisions made at the district level were often not conveyed to operations staff at the
township and village level. As a result, field managers were often forced to engage in their
own negotiations with other stakeholders which led to gaps in coverage and inefficiencies in
the distribution of emergency assistance. At the same time, WVM was under pressure to
distribute all available materials to beneficiary communities. Much of the resulting tension
created between program, finance and operational staff was attributed to inadequate
communication that limited the flexibility of CNER Program implementation in response to
changing conditions. Similar communication and coordination challenges existed between
CNER staff and members of WV’s Global Rapid Response Team (GRRT). CNER staff felt that
seconded staff from Support Offices did not use local staff’s contextual understanding of
disaster-affected areas and that TDYs were too short to be effective; at the same time,
external staff felt that WVM staff did not have the right skills and experience to mount an
effective response.
•   Technical Capacity
Although WVM had capacity in child protection and some experience in food assistance
programming prior to Cyclone Nargis, it was considerably less prepared in the areas of
shelter, WASH and livelihoods. Despite these gaps in technical capacity, CNER Program staff
deserve considerable credit for learning on the job and drawing on the experience of
institutional partners. Staff members acknowledge that the CNER Program was negatively
affected by the high turnover in the critical position of Response Manager. In the short-
term, many of the staffing needs for WVM’s emergency response were met by the
secondment of ADP staff. Over subsequent months, WVM has increased its technical
capacity by recruiting for technical staff in WASH and livelihoods and providing training to
both program and field staff in assessment, program design and project monitoring. In order
to effectively implement the CNRP, WVM will need to proactively address capacity needs,
particularly in community-based disaster risk management.
•   Monitoring and Evaluation
The rapid assessments carried out in the days following Cyclone Nargis accurately identified
the most urgent needs of affected communities and served as the basis for activities
implemented under the CNER Program. However, the lack of context-specific information
from subsequent needs assessments likely inhibited WVM’s ability to rapidly adapt
programming to changes at the community level. As noted, WVM took steps to ensure the
accountability of the CNER Program to beneficiaries, but needs to intensify its methods of
obtaining feedback from participating communities and households to encourage them to
play an active role in project monitoring. This will help identify lessons learned for the
improvement of current and future programs.
•   Linkages with the Cyclone Nargis Rehabilitation/Recovery Program (CNRP)
The CNRP activities build on the successes of WVM’s emergency response and draw on its
organizational strengths, technical capacities and financial resources. They include a strong
emphasis on livelihood recovery but do not address other critical needs identified by survey
respondents, including housing, credit or income generation opportunities. In the current

Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   v
Summary
transition period, there is a need for a more comprehensive assessment of rehabilitation
needs in target communities. In addition, new project design documents have been
developed under considerable time pressure and without adequate input from technical
specialists or field staff. These factors inhibit WVM’s objective to transition from “supply-
driven” emergency assistance to a more community-driven approach to rehabilitation and
recovery. The slow pace of CNRP implementation is due in part to inadequate numbers of
qualified staff at the field level and an inefficient transition from emergency activities
toward longer-term recovery interventions. Both of these issues pose significant challenges
to the achievement of CNRP objectives before the end of the project in December 2009.
The limited progress made by the CNRP also affects beneficiaries, especially in Ayeyarwady
District, where a poor harvest and an inability to obtain seeds and other agricultural inputs
are hampering livelihood recovery.

HEA Standards
WVM adhered to HEA standards by carrying out an initial emergency needs assessment and
introducing accountability measures in its food distribution procedures. It also held a real-
time learning event three months into the emergency response. The recovery strategy
focuses on livelihoods and community recovery, guided by several strategy documents.
Post-disaster risk reduction activities among target communities are planned.

Recommendations

    Appropriate programming depends on quality needs assessments. WVM should ensure
    that adequate human and financial resources are allocated to regular and
    comprehensive assessments of target communities to ensure that rehabilitation and
    longer-term recovery programming adequately responds to changes in conditions and
    evolving beneficiary needs. Rather than focusing strictly on short-term beneficiary needs
    and programming priorities within pre-determined technical sectors, a comprehensive
    vulnerability assessment examines the full range of factors that contribute to persistent
    vulnerability to food and livelihood insecurity. This effort will entail building the
    technical capacity of field staff in assessment methodologies and program design.

    In order to successfully implement rehabilitation programs in target communities, CNRP
    staff must each share a common understanding of strategic priorities and core activities.
    To support this effort WVM should develop a concise, comprehensive document on
    CNRP rehabilitation strategies and implementation methods. The document should be
    disseminated and discussed with all CNRP staff, especially field staff.

    Both the CNER Program and the CNRP have at times suffered from a lack of coordination
    and communication among various staff. To alleviate this situation and prepare for
    future emergency and recovery programming, WVM should initiate development of a
    staffing plan that accurately identifies staff recruiting priorities and clarifies roles and
    responsibilities at all levels of the organization. Delineation of specific responsibilities
    should not be at the expense of the flexibility or teamwork that is vital in emergency
    contexts such as that following Cyclone Nargis. Rather, the staffing plan should place
    special emphasis on improving communication and coordination between Finance,
    Program, and Operations staff.

Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   vi
Summary
    WVM would benefit from strengthening its accountability procedures, and could
    complement previous training provided on logframe development by building the
    capacity of CNRP staff to facilitate participatory project monitoring. In addition,
    effective monitoring of the CNRP should be supported by Program unit staff that are
    trained and knowledgeable on the logframe, indicators and means of monitoring their
    sectors. At the same time, Program unit staff must have the necessary skills to ensure
    that means of monitoring indicators are appropriate for changing conditions at the
    community level.

    The CNER Program benefited from WVM’s knowledge of and close working relationship
    with many beneficiary communities. However, in other communities in which it had not
    worked prior to Cyclone Nargis, WVM rely heavily on village authorities to manage
    beneficiary selection and distribution of assistance. To remedy this, WVM should make a
    concerted effort to involve a wider range of traditional leaders, community based
    organizations, and prospective beneficiaries in the targeting of rehabilitation and longer-
    term recovery programs.

    Through the CNER Program, WVM staffed gained an appreciation for the importance of
    flexibility in emergency program design, logistics, operations management, financial
    procedures and implementation. In order to build on these lessons, the CNRP should
    look for ways to improve the flexibility of logistical systems, implementation methods,
    financial procedures and reporting methods, while maintaining an overall focus on
    achievement of strategic rehabilitation/recovery objectives.

    The Real-Time Learning Event convened by WVM in August 2008 provided a valuable
    opportunity for the organization to process lessons learned during the CNER Program.
    Eight months later, the CNRP program would benefit from holding a similar learning
    event that clearly identifies successful approaches to rehabilitation programming,
    needed improvements in efficiency and technical capacity, and strategic priorities for
    the remaining life of the program.




Cyclone Nargis Emergency Response (CNER) Program – End of Program (EOP) Evaluation-Executive   vii
Summary