FIELD RESEARCH METHODOLOGY

Global Environment Facility August 15th 2003 THE NATURE AND ROLE OF LOCAL BENEFITS IN GEF PROGRAM AREAS GEF OFFICE OF MONITORING AND EVALUATION METHODOLOGY David Todd, John Soussan and Lee Risby Study Document Number Thirteen This report has been prepared for the Office of Monitoring and Evaluation of the Global Environment Facility (GEFME). The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the GEF Secretariat, Implementing and Executing Agencies, donors, or Governments. 1 ACRONYMS CIDA CSO DAC Danida D.C. DFID GEF IAs IFAD M&E NGO OECD OED UNDP UNEP UNIDO UNICEF Canadian International Development Agency Civil Society Organization Development Assistance Committee Danish International Development Agency District of Columbia Department for International Development (of the United Kingdom) Global Environment Facility Implementing Agencies International Fund for Agricultural Development Monitoring and Evaluation Non-Governmental Organization Organization for Economic Cooperation and Development Operations Evaluation Department of the World Bank United Nations Development Program United Nations Environment Program United Nations Industrial Development Organization United Nations Children‘s Fund 2 INTRODUCTION This document focuses on the methodology of the GEF Office of Monitoring and Evaluation Study of the Nature and Role of Local Benefits in GEF Program Areas. The detailed implementation of the study is presented in a separate companion document, Study Document Number Fourteen, Inception Report. The current document and its companion Inception Report have taken account of detailed comments and suggestions from a number of parties. Particular thanks are due to: CIDA: Ellen Hagerman, Valerie Young UNDP: Juha Uitto, Jyotsna Puri World Bank: Samuel Wedderburn UNEP: Yoshiyuki Yoichi, Mark Zimsky GEFSEC: Kanta Kumari, Yasemin Biro GEF M&E: Jarle Harstad, Aaron Zazueta, Claudio Volonte Special Adviser: Michael Cernea. 3 CONTENTS Page Acronyms Introduction Section One: Section Two: Objectives, Audiences and Purposes of the Study The Conceptual Framework of the Study: Linkages Between Local and Global Environment Benefits 2 3 5 8 11 12 19 21 Section Three: Application of the GEF Evaluation Criteria Section Four: Overview of the Study Methodology and Timetable Section Five: Section Six: Knowledge Sharing Sample of Projects Annex One: Annex Two: GEF Focal Area Strategies Focal Area Global Environment Benefit Indictors 24 26 30 40 Annex Three: The Case Study Method Annex Three: Ethical Guidelines 4 SECTION ONE: OBJECTIVES, AUDIENCES AND PURPOSES OF THE STUDY 1. This study will analyze how attaining global environmental goals can contribute to the generation of local benefits and how local benefits can contribute to the attainment of global environmental goals, in accordance with the GEF mandate and focal area strategies1. 2: Specifically, the study will explore the following dimensions of selected projects in the GEF portfolio:  The nature of links between attaining global environmental benefits and generating local benefits. This will be based on an analysis of how global environmental benefits can affect benefit streams at the level of project area communities and how the generation of local benefits can affect global environmental goals. Global environmental benefits of the projects will be assessed in relation to specific project objectives, outputs and monitoring indicators2 in the context of applicable GEF focal area strategies and programs; and based on available environmental baseline and monitoring data held by the project3, including any reviews undertaken. The analysis may include the use of existing indicators developed by GEF M&E which will be used to judge progress towards global environmental objectives. The types and scale of local benefits and of any negative impacts, intended or unintended, which have resulted from GEF projects, including local perceptions of these impacts. The extent to which project design and the environmental management options selected in the project can maximize opportunities to generate greater levels of local benefits, or can miss out on or not sufficiently exploit such opportunities. Essentially, this implies taking stock of good project practices and identifying existing constraints, weaknesses and lessons for improving future projects.    3. The reason for examining these issues (study objective) is to assist in increasing the long term sustainability of global benefits in sensitive areas by enhancing the level of direct and tangible gains accruing to local communities and actors in future GEF policy, strategies and programs, within the requirements of the GEF mandate. 4. Audiences. Achieving these objectives responds to the interests and concerns of several audiences, primary and secondary, which will benefit from the review. The 1 2 See Annex 1 for focal area objectives. Using project indicators for the Biodiversity, Climate Change and International Waters focal areas. See Annex 2 for some examples. 3 It will not be possible for the study to carry out a comprehensive assessment of the project‘s performance against its environmental objectives and field case studies will largely rely on the ‗quality and quantity‘ of national and local environmental data held by project monitoring systems. 5 present review adopts a broad definition of the audience it addresses. These include not only the management and staff of GEF and its implementing agencies, the World Bank, UNDP and UNEP, but also a broad array of other stakeholders, and the world‘s environmental agencies and activists at large. Local communities within project areas are primary stakeholders, but so are distant beneficiaries of global environmental goods and services, whose interests are served by ensuring an increased sustainability of these goods. It is expected that governments (primarily government agencies in developing countries), civil society organizations and NGOs, will be able to derive lessons for effective policy and practice from this review. Building up such audiences will also help increase international cooperation around these issues, as mandated for GEF by the Instrument for the Establishment of the Restructured Global Environment Facility (1996:6): “The GEF shall operate, on the basis of collaboration and partnership among the Implementing Agencies, as a mechanism for international cooperation for the purpose of providing new and additional grant and concessional funding to meet the agreed incremental costs of measures to achieve agreed global environmental benefits…”. 5. The GEF mandate incorporates the role of local benefits through its emphasis on sustainable development: “The GEF shall fund programs and projects which are country-driven and based on national priorities designed to support sustainable development” (P6). 6. In this study, the concept of local benefits is defined as including those project outcomes which, directly or indirectly, have positive impacts upon people and ecosystems within or adjacent to project areas, and which provide tangible gains, present and future, in the livelihoods of communities and to the integrity of ecosystems4. A Glossary of terms relevant to the study and its livelihoods-based approach is provided in the separate Inception Report. 7. The review will also examine local costs, and this will require identifying possible limitations and adverse effects, and recommending corrective measures. 8. Poverty reduction. Today Governments, multilateral and bilateral donor agencies, and specifically the World Bank and UNDP place poverty reduction at the center of their development agenda. In this context, this review will illuminate important ways in which more ―poverty reduction mileage‖ can be gained directly through deliberate benefits to poor local communities that contribute to special projects for global environmental management. This directly follows recent guidance given in a paper jointly prepared by the World Bank, UNDP, the European Commission and DfID on linking poverty reduction and environmental management, which states; “environmental management cannot be treated separately from other development concerns. It must be integrated into poverty reduction and sustainable development efforts in order to achieve significant and lasting results.” 9. In sum, the overall purposes of the study are to contribute towards: 4 As measured by project monitoring systems. 6  Enhancing GEF policies, strategies and project design and implementation, in order to fully realize the potential for local gains in global environmental programs, to mobilize local actors for long term support to sound environmental management, to reduce local costs incurred by local communities for supplying global environmental goods, and to ameliorate possible negative impacts. Strengthening GEF M&E policies and processes to identify indicators for and strengthen the tracking of local benefits and negative impacts. Expanding the body of existing operational knowledge about good practices and experiences germane to pursuing global environmental issues, and of constraints or fallacies to be avoided in operations. Disseminating widely—through a participatory review process in a number of countries, and through post-review education and awareness raising – the most valuable lessons of existing experience, and showing how these lessons can be implemented in future GEF, UNDP and UNEP operations This review will benefit the world‘s environmental community at large, empowering it to be more effective both globally and locally, and help gain more enduring support from local actors and communities residing in the world‘s most sensitive and vulnerable ecological areas.    7 SECTION TWO: THE CONCEPTUAL FRAMEWORK OF THE STUDY: LINKAGES BETWEEN LOCAL BENEFITS AND GLOBAL ENVIRONMENT BENEFITS 10. The Study Inception Report gives a detailed overview of the conceptual framework for the study. The main elements of this framework are a typology of local benefits, an identification of the ways that local benefits can enhance global environmental benefits and a model that links both local and global benefits to the dynamics of local people‘s livelihoods. The framework is depicted in Figure 1 (Page 10). 11. The typology identifies five generic categories of improvement to livelihood capital, which can be seen as the core of local benefits in global environmental projects:  Improved access to natural capital, including plants and animals harvested from the local resource base, surface and ground water, fuelwood and environmental services such as safe waste disposal, clean water, diseaseprevention, etc. Such changes will increase global environmental benefits, reflected in factors such as the reversal of ecosystems deterioration, retained biodiversity values, the regeneration of forests, rangelands and wetlands and improvements to water quality. Costs to local communities resulting from restrictive uses, and related remedies, will be considered. Improved social capital and institutional capacities in local communities. This reflects the enhancement of community-level institutional capacities and contact networks and improved ability in local communities to deal with outside agencies. It also reflects gender-sensitive improvements in social equity at the local level, especially through the empowerment of women and minority groups in decision-making. Improvements to physical capital, including investments in tools and machinary, access to or ownership of buildings and access to infrastructure such as transport, telecommunications or water supply and irrigation. Improvements to human capital which include skills, knowledge, work ability and management capabilities of local community members. There is typically a need for a particularly strong gender focus in this area, in order to assess changes in such dimensions as the functional literacy and management skills of women, compared with those of men. The cumulative outcomes of the above four forms of capitals are to be identified in increased livelihood opportunities and incomes. This includes higher productivity of existing activities and new opportunities for farming, fishing or small businesses, increases in cash income and improvements to the ability to save, or access to micro-capital.     8 12. Increases in the livelihood capitals available to communities will promote improved health and food security, including improvements to key indicators such as child and infant mortality, reduced morbidity from diseases that reflect poor environmental conditions and improvements to both the absolute level of nutrition and a balanced diet. 13. Strengthened livelihood capitals and improved health and food security will, in turn increase the resilience of local communities to withstand shocks from external factors that are beyond their effective control. Increased resilience promotes reduced vulnerability to, for example, natural disasters such as floods, droughts and cyclones, environmental degradation, loss of ecosystem integrity, deforestration and climate change and variability as well as to such forces as social, political and market disruption. 14. These patterns of increased livelihood capitals (local benefits) can be linked to enhanced global environmental benefits in four specific ways:  Changes to consumption patterns such as switching to renewable energy sources, changes in diet or the acquisition of more sustainable consumer goods. Capabilities to move to more sustainable patterns of consumption are closely linked to the reduction of vulnerabilities and the overall prosperity and security of different stakeholder groups. Improvements to the local resource base, as global environmental processes in areas such as the hydrological cycle, land degradation and atmospheric pollution are in large measure the accumulation of local resource dynamics over larger areas. Anything that enhances the quality and sustainability of local level resource management has positive global consequences. The reduction of vulnerabilities is important both directly and indirectly. Reduced vulnerability to hazards such as droughts, floods and storms at a local level is a direct global environmental benefit as it means that global processes such as climate change and variability are less of a threat to vulnerable people. Reduced vulnerability and increased household resilience also means that people will be less risk aversive and more maximizing in their livelihood strategies, which in turn reduces pressures on the resource base and generates greater global environmental benefits. Changes to the external institutional environment is also an important consequence. In particular the development of better governance as a consequence of local level empowerment and greater public awareness and political support for environmental issues will potentially foster changes in the balance of priorities as the urgency of poverty reduction and development pressures is reduced. This will create greater scope for direct environmental priorities in national policies.    15. Taken together, these four links between local and global benefit flows can mean that a focus on local benefits linked to local livelihood dynamics will not just generate gains in poverty reduction. They can also directly and materially contribute to the attainment of global environmental goals. 9 FIGURE 1: A MODEL OF LIVELIHOODS AND BENEFITS FLOWS IN GLOBAL ENVIRONMENTAL PROGRAMS Global Environmental Processes Atmosphere and Climate Hydrological Cycle Ecological Processes Biodiversity Local Resource Base Entitlements/Access Vulnerability Context Natural disasters Ecosystem deterioration Market failure Climate variability pollution ill health, etc R E S I L I E N C E Natural Physical Livelihood Capital Social Human Financial External Institutions (societal, policy, legal, institutional & economic context) Local Institutions Livelihood Activities Farming, Labour CPR utilisation Non-farm income enterprises Household maintenance Flows affecting local benefits Flows affecting global benefits Patterns of Consumption Outcome well being:opportunity security, empowerment The model is adapted from a livelihoods model developed as part of a DFID-funded research project on policy-livelihood relationships in South Asia led by SEI. The flows affecting local benefits (black arrows) that result from global environmental programs relate to the dynamics of local livelihoods. Changes to local benefits will enhance livelihoods and can generate additional flows of global environmental benefits through changes to more sustainable patterns of consumption, reductions to vulnerability factors that affect environmental integrity, enhancements to local resources that multiply up to have global significance and changes to institutional processes that bring global environmental benefits 10 SECTION THREE: APPLICATION OF THE GEF EVALUATION CRITERIA 16. Generic criteria for evaluating activities in the development field have been established by the Working Party on Aid Evaluation of the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (OECD). The GEF Office of Monitoring and Evaluation specifies the five DAC criteria as5: Impact: measures both the positive and negative, foreseen and unforeseen, changes to and effects on society caused by the project(s) or program(s) under evaluation. Effectiveness: measures the extent to which the objective has been achieved or the likelihood that it will be achieved. Efficiency: assesses the outputs in relation to inputs, looking at costs, implementing time, and economic and financial results. Relevance: gauges the degree to which the project or program at a given time is justified within the global and national/local environment and development priorities. Sustainability: measures the extent to which benefits continue from a particular project or program after GEF assistance/external assistance has come to an end. 17. These criteria will be applied to the analysis of local benefits and (negative impacts) of GEF activities. However, care will be needed in the application of these criteria, in view of the fact that in the GEF portfolio, the generation of local benefits is always likely to be a secondary objective of the activity, whose primary objective is the stimulation of global environment benefits. 5 ―Monitoring and Evaluation Policies and Procedures‖, The GEF, Washington D.C., 2002, Page 8. 11 SECTION FOUR: OVERVIEW OF THE STUDY METHODOLOGY AND TIMETABLE Context of the study 18. The GEF has provided nearly US$ 4.1 billion in grants for over 1200 projects during the last 10 years of its operations. A preliminary desk analysis of documents concerning 220 projects in the GEF portfolio showed that slightly more than half the projects indicated an intention or expectation of providing local benefits. Enhancing local benefits is a strategic premise towards building enduring sustainability of the projects‘ outcomes related to global environmental gains. 19. To date GEF evaluations and reviews of Biodiversity (GEF, 2000a), Climate Change (GEF, 2000b) and International Waters (GEF, 2001) have primarily focused on identifying impacts that produce global environmental benefits in accordance with the GEF Mandate (GEF, 1996a). No studies have had a primary focus on the assessment of local benefits and impacts, including the factors that facilitate or hinder them. 20. Building on the important knowledge generated by these prior studies, the present review will carry GEF research further into the important, yet unexplored, terrain of community level benefits emanating from interventions for the global good. 21. To address this complex new area, the GEF Office of M&E and the review team have focused first on evolving a comprehensive methodology, covering 10 years of GEF project experiences both in breath and depth, aiming also to involve a number of beneficiary countries in the review process. 22. The methodology for the review was designed and evolved in cooperation with implementing agencies, through close interaction. It incorporates suggestions received during the May 2003 workshop on the first draft methodology and the comments on a second draft circulated at the beginning of July. 23. A separate Inception Report (Study Document Number Fourteen) gives more details of the theoretical model, which underpins this Methodology and of the Study Work Plan. 12 24. Main phases. The building blocks of the evaluation framework are presented below, to provide a ‖bird‘s eye‖ view on the review‘s approaches and unfolding, while details on each block follow later. These framework components rely on a set of tools and analytical activities, which are carried out in sequence, with three main phases: Phase One: Preliminary stock-taking and portfolio analysis, including consultations among the review‘s supporting agencies and GEF implementing agencies. Analytical and empirical field work on a relevant set of selected Projects, and consultations in benefiting countries. Phase Two: Phase Three: Overall analysis of findings, preliminary report writing, country consultations on drafts and recommendations, and Final Review Report completion. 25. Within each phase, the research, analytical and writing activities will be staggered along a time bound work plan, to maximize the use of existing limited resources, and to enable the flexible adjustment of each phase‘s calendar as the work progresses. 26. For these three phases, the methodological approaches and tools are the following: a. Conceptual Evaluation Framework The Office of M&E and review team have elaborated, discussed, and finalized a conceptual/analytical framework for the evaluation exercise. It defines: the dialectic relationships between global and local benefits of environmental interventions; the potentials embedded in this relationship; the costs and trade-offs between environmental management and safeguards; and a number of other premises that will guide the review‘s analytical work. This conceptual framework also sets out the current thinking on the linkages between poverty reduction and environmental management. Overall, it specifies the researchable issues and questions, the key performance indicators, data-sources and data generation techniques. The Typology of Local Benefits To ensure interpretative unity throughout the review and across the 20 field research studies, the review team has constructed a typology of local benefits. Specific local benefits inhabit a wide spectrum, given the variety of projects, sectors and involved populations. Therefore, classifying and grouping ex-ante these possible benefits into a basic typology aims to facilitate on-the-ground benefit identification, measurability and comparability of observed impacts, between sectors, and geographically across countries. This typology tailors the analysis to distinctly defined b. 13 program areas and will be the basis for analyzing local benefits in the field. The typology will also be informed by, and in turn will infuse added analytical power into, the stakeholders‘ analysis. The latter will help assess whether the interests of particularly vulnerable groups, including indigenous people, women, populations with only customary (not formal or legal) ownership of land and trees, are considered. c. Issue Identification The review is focused upon issues that have surfaced during the design and actual implementation of GEF projects. It thus responds to real and major policy and operational needs, particularly to concerns that have been frequently expressed, both by local communities and by local authorities and organizations. Such questions can be summarized straightforwardly into one: “What is in it (this project) for us?” To best identify such questions and issues, the GEF M&E has also started a preliminary review of the portfolio (see also Point 4 below ) during 2002, plus consultations with other donors and agencies and with individual staff members. These consultations were held to learn about and select topical issues for this review, concentrating on the theme of local community benefits. On the other hand, the issue identification and selection also aimed to avoid spreading the review too thinly over many issues. d. Desk Reviews for Preliminary Assessment It was realized early that, before defining the focus and selecting the sites for field work review, it is important and sound to take stock of the entire portfolio in terms specific to this review, namely the presence of provisions directed at the local communities in the design and the financing of projects. A substantial, highly labor intensive effort was undertaken by GEF M&E to carry out such a comprehensive desk review of the portfolio (covering a total of some 220 projects) during 2002 and the first half of 2003. The desk review has yielded several studies (all distributed in first or second version to the review co-sponsors). Their principal methodological and substantive contribution to the review is the preliminary assessment of community issues and of broad trends (what are these?) in project performance for addressing them. The informed initial assessment generated through desk studies will serve as a ―knowledge platform‖ from which more in-depth, field-based case studies will be initiated in the next phase of the review. The findings of the desk analyses will continue to be refined in the Review‘s 2nd phase. Methodologically, they will also be used for constructing the macro-image (for the portfolio) indispensable for 14 the Final Review Report, which cannot be produced in any other way, while field empirical analyses will provide rich micro-data. The desk studies were presented at a Workshop in May 2003 and their discussion by workshop participants has contributed to decision-making on further field work methodology and for balanced and informed site selection for empirical inquiry. e. Field Case Studies and Individualized TORs. The prior steps and activities have prepared the ground for proceeding to field empirical analyses. After iterative pre-selection rounds, a total of twenty projects were selected by GEF M&E, in consensus with all review sponsors, for field work review. These are spread out over more than 15 countries. Extreme care was exercised in selecting cases that will ensure coverage of a variety of sectors and approaches, and of population segments such as indigenous populations, coastal and mountain communities and others. The detailed description of the methods for carrying out field studies is given in Annex Three of this document. To refine and specify the field research techniques in line with the different content of case-studies in various sectors, individual TORs will be elaborated for guiding the analyst/consultants for each project. This phase of the study will also employ baselines (where present) that were constructed during project preparation. This has the advantage that baselines usually have detailed quantitative data which may be used fruitfully by the project team to understand project impacts and to understand the project on the ground. f. Desk Case Studies Complementing the 20 field case studies, an additional number of projectcases (approximately 30) will be subjected to in-depth desk case-by-case study, based on detailed project files, supervision reports, ICRs, other available literature and interviews with task and project managers Field Work Syntheses At completion of the field work phase, the findings of all case studies will be synthesized in issues-papers and draft sections for the final report. These will be then discussed by the Review‘s team and Advisory Committee in special session (s). g. 15 h. Stakeholders’ Analysis. In all these studies, the Review‘s analytical ―lens‖ will be informed by a stakeholders‘ perspective. It is essential to explore impacts, costs, and benefits from the various viewpoints of the local communities. Communities are not homogeneous, and interest varies among subgroups even in the poorest strata. This is why social and gender differentiation will be a cross-cutting principle throughout. Consultations with Countries and Participatory Analysis Linked to and germane to stakeholder analysis, another feature of the entire review methodology is and will remain the consultation of relevant agencies and specialists from beneficiary countries, including GEF and other agencies‘ operational staff. This is and will be further accomplished through country workshops and briefings in each of the countries where field work is carried out. These workshops will be conducted in the prefinal stage, with groups of countries or specialists, as allowed by financial resources . Discussions in the these workshops will center around drafts of the final review report. Consultations at community levels will be an integral part of the methodology of each field case study. The purpose of the overall effort towards employing participatory approaches in analysis is twofold: To increase the accuracy of the Review‘s findings; and To build ownership around the Review in participating countries, and enhance local environmental education and policy dissemination. Finally, an international workshop will be organized around the Review‘s final report, as part of the communication strategy. i. j. Focus Groups. The design and content of the GEF projects under review, as well as their implementation processes and the lessons to be learned, depend to a very high extent on the staff-specialists responsible for preparing, designing, appraising, writing-up, supervising and implementing these projects. It is therefore important to harvest their rich knowledge and experiences and to gain an in-depth insight into how they directly perceive difficulties and challenges of these projects. This will also help fine-tune the Review Report to operational demands and realities. The Review team will commission two or three focus groups, moderated by a focus-group specialist, to generate staff views in an interactive and open format. This will be more time-effective and contentrich than resorting to lengthy written survey questionnaires. h. External Expert Interviews. Project case study data will be supplemented and placed in a broader context of implementation experience through a series of expert interviews with bilateral, multilateral organizations, international NGOs and academics working in GEF focal areas. 16 i. Other data such as baseline data or data available from the country, such as GIS data on protected areas in the Biodiversity focal area and survey data j. Final Report The product of the entire exercise, the final Review Report, will be submitted to the management bodies of the participating agencies, and will be made publicly available, together with other review documents, on a web site. The final report will include a chapter that will use the methodology outlined above and will derive also methodological lessons for further GEF and similar reviews. 27. Broadly speaking, the nine elements described above correspond to the three phases of the review with some partial and necessary overlaps, as follows: Phase One: Instruments and activities described as points (a) to (d) above (implemented 07/2002 to 07/2003) Phase Two: Instruments and activities described as points (e) to (i) above (implemented 08/2003 to 06/2004) Phase Three: Activities to generate final products, described as point (j) above (implemented 07/2004 to12/2004) Matters under consideration. The vast scope of the present review and the diversity of sites necessitates continuous consideration by the Advisory Committee of the feedback arriving from field work and of the effectiveness of various tools and approaches. It is likely that some adjustments, additions or simplifications will be necessary as the review advances, in a flexible manner. Among the matters identified, and still to be considered and decided upon, are, for instance, a desk-review of the financing of GEF projects, to analyze patterns of resource allocations within project-budgets and distribution of envisaged benefits. Similarly, it is clear that the review‘s final findings will need to be widely disseminated in the post-review period. To this end, while communication about the review itself is going out to countries starting from Phase one, a communication plan for the post-review period, centered on disseminating findings and recommendations, will be formulated during phase two, before the final report is written. 17 Participation from a number of agencies in the review will include their direct engagement the field case-studies. As these partners inform the GEF M&E of their ability to send their specialists to the field, this will be incorporated in the review‘s logistic work plan (see Timetable below and separate Inception Report). Timetable 28. The time schedule for the study can be planned in a preliminary manner now, to be adjusted when all institutional contacts with countries and cases are made. It will be a complex logistical exercise to complete twenty field studies in more than ten countries, with six different field team leaders. A detailed schematic work plan is provided in a separate Inception Report. Figure 2: Time Schedule of Case Studies and Analysis Task/Month (03) M J J A S O N D (04) J F M A M J J A S O Selection of cases *********** ************* and preparation First set of field studies Non-Field Studies Second set of field studies Analysis, findings, recommendations N D (05) J **** ********* ********* **** *********** ** ******* *** ************** ***** ************* 18 SECTION FIVE: KNOWLEDGE SHARING Knowledge Sharing 29. The study will share the knowledge it generates and obtain feedback, through a range of workshops and reports, related to its phases. These will be further developed in the light of study findings and in response to suggestions from within and outside the Study Team. Documents will be placed on a project-specific web-site, initially as a link from the general GEFSEC site. This will enable a range of stakeholders and external parties to access and respond to project outputs. Documents to be placed on the website will include;  Revised versions of reports submitted to May 2003 international workshop; o Desk Review of the GEF portfolio o Desk Review of international experience of local benefits of environmental projects and programs (incorporating inputs from Implementing Agencies o Revised fieldwork methodology   Summaries of national workshops, held at the end of each case study to present and discuss key findings Draft Final Report (including detailed fieldwork-based case studies) on the main findings of the review, lessons learned and key recommendations, based on the ten detailed and ten secondary case studies and on the non-field reviews. Summary of international workshop held at end of fieldwork phase; with participants from the study team, other members of GEF and IAs, and the study Advisory Group, to discuss the draft Final Report Final Report, responding to comments from study stakeholders Follow-up products, such as lessons notes and good practice toolkits Integration of desk review database (with provision for regular updating) into GEF Secretariat database to facilitate internal knowledge management and future evaluation. This will provide a basis for the integration of the tracking of participation and local benefits into the overall GEF data management system.     19 SECTION SIX: SAMPLE OF PROJECTS 30. Table 1 details ten proposed projects each for primary and secondary fieldwork (See Annex Three for details of the Case Study Methodology). The projects have been selected based on information reviewed during the desk study and also take into account comments received from parties who attended the May 2003 workshop. The selection provides for representation according to:     Operational programs across Biodiversity, Climate Change and International Waters Focal Areas; Regional distribution; Implementing Agency; Project maturity (both completed and projects under implementation) Local Benefits Review Projects Proposed for Non-Field Reviews 31. Table 2 details twenty-six projects for non-field reviews. The projects are selected based on information reviewed during the desk study and also take into account comments received from parties who attended the May 2003 workshop. The selection takes account of:    Availability, quality and relevance of implementation documentation. Project maturity. Projects selected are predominantly those which are completed or have been under implementation for more than two years and therefore have demonstrated a potential to provide substantive information. Operation programs across Biodiversity, Climate Change and International Waters Focal Areas. 20 TABLE 1: PROJECTS FOR PRIMARY AND SECONDARY FIELDWORK Primary Fieldwork Project(s) 1. Kenya: Lewa Conservancy 2. Belize: Conservation and Sustainable Use of the Barrier Reef Complex 3. Romania: Danube Delta Biodiversity 4. Ghana: Natural Resource Management 5. Nepal: Upper Mustang Biodiversity Conservation 6. India: Optimizing Small Hydel Resources in Hilly Regions 7. Philippines: Palawan New and Renewable Energy and Livelihood Support Project 8. Senegal: Sustainable and Participatory Energy Management 9. Regional: Implementation of the Strategic Action Program (SAP) for the Red Sea 10. 10. Regional: Strategic Action Plan for Bermejo River FA BD BD BD BD BD CC CC CC IW IW OP 1 2 2 3 4 6 6 STRM 9 9 Project Type MSP Full Full Full MSP Full MSP Full Full Full IA Bank UNDP Bank Bank UNDP UNDP UNDP Bank Bank UNEP PAIRED SECONDARY FIELDWORK PROJECT(S) Tanzania: Jozani-Chwaka National Park Conservation (OP3) (UNDP) Belize: Community-Co Managed Protected Areas (OP3) (UNDP) None Ghana: Renewable Energy-Based Electricity for Rural, Social and Economic Development (OP6) (UNDP) Pakistan: Fuel Efficiency in the Road Transport Sector (OP5) (UNDP) INone Philippines: Conservation of the Tubbhatata Reefs National Park (UNDP) (OP2) (UNDP) None Yemen: Protected Areas Management (OP1) (Bank) Bolivia: Biodiversity Conservation / Achieving Sustainability (Biodiversity Sustainability Project) (OP3) (Bank) 21 TABLE 2: PROJECTS FOR NON-FIELD REVIEWS Project for Non – Field Review(s) 1. Kenya: Tana River Conservation 2. Kenya: Lake Baringo Community-based Management 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. Global: People Land Management and Environmental Change Bangladesh: Aquatic Biodiversity Conservation Algeria: El Kala National Park and Wetlands Management Ghana: Coastal Wetlands Management Lao: Wildlife and Protected Area Management Sri Lanka: Development of Wildlife Conservation and Protected Area Management Micronesia: Community Conservation and Compatible Enterprise Development Guyana: Program for Sustainable Forestry: Iwokrama Rainforest Ecuador: Biodiversity Protection India: Ecodevelopment Uganda: Bwindi-Mgahinga National Park Conservation Project Pakistan: Mountain Areas Conservancy Nepal: Biodiversity Conservation Philippines: Conservation of Priority Protected Areas Argentina: Biodiversity Conservation Project Indonesia: Solar Home Systems Uganda: Photo-voltaic Pilot Project for Rural Electrification Zimbabwe: Photo-voltaics for Household and Community Use Mali: Household Energy Tunisia: Solar Water Heating Guatemala: Renewable Energy for Small Enterprises in El Quiche Sudan: Community-based Rangeland Rehabilitation for Carbon Sequestration and Biodiversity Regional: Pollution Control and Other Measures to Protect Biodiversity in Lake Tanganyika Regional: Building Partnerships in Environmental Protection / Management of the East Asian Seas Regional: Lake Victoria Environmental Management Project FA BD BD BD BD BD BD BD BD BD BD BD BD BD BD BD BD BD CC CC CC CC CC CC CC IW IW IW OP 1 1 1 2 2 2 3 3 3(2) 3 3 3 3 4 4 STRM STRM 6 6 6 6 6 6 STRM 8 9 9 Project Type Full MSP Full Full Full Full Full Full MSP Full Full Full Full Full Full Full Full Full Full Full Full Full MSP Full Full Full Full IA Bank UNEP UNEP Bank Bank Bank Bank UNDP UNDP UNDP Bank Bank Bank UNDP UNDP Bank Bank Bank UNDP UNDP Bank Bank UNDP UNDP UNDP UNDP Bank 22 ANNEX ONE: GEF FOCAL AREA STRATEGIES BIODIVERSITY. The strategy of the GEF-financed Biodiversity focal area is derived from the Convention of Biological Diversity (CBD): “… the conservation of biological diversity, the sustainable use of its components and the fair and equitable sharing of the benefits arising out of the utilizing of genetic resources, including by appropriate access to genetic resources and by appropriate access to genetic resources and by appropriate transfer of relevant technologies, taking into account all rights over those resources and to technologies, and by appropriate funding.”6 The main strategic considerations that guided GEF-financed biodiversity activities during the GEF1 and 2 replenishment periods7 to secure global environmental benefits8 are:      Integrating conservation and sustainable use of biodiversity within national and, as appropriate, sub-regional and regional sustainable development plans and policies Helping to protect and sustainably manage ecosystems through targeted and costeffective interventions Integrating efforts to achieve global benefits in other focal areas, where feasible, and in the cross-sectoral area of land degradation, primarily desertification and deforestation Developing portfolio that encompasses representative ecosystems of global significance Targeting and designing activities to help countries achieve agreed biodiversity objectives in strategic and cost-effective ways. CLIMATE CHANGE The strategic objective of GEF-financed CC programs is ―support sustainable measures that minimize climate change damage by reducing the risk, or the adverse effects of climate change.‖9 The GEF will finance agreed enabling, mitigation 10 and adaptation11 6 7 GEF (1996: 13) During the Pilot Phase the BD focal area was guided by several considerations not necessarily specific to biodiversity such as innovation and replication. See GEF (2001: 2 – 3) for more information on the evolution of the BD focal area and also GEF C.21/Inf.11. 8 GEF C.21/Inf.11 provides the ‗new‘ BD priorities that provide the strategic focus for GEF-3 replenishment period. 9 GEF (1996: 31) 10 Mitigation measures reduce or lead to the reduction of GHGs from anthropogenic sources or protect or enhance removal of such gases by sinks. 11 Adaptation activities minimize adverse effects and / or risks of climate change. Initially GEF has met the agreed full costs of relevant adaptation activities undertaken in the context of the formulation of national communications (Stage I activities as outlined by the COP of UNFCCC) 23 activities in eligible countries. Projects encompass interventions which encourage more efficient use of energy, sustainable transport technologies, reduce deforestation and sequester carbon or substitute fossil fuel energy generation with renewable energy technologies and adaptation, based on guidance received from the Conference of Parties of the UNFCCC. INTERNATIONAL WATERS The strategic objective of the International Waters focal area is to contribute primarily as a catalyst to the implementation of more comprehensive, ecosystem-based approach in managing international waters. GEF Council calls for a comprehensive approach to water resource management, an approach that is: “… Cross sectoral, integrates ecological and development needs, and is based on holistic analyses of carrying capacity of the water environment … The GEF will act as a catalyst to ensure that countries better understand the functioning of their international waters systems, gain an appreciation of how their sectoral activities influence the water environment, and find a means for collaborating with neighboring countries to collectively pursue effective solutions.”12 The main strategic thrust is to meet the incremental cost of:    Assisting countries to better understand the environmental concerns of their international waters Building capacity of existing institutions to utilize a more comprehensive approach for addressing transboundary water related environmental concerns Implementing measures that address the priority transboundary environmental concerns 12 GEF (1996) 24 ANNEX TWO: FOCAL AREA GLOBAL ENVIRONMENTAL BENEFIT INDICATORS Biodiversity Below are some examples of indictors used to monitor progress towards global environmental benefits in two GEF projects included in the project sample (see Inception Report). Jozani-Chwaka National Park Development (OP3) Objective Impact (final goal) The core conservation area is effective, productive, harmonized and sustainable Effects (intermediate goals) 1. The biodiversity of the Jozani-chwaka bay area is protected, managed and utilized sustainably Indicators Verification     Legal and sustainable use of the proposed national park use zones established Low level of illegal activities maintained by end of project Jozani population of red colobus monkey stable and increases Jozani population of ader‘s duiker decline halted and recovery promoted Baseline and end of project surveys Patrol records Baseline population surveys and monitoring Lewa Wildlife Conservancy (OP1) Objective Impact (final goal) conservation of threatened Black Rhino and Grevy‘s Zebra populations Effects (intermediate goals) 1. Protection of endangered species will be made more effective and efficient enabling the conservancy to continue to provide protection to endangered species on a sustainable basis. Indicators Verification    Increase in populations of threatened species within the conservancy and increased availability for relocation Increased land area maintained as high quality wildlife habitat and maintenance of key wildlife corridors Effective wildlife protection Baseline population surveys and regular monitoring Baseline vegetation surveys and regular monitoring Patrol records 25 Climate Change Examples of indicators for Climate Change projects Indicators for Solar Home Systems and Rural Energy Services Project Cluster INDICATORS 1. Energy production or savings and installed capacities 2. Costs per technology unit or measure installed 3. Businesses and supporting services development 4. Financing availability and mechanisms 5. Policy development Examples (a) Number of individual solar home systems installed (b) Capacity of off-grid village power supplies from mini-hydro, biomass, wind, and solar PV(MW) (a) Installed costs or lifecycle system costs of solar home systems. (b) Unit electricity costs of renewable-energy produced power relative to conventional power costs (i.e., from diesel generators). (a) Number of solar home system manufacturers, system assemblers, dealers, installers, and service firms (including firms where SHS is not the primary business). (b) Existence and appropriateness (to local needs) of equipment quality standards and certification procedures/institutions for equipment and installation. (a) Availability of consumer credit for purchase of solar home systems, including dealer-supplied credit, microfinance, and credit from development banks. (b) Number of financial institutions and volume of lending for off-grid village power. (a) Existence of rural electrification policies and/or plans that explicitly recognize and account for the role of renewable energy technologies in rural electrification. (b) Existence of working regulatory/social models for village power schemes, including tariffs, responsibilities for ownership and maintenance, and equity. (a) Awareness among rural households of benefits and costs of solar home systems. (b) Abilities of village leaders or project developers to implement and manage villagepower schemes. (a) Percentage of off-grid households receiving energy services from renewable energy sources relative to conventional sources (by income group or other social parameters). (b) Consumer satisfaction (by income group or other social parameters). 6. Awareness and understanding of technologies 7. Energy consumption, fuel-use patterns, and impacts on end-users Indicators for Grid-Connected Power Project Cluster Examples INDICATORS 1. Energy production or savings and installed capacities 2. Costs per technology unit or measure installed 3. Business and supporting services development 4. Financing availability and mechanisms (a) Installed capacity (MW) of wind, biomass, geothermal, small-hydro (b) Annual or cumulative production (MWh) (a) Installed costs per kW in selected GEF client countries for each technology (b) Levelized production costs (per kWh) in selected GEF client countries for each technology (a) Number of domestic commercial businesses that can manufacture and assemble, sell/install, and service/maintain technologies. (b) Existence of equipment quality standards and certification procedures for equipment and installation. (a) Amounts of government, commercial and bilateral/multilateral financing for gridconnected renewable energy generation facilities. (b) Enumeration and characteristics of financing programs dedicated to renewable energy technologies. 26 5. Policy development 6. Awareness and understanding of technologies 7. Energy consumption, fuel-use patterns, and impacts on end-users (a) Existence and characteristics of electric power regulation and policies for independent power producers, power-purchase agreements, transmission wheeling of generated power, and power dispatch requirements. (b) Fairness/equivalence of wholesale tariff structures for generation from renewable energy sources relative to those for conventional fuels. (a) Awareness amongst financiers, project developers, and utilities. (b) Knowledge base for technology application (i.e., wind resource maps, wind turbine siting experience, wind farm operations and maintenance experience). (a) Share of renewable-energy-based generation relative to total generation (% kWh). (b) number of equivalent beneficiaries (households) supplied by renewable energy generation (same as kWh share but translated into household equivalents). Indicators for Energy Efficiency Products and Manufacturing Cluster INDICATORS 1. Energy production or savings and installed capacities 2. Costs per technology unit or measure installed 3. Business and supporting services development 4. Financing availability and mechanisms 5. Policy development Examples (a) Cumulative or annual energy savings (i.e., MWh) from stream of energy-efficient manufactured products. (b) Electric power demand reductions (MW) from stream of energy-efficient manufactured products. (a) Market price of manufactured product (b) Lifecycle cost of manufactured product relative to that of conventional product (e.g., compact fluorescent lamps relative to conventional lighting) (a) The number of in-country commercial businesses that can manufacture and assemble, sell/install, and service manufactured product. (b) Existence of equipment quality standards and certification procedures for service personnel. (a) Availability of financing to manufacturers that serve target market to upgrade product designs and production processes for energy-efficient products. (b) Amount of relevant investment, if obtainable, by these manufacturers. (a) Existence of policies or regulations that have a significant influence (+/-) on demand and/or supply of manufactured product (efficiency standards, equipment labels, environmental emissions limits, import quotas, etc.). (a) Consumer or industry awareness of characteristics, costs and benefits of manufactured product. (b) Mentions of targeted technologies in communications media. (a) Market share of manufactured product (e.g., sales of efficient vs. normal boilers). (b) Penetration of manufactured product (e.g., share of households or industrial enterprises with efficient product). 6. Awareness and understanding of technologies 7. Energy consumption, fuel-use patterns, and impacts on end-users 27 Indicators for Energy-Efficiency in Industry Project Cluster INDICATORS 1. Energy production or savings and installed capacities 2. Costs per technology unit or measure installed 3. Business and supporting services development 4. Financing availability and mechanisms 5. Policy development Examples (a) Annual or cumulative energy savings (MWh) from energy efficiency investments in industry, either by industrial firms or by energy service companies. (b) Electric power capacity (MW) reduced through energy efficiency investments. (a) Rates of return achieved from energy-efficient investments in industry. (b) Costs of conserved energy (e.g., cents/kWh). (a) The number of operating energy service companies or other companies offering efficiency improvement equipment or services in target market. (b) Existence of ESCO trade association that includes standards of professional practice and measurement. (a) Availability of business financing for ESCOs. (b) Availability of lease financing and performance contract financing for energyrelated in-plant projects (c) Volume of financing in energy-efficiency investments from ESCOs or by industry. (a) Existence of regulatory/contracting frameworks that support ESCOs (i.e., policies supporting performance contracting). (b) Existence of policies creating incentives for industry to improve energy efficiency. (a) Awareness amongst industry of the benefits of energy efficiency investments and of potential contracting approaches with ESCOs (i.e., performance contracting). (b) Awareness and capability of ESCOs to make profitable investments in industry and to sustain a profitable business. (a) Energy-intensities of particular industrial sub-sectors, compared with past years and with baseline projections. 6. Awareness understanding technologies and of 7. Energy consumption, fuel-use patterns, and impacts on end-users International waters. Examples of environmental stress reduction indicators (by category of threat) 1. Trans-boundary Pollution 1.A. Excessive Nutrient Load  Reduction in fertilizer application rates kg/ha (N, P and K).  Increase in percentage of urban sewage that is treated  Increases in total and proportion of animal waste stored and spread properly 1.B. Persistent Organic Substances  Reduction in pesticide/herbicide use (weighted according to ecotoxicity) per hectare of farmland. 1.C. Other Contaminants  Reduction in area of water (in km2) affected by oil spills; reduction in total volume of oil spills.  Reduction in total pollutants from point sources by type and source.  Increase in completed investment projects to reduce point source pollution.  Basin-wide increase in market share of zero-P detergent.  Reduction in the amount of mine tailings containing heavy metals. 28 2. Overfishing  Elimination or reduction of the gap between actual fish catch and estimated maximum sustainable level of fish catch for modeled fish stocks.  Reduction of fishing capacity, measured by total number of vessels multiplied by estimated average catching capacity per vessel at full utilization, as a proportion of estimated fishing overcapacity.  Reduction of rate of by-catch of non-target species.  Increase in area (in km2) of no-fishing zones.  Increase in area of fishery with seasonal limits on fishing. 3. Habitat Loss/Destruction  Reduction in area of wetlands (in km2) converted to agriculture or other economic activities annually  Reduction in annual fishing trips by demersal trawlers.  Reduction in area fished (in km2) by demersal trawling.  Reduction in the area of mangroves (in km2) converted to mariculture or other economic activities annually.  Reduction in the annual rate of loss of coral reefs due to destructive fishing practices or other economic activities. 4. Excessive Water Withdrawals  Reduction in annual withdrawals of groundwater or surfaced water as % of available water (m3/m3)  Reduction in irrigation water used per hectare of irrigated farmland (m3/hec)  Reduction in water consumption per capita (m3/cap)  Reduction of the rate of volume extraction to volume recharged into the aquifer (m3/m3)  Increase in water subject to recycling and reuse schemes annually (m3/year) Land Degradation/Sedimentation  Increase in annual area of eroded land stabilized (in hectares/year)  Increase in annual area reforested (in km2/year)  Increase in percentage of land users/managers using agreed best practices  Reduction in anthropogenic sediment load to rivers/coastal areas Invasive Species  Increased proportion of ships visiting ports in demonstration sites following IMO ballast water management guidelines.  Reduction in number of invasive species in ballast water  Increase in number/percentage of ships conducting open ocean ballast flushing or using technologies designed to destroy invasive species 5. 6. 29 ANNEX THREE: THE CASE STUDY METHOD Introduction 1. In many of the projects to be examined, local benefits were not initially identified as a core outcome of the project. Furthermore, the anticipated relationship between local benefits and overall project effectiveness was often not specified in advance. The main objective of the case studies is therefore to identify lessons learned from project experiences, with regard to the generation of local benefits and to the contribution, which such benefits have made to the attainment of global environmental objectives. Where local benefits were explicitly targeted in the project design, the case studies will also have an element of accountability, in assessing the extent to which intended outcomes have been or are likely to be achieved. Types of Case Study 2.  There will be three types of case study: Full fledged field case studies that include a substantial program of fieldwork, supported by detailed investigation of all relevant documentation and detailed interviews with project staff, policy makers, counterparts in government, civil society representatives and other stakeholders. Case studies with limited fieldwork. These will be similar in scope to the full field case studies but will contain a less extensive range of both field research and interviews with other stakeholders. Desk in-depth case studies. These case studies will primarily focus on reviewing available documentation, supplemented by interviews with key stakeholders where this can add depth to the information available on specific issues. One of the key roles of these case studies is to clarify whether specific insights from the full case studies are reflected in a wider range of projects.   3. Twenty projects from the 127 reviewed in the desk study have provisionally been selected for field level review: ten for full field level study and a further ten for limited fieldwork. Details of the sampling method and case studies selected are given in Section Eight. The approach to the case studies will reflect the conceptual framework outlined in Section Two and presented in more detail in Annex Two. 30 Case Study Teams 4. In-country fieldwork will be conducted by case study teams, each of which will include at least:  A member of the GEF M&E or a consultant reporting directly to the Office, who will be Team Leader for that fieldwork component. The Team Leader should aim to spend about three weeks in country in the initial visit and return for one week to manage a concluding national workshop. One or more national consultants to the study. The budget allows for approximately twelve weeks of national consultant input per country. National consultants, with expertise in applied qualitative and participatory field research methods will be recruited through the most appropriate GEF networks, including the national Focal Points, Advisory Group members and international evaluation networks. In addition to fieldwork expertise, experience in the following areas will be important assets: environmental programs, evaluation, gender analysis, project management. It may not always be possible to recruit a consultant with all of the desirable characteristics, in which case the Case Study Team Leader will need to ensure that all aspects of the project are covered as well as possible, where appropriate through increased personal participation in the fieldwork. In some countries, the budget may permit recruiting more than one national consultant. In all cases, it will be important to ensure that the national consultant has no conflict of interest, such as current or recent engagement with the Implementing Agency, or with any other project stakeholder.  In addition, a fieldwork team may include:  One or more additional Study Team members, who may include representatives of Implementing Agencies, the Scientific and Technical Advisory panel (STAP) of the GEF and members of the Advisory Panel. Key Questions to be Answered by the Case Studies 5. The three types of case study will attempt to gather and analyze data to answer a same set of key questions. The depth and quality of information will be greatest for the full field-based case studies, but studies pursued through the other two methods will attempt to cover the same topics. 6. Key questions for all types of case study research, based on the Model of Livelihoods and Benefit Flows, are as follows: Questions concerning the five livelihood capitals:  Has the project led to improved (or worsened) access to natural capital, as evidenced by changes to entitlements and access to the local resource base for 31 different stakeholder groups and changes to the natural capital that households possess?  Has the project promoted increased (or reduced) livelihood opportunities, income and financial capital, as shown by the range and productivity of livelihood activities, the accumulation of household financial capital and improvements to consumption levels and the overall welfare of the households? Has the project led to improved (or worsened) social capital, equity and institutional capacities in local communities, as shown by changes to the local institutional base (and especially the promotion of greater equity within the community and enhancement of links to external institutions), the empowerment of women, the poor and minority groups and the growth of the social capital of key stakeholder groups? Has the intervention promoted improvements to (or deteriorations in) physical capital, as shown by the acquisition of new tools and technologies, investments in buildings, private land, infrastructure, water supplies, boats, vehicles and any other sort of physical asset that can be used in livelihood activities? Has the project led to improvements to (or deteriorations in) human capital, as shown by enhanced skills, knowledge, social capabilities and productive potential of all household members? Is there evidence of better nutrition, less illness and/or time saved from tasks such as gathering water, processing crops or other activities?    Derived questions, related to changes in livelihood capitals:  Is there evidence of reduced (or increased) vulnerability, for example, to natural disasters, environmental degradation and variability, social/political disruption or market disruption? Are there indications of improved (or worsened) sustainability of resource management, as reflected in improvements to the natural resource base; and, in particular, the reversal of resource degradation through reduction of forest clearance, over-exploitation of plant and animal stocks, pollution and soil erosion.  32 Case Study Analysis Framework 7. Based on the fieldwork (and/or secondary) data collected to explore questions listed in Section Six, the case study analysis framework will explore the following dimensions of the projects:  Overview of the investment: a brief profile of the project being evaluated, which describes the project policy and institutional context, structure, objectives and anticipated results (outputs, outcomes, impacts) and relates this to the host country‘s development context. It specifies intended local benefits and target groups based on existing documents and on interviews with stakeholders. Overview of Global Environmental objectives and achievements of the project: based on existing documents and interviews with expert stakeholders. Although this cannot be a major focus of the case study, the overview will provide the context necessary to understand the nature and role of local-global benefit interactions. It will include a brief review of the environmental resource characteristics of the area. The types and scale of local benefits and negative impacts identified, based on the Model of Livelihoods and Benefit Flows. The relationship of local benefits and/or negative impacts to the livelihoods of different stakeholder groups, based on the Model. This analysis will be differentiated by gender within each stakeholder group. Attention will be paid to vulnerable groups, such as indigenous people and scheduled castes, where these constitute a distinct group in the project area. The relationship of local level processes to wider social (including gender), economic and environmental processes. This will be reflected in the external institutions, global environmental processes and vulnerability context components of the model. The nature of the links between local level benefits/impacts and the attainment of global environmental benefits. The extent to which the environmental management options selected in the projects represent missed opportunities to generate greater levels of local benefits. Specific attention will be paid to opportunities for women, the poor and minority groups, as these are more likely to be overlooked in project design and implementation. Lessons to be learned from the project, including any relevant accountability issues.        33 Fieldwork Case Study Phases Preparation at Head Office level and recruitment of National Consultants 8. Case study preparation will begin at the Head Office of each Implementing Agency and the GEF. Staff responsible for, or with particular knowledge of, the project will be asked to collate documents available at Head Office and will be interviewed about the project. They will contact the Country Office to inform them about the purpose and methods of the assessment. GEF M&E will contact the GEF Focal Point in each fieldwork country, to initiate his/her collaboration with the field research. 9. Participating Country Offices will be asked to collate preliminary project documentation and, in liaison with the GEF Focal Point, to help make appropriate contacts in-country, including Government Officers responsible for the project and any non-government stakeholders. In-Country Initiating Meetings and Fieldwork Preparation 10. The fieldwork study will be launched in each country through means of initiating meetings with key stakeholders, including the GEF Focal Point, Government and CSO representatives, other donors and NGOs active in social, gender and environmental fields, the private sector, academics, environmental practitioners and representatives of vulnerable groups in the project area. These stakeholders will be asked to suggest data sources, key informants, comparable local projects and activities for additional field visits. 11. The fieldwork team will then move to the project area, conduct interviews with and collect relevant documents from, local stakeholders, including:         Project management and team members. Local government officers. Civil Society Organizations. Other donors, international or national bodies with environmental activities in the area. Representatives (formal or informal) of gender and vulnerable interest groups. Representatives (formal or informal) of the private sector. Any researchers (academic, government or consultants) working in the area. People affected by the project. 34 Local Work Plan and Methodology 12. On the basis of written and oral information collected and reviewed, the fieldwork team (international and national members) will define a detailed field study framework, including specific locations, work plan and methodology, based on their interpretation of the key questions in the specific situation of the project to be reviewed. The range of research methods likely to be adopted is illustrated in Table 1 below. Selection of sites to visit will avoid ―tarmac bias‖13 or the temptation of project staff to show only the most successful areas14. 13. The fieldwork study will utilize the key questions and analysis framework outlined above. Every effort will be made to involve country partners as fully as possible15. National consultants will use participatory methods16 to derive a comprehensive and legitimate understanding of the local social and political situation. The range of potential fieldwork methods for specific key questions is outlined in Tables 2 and 3 below. 14. Since this is a complex evaluation, with many team members investigating a range of potentially sensitive issues, all personnel will be asked to subscribe to the ―Code of Evaluation Ethics‖ presented in Annex Three. 15. The GEF has defined policies on public participation, but has no specific gender policy and a gender perspective has rarely been explicit in the design of GEF projects. 17 A similar position pertains with regard to vulnerable groups, including indigenous peoples. However, the Implementing Agencies do have relevant policies, which should apply with regard to implementation of projects, which are partly or wholly funded by the GEF. Care will be taken to ensure that the gender and equity impacts of each project are explored, according to the approaches outlined in Tables 2 and 3. 13 See, for example, ―Shortcut Methods of Gathering Social Information for Rural Development Projects‖, Robert Chambers in Michael M. Cernea, editor, Putting People First – Sociological Variables in Rural Development, New York, Second Edition, 1991, Page 518. 14 A common occurrence, recognized, for example in ―A UNICEF Guide to Monitoring and Evaluation – Making a Difference?‖ UNICEF Evaluation Office, New York, 1991, P85. 15 The importance of this practice has been shown as a growing international priority. ―Involving developing country partners in evaluation and lesson learning is seen as a major priority by many agencies, as most recognize that current practices fall far short of what is required for meaningful partnerships‖. From ―Evaluation Feedback for Effective Learning and Accountability‖, Evaluation and Aid Effectiveness Report No.5. OECD-DAC, Paris, 2001, Page 12. 16 Participatory research methods are highly developed and effective. However, their relationship to outsider-focused, mainly quantitative, ―expert‖ research methods remains controversial. The World Bank sponsored Poverty Assessments of the 1990s made a concerted effort to marry the two approaches. See, for example, ―Poverty in the Midst of Plenty – the Nigeria Poverty Assessment‖; World Bank, Washington D.C., 1996; Addison, D., Canagarajah, S., Montenegro, S., Todd, D., Woo. J., and Zanini, G. 17 See, for example, ―Consideration of women in evaluation‖ in Guidelines for the Evaluation of Projects and Programmes, Volume I/II – Policies and Procedures, UNIDO, Vienna, Austria, 2002, Pages 27-28. 35 16. After the international team members depart, the national consultant will undertake a series of follow up activities, according to the agreed local fieldwork methodology. 17. Key field research findings will be discussed with local stakeholders in focus group discussions in the final stage of the field research. Care will be taken to record local perceptions of and priorities for local benefits, along with specific information on changes to livelihood activities, consumption patterns and access to livelihood capitals resulting from the project. Project Study Report and National Workshop 18. On completion of the fieldwork, the local consultant will prepare a draft report of key findings. This will be reviewed and revised by the Team Leader of the case study, in consultation with the overall Study Manager. Key findings of the case study will be presented at a national workshop, at which selected national and local stakeholders will have the opportunity to review the findings and their interpretation. A revised case study report will be prepared based on the workshop outcomes. 36 Table 1: Range of Potential Field Research Methods Method 1. Case Studies Characteristics In depth review of one or a few selected instances of the research topic. Well suited for understanding processes and for formulating hypotheses. Literature, project documents, statistical data. Economic and efficient way of obtaining information. Difficult to assess the validity and reliability of secondary data. Flexible, in depth approach. Easy to implement. Risk of biased presentation/interpretation from informants and/or interviewer. Registration of quantifiable or classifiable data by means of analytical instrument. Precise, reliable and may require few human resources. Registers only facts, not explanations. Some Government data, if sound, may serve as a proxy for direct measurement. Involves inspection, field visits, and observation to understand processes, infrastructure/services and their utilization. Dependent on observer‘s understanding and interpretation. Low-cost, efficient. Direct contact with stakeholders. Needs good moderator to avoid manipulation, less suitable for sensitive issues. Variant of group discussions, used for analysis of specific, complex problems, to identify attitudes and priorities among small groups. Reasonable cost and efficiency. Good for production of ideas. Needs good moderator and participants. Oral interviews or self-completion questionnaires among a statistically representative sample of respondents. Demanding procedure, which can produce reliable and informative results on well-focused and discrete topics. Difficult to use for complex issues or where the overall population to be sampled is not accurately known. Involves quantitative surveys of small samples, selected by quasi-random procedures. Reasonably quick and cheap. Risk of sampling errors and biases. Not suited for accurate generalization. In depth observation over an extended period of time, participatory or non-participatory. Well-suited for understanding complex processes, but with limited potential for generalization, other than through the development of theories for further testing. Included in Study Yes. 2. Desk Review Yes 3. Key informant interviews 4. Direct Measurement Yes Possibly. (e.g. Government statistics on trading licenses, tax data, etc). Yes 5. Direct observation 6. Group discussions 7. Focus group discussions Yes Yes 8. Formal survey Possibly 9. Informal survey Probably 10. Observation No 37 Table 2: Fieldwork Questions on Local Benefits, Data, Methods, Gender and Vulnerability Question Has the project led to improved (or worsened) access to natural capital in local communities? Has the project promoted increased (or reduced) livelihood opportunities, income or financial capital? Has the project led to improved (or worsened) social capital, equity and institutional capacities in local communities? Has the project promoted improvements to (or deteriorations in) physical capital? Data Requirement Evidence of changes to entitlements and access to local resource base and of natural capital held by households. Range and productivity of livelihood activities; accumulation of household financial capital; improvements to consumption levels and overall household welfare. Evidence of growth of social capital, strengthened local organizations, greater links to external institutions, empowerment of women, vulnerable groups. Acquisition of new tools and technologies, investments in buildings, private land, infrastructure, water supplies, vehicles and other physical assets that can be used in livelihood activities. Evidence of enhanced skills, knowledge, and productive potential of community members, including better nutrition, less illness or time saved from manual tasks. Possible Research Methods Case studies of specific communities, document review, GIS, comparison of baseline data with new situation, semi-structured interviews, group discussions, small scale surveys. Case studies of specific communities, document review, financial data, comparison of baseline data with current situation, semi-structured interviews, group discussions, small scale surveys. Case studies of communities and organizations, document review, institutional records, comparison of baseline data with current situation, semi-structured interviews, group discussions. Case studies of communities and organizations, document review, institutional records, land registers, sales records, comparison of baseline data with current situation, semistructured interviews, group discussions. Case studies of communities, review of documents and records, comparison of baseline data with current situation, official health records, sales of cures, semistructured interviews, group discussions and small-scale surveys. Potential Vulnerability Issue Has the improved (or worsened) access to natural capital affected vulnerable groups in the locality in the same manner and degree as other social groups? If not, what major differences have there been? Have improved (or worsened) livelihood opportunities, income or financial capital affected vulnerable groups in the locality in the same manner and degree as other social groups? If not, what major differences have there been? Have vulnerable groups benefited from improvements in equity, strengthened local organizations or empowerment in the same manner and degree as other social groups? If not, what major differences have there been? Have vulnerable groups made gains in physical capital of the same order as those made by other social groups? If not, what major differences have there been? Potential Gender Issue Has the improved (or worsened) access to natural capital affected men and women in the same manner and degree? If not, what major differences have there been? Have the improved (or worsened) livelihood opportunities, income or financial capital been realized by men and women in the same manner and degree? If not, what major differences have there been? Have women benefited from improvements in equity, strengthened local organizations or empowerment in the same manner and degree as men? If not, what major differences have there been? Have women made gains in physical capital of the same order as those made by men? If not, what major differences have there been? Has the project led to improvements to (or deteriorations in) human capital? Has the human capital of vulnerable groups (including children) changed in the same way as that of other social groups? If not, what major differences have there been? Has the human capital of women changed in the same way as that of men. If not, what major differences have there been? 38 Table 3: Fieldwork Questions on Generation of Global Environmental Benefits from Local Benefits Question Have local benefits led to changes in consumption patterns (e.g. use of wood fuel or water) that influence global environmental benefits? Are more sustainable management patterns generating improvements to the resource base? Are reductions in vulnerability mitigating the effects of global environmental hazards such as climate change and biodiversity loss? Are improvements to local governance producing changes to external institutions and influencing environmental policies and awareness? Data Requirement Evidence of changes to consumption patterns and levels of exploitation of natural resources Possible Research Methods Case studies of different communities, documentary review, comparison of baseline data with new situation, semi-structured interviews, group discussions, small scale surveys. Case studies of specific communities, document review, GIS, resource assessment, comparison of baseline data with current situation, semistructured interviews, group discussions, small scale surveys. Case studies of communities, document review, assessment of climate and disasters data, comparison of baseline data with current situation, semi-structured interviews, group discussions. Case studies of communities and organizations, document review, institutional records, policy reviews, key informant interviews, comparison of baseline data with current situation, semi-structured interviews, group discussions. Potential Vulnerability Issue Have the changes in consumption patterns been made by vulnerable groups in the locality in the same manner and degree as other social groups? If not, what major differences have there been? Have changes to the quality of and access to the resource base reduced the vulnerability of poor people? Potential Gender Issue As many aspects of consumption are managed by women, what level of control over decisions do they have? Are they able to make changes that they wish and, if not, what factors constrain them? Has their situation changed? Have the improved (or worsened) access to natural resources been realized by men and women in the same manner and degree? If not, what major differences have there been? Has the resilience of women to hazards increased as much as or more than that of men? If not, what major differences have there been? Assessment of improvements to the quality of the resource base and integrity of ecosystems Incidence of disasters, changes to risk aversive livelihood patterns Have vulnerable groups benefited from improvements in reduced hazards in the same manner and degree as other social groups? If not, what major differences have there been? Do vulnerable groups have access to external institutions to the same level as other social groups? If not, what major differences have there been? Information on changes to local institutions, their relationship to external institutions (government and others) and changes to wider environmental policies and awareness Have women benefited from improvements in equity, strengthened local organizations or empowerment in the same manner and degree as men? If not, what major differences have there been? 39 ANNEX FOUR: ETHICAL GUIDELINES All evaluators should aspire to conduct a high quality evaluation process. The following ethical considerations are integral to this:   Cultural intrusion: Local customs regarding dress, personal interaction and gender roles, religious beliefs and practices should be respected. Anonymity / confidentiality: Evaluators must respect people‘s right to provide information in confidence, and must ensure that sensitive information cannot be traced to its source. Individual responsibilities: All team members should have an opportunity to disassociate themselves from particular judgments and recommendations. Any unresolved differences of opinion within the team should be acknowledged in the report. Participation / privacy: People are usually very busy and may not want to be involved. Evaluators should provide maximum notice, minimize demands on time, and respect people‘s right to privacy. Fundamental values: While evaluators should respect other cultures, they must also be aware of international values regarding minorities and particular groups, such as women. In such matters the UN Universal Declaration of Human Rights (1948) is the operative guide. Omissions and wrongdoing: Evaluators have a responsibility to note issues and findings, which may not relate directly to the Terms of Reference. The evaluation team should consult with GEF M&E when there is any doubt about if and how issues, such as evidence of wrongdoing, should be reported. Evaluation of individuals: Evaluators are not expected to evaluate individuals, and must balance an evaluation of management functions with this general principle. Disclosure: Briefings and unofficial summaries may be shared outside of GEF M&E. However, responsibility for distribution of published material rests with GEF M&E or its delegated agent. Integrity: Evaluators have an overriding responsibility to ensure that the evaluation is independent, impartial, and accurate. This principle should not be compromised.        Evaluators are not solely responsible to GEF M&E as the commissioning organization. They also have a wider responsibility to the institutions, groups and individuals involved, affected by, or interested in the evaluation. (These ethical standards for evaluators are adapted from CIDA and DFID). 40