A Proposed Monitoring & Evaluation Framework for Climate Change Adaptation
UNDP working paper, Nick Brooks and Jennifer Frankel-Reed
Xianfu Lu GEF International Workshop on Evaluating Climate Change and Development Alexandria, Egypt 10-13 May 2008
Presentation outline
1. Background
2. The Proposed M&E Framework 3. An example
4. Next steps
1 1
1. Background - What do we mean by
climate change adaptation?
Adaptation is to
Securing development benefits that might otherwise be undermined by climate change
2 2
1. Background - UNDP context for monitoring adaptation
UNDP supports the implementation of a diverse portfolio of adaptation initiatives, which focuses on the following outcomes:
1. Institutional capacity development for managing climate change
risks;
2. Integration of climate change risks into climate sensitive policies at
national, sectoral, or sub-national scales;
3. Implementing climate-resilient practices and adaptation measures;
and
4. Implementing information management systems for climate change
decision support.
3 3
4 4
1. Background
- M&E Framework motivation: Responding to
stakeholder demands
• Conceptually… How can adaptation M&E reflect „going
beyond’ business-as-usual development?
• Substantively… How can adaptive capacity be tracked by
projects (given uncertainties associated with climate change and its impacts)?
• Consistently…Is there a reference for developing and
illustrating project-level adaptation M&E frameworks?
• Practically…How can the diversity of adaptation project results
be aggregated?
5 5
1. Background - Purposes of the proposed UNDP M&E
Framework for adaptaiton
• Guide stakeholders for developing adaptation projects
and programming in overcoming challenges for meaningfully monitoring climate change adaptation
• A draft M&E framework for further testing, revision and review
6 6
2. The M&E framework • Attribution
• Relevance
Complexities for adaptation M&E
o Climate risks to and impacts on development objectives compound nonclimatic risks and impacts o Stakeholder perception and narrative description are employed o Development objectives will likely be influenced beyond project lifetimes; o Sustainability factors are prioritized
• Calibration
o Climate-related hazards are changing (moving baseline); o Adaptation is evaluated against projected conditions
7 7
2. The M&E framework- Rationale
• Goal • Objective
Securing development benefits that might otherwise be undermined by climate change Improving adaptive capacity and/or reducing vulnerability of human populations and natural and economic systems on which they depend to climate change and its impacts Multi-sectoral and multi-scale, “the ability of a system to adjust to climate change” (IPCC) Enabling factors Good practices from experience suggest a set of key processes…
8 8
• Adaptive Capacity
•
2. The M&E framework- thematic areas
Thematic Areas (Development benefits)
• • • • • •
TA1- Agriculture/food security TA2 - Water resources and quality TA3 - Public health TA4 - Disaster risk management TA5 - Coastal zone development TA6 - Natural resources management
9 9
2. The M&E framework- adaptation processes
‘Good Practice’ Processes • Policymaking and planning (including budget and regulatory processes)
• • • •
Capacity building and awareness raising Information management (e.g. EWS, monitoring,
analysis processes)
Investment decision-making Risk reduction practices/livelihoods and/or resource management processes
10 10
2. The M&E framework- Indicator types
Indicators will measure the success of a portfolio / project in achieving:
• Coverage - the extent to which projects engage with
stakeholders
• Impact - the extent to which projects deliver the intended
results, or bring about changes in behaviour that support the portfolio‟s objectives.
• Sustainability - the ability of stakeholders to continue to adapt
beyond project lifetimes.
• Replicability - the extent to which experiences, results and
lessons are captured and disseminated for broader benefits.
11 11
2. The M&E framework- Structured approach for M&E
Adaptation Thematic Areas Adaptation Processes Indicator types
TA6
TA1
Policy/planning
Coverage
Capacity building/awareness
TA2 TA5
Information management
Impact
Sustainability
Investment decisions
TA4
TA3
Practices/livelihoods/resource management
Replicabilty
12 12
2. The M&E framework- Indicator aggregation based on common
units applied at project Outcome level
Project Level Portfolio Level
Output 1 Policy/plan
Output 2 Practices
Outcome 1
Objective A (V)
Outcome 2
Objective
TA Goal
Objective B (AC)
Output… Investment
Output x
Outcome 3
13 13
2. The M&E framework- Portfolio goals and objectives link to MDGs and
development benefits
TA1. Food Security/Agriculture Food insecurity resulting from climate change minimized or reversed, and GOAL new opportunities for food production resulting from changes in climate exploited cf. MDG Goal 1 Eradicate extreme poverty and hunger Reduced vulnerability of communities and food-production systems threatened by changes in mean climatic conditions and climatic variability; Objective and/or enhanced capacity of individuals, communities and institutions to plan for and respond to the impacts of climate change on food security TA2. Water Resources & Quality GOAL Water managed to reduce stress and scarcity of clean water and minimize the negative impacts of extreme hydrological events resulting from CC
cf. MDG Goal 7, Halve, by 2015, the proportion of people without sustainable access to safe Target 10 drinking water Reduced vulnerability to water stress, scarcity of clean water, and/or extreme Objective hydrological events; and/or strengthened capacity of water sector-related institutions and communities to respond to climate variability and change
14 14
2. The M&E framework- Standard indicators I
I. Coverage i. ii. Number of policies, plans or programmes introduced or adjusted to incorporate climate change risks. Number of stakeholders (e.g. communities, households, agencies, decisionmakers) engaged in capacity building activities for vulnerability reduction or improved adaptive capacity. Number of stakeholders served by new or expanded climate information management systems (e.g. early warning systems, forecasting, etc.). Number of investment decisions revised or made to incorporate climate change risks). Number of risk-reducing practices/measures implemented to support adaptation of livelihoods and/or resource management.
iii. iv. v.
15 15
2. The M&E framework- Standard indicators II
II. Impact i. Percent change in stakeholders’ behaviours utilizing adjusted processes, practices or methods for managing climate change risks, assessed via QBS or other evidence (relevant across processes i-v). Percent change in stakeholders’ capacities to manage climate change (e.g. communicate climate change risks, disseminate information, or make decisions based on high quality information), as relevant, assessed via QBS.
ii.
iii. Percent change in use of/performance of information management systems (e.g. early warning response times).
iv. Percent change in stakeholder perceptions of vulnerability (or capacity to adapt to) to a recurrence of primary climate change-related threat(s), assessed via QBS.
v. Narrative description of the role of project interventions in reducing vulnerability (or improving capacity to adapt to climate change-related threat(s)), assessed via QBS.
vi. Change in the relevant quantitative development outcome (food security, water resources, health outcomes, etc.) either in relation to average historic data or to years marked by extreme conditions.
16 16
2. The M&E framework- Standard indicators III, IV
III. Sustainability i. ii. Number of project beneficiaries involved in capacity development for implementation of specific adaptation measures or decision-support tools. Availability of skills and resources necessary to continue adaptation after conclusion of project (at relevant scale), assessed via QBS.
iii. Stakeholder perceptions of adaptation sustainability, assessed via QBS.
IV. Replicability i. ii. Number of ‘lessons learned’ codified. Number of ‘lessons learned’ disseminated through relevant networks, knowledge sharing fora, and/or potential stakeholder groups outside the project.
17 17
2. The M&E framework- Contributions and implications
What are the implications of the proposed framework for evaluation practice? • Semi-standardizing M&E by theme and key „good practice‟ processes; • Proposing standard indicators to adapt to project contexts
18 18
2. The M&E framework- Contributions and implications
What does the Framework contribute that is new and/or innovative? • Proposing common units for aggregating indicator results • Employing stakeholder perception surveys using the H-form alongside quantitative “development outcome” data
19 19
2. The M&E framework- Contributions and implications
What are the implications for capacity at all level(s)? What needs to be done to mainstream these approaches, integrate lessons learned, scale-up, etc? • Both conceptual and practical capacity is key, piloting M&E can be a capacity development vehicle. • To integrate: standard frameworks applied across agency or funds
20 20
2. The M&E framework- Contributions and implications
What are the implications for regional networking and capacity-building? • Need for a structured system to review, share M&E experiences (e.g. ALM); • To codify and integrate lessons learned and scale up.
21 21
3. Example - Project-scale application of indicators to outcomes
TA 1. Agriculture/Food Security Project Objective: Vulnerability of farmers and pastoralists to drought and rainfall variability reduced. Outcome Indicator
1.1 Number of agriculture-related policies, programmes and plans incorporating climate projections into their design (I.i from standard indicators table) 1. Information from midterm climate projections integrated into agriculturerelated policies and climate forecasts integrated into agriculture-related planning on appropriate time scales 1.2 Percent change in policymakers‟ use of climate information in agriculture and fisheries policies and plans, assessed via survey (II.i) 1.3 Narrative stakeholder description of the role of integrating climate projections into agriculture policies and plans in reducing vulnerability to drought and rainfall variability, assessed via qualitative survey (II.v)
Type Coverage
Impact
Impact
22 22
3. Example - Project-scale application of indicators to outcomes
Outcome Indicator 4.1 Percent change in vulnerability of food security to rainfall variability and/or drought, via perception-based stakeholder survey such as VRA (II.iv) 4.2 Availability of skills and resources necessary for farmers and/or pastoralists to sustain climate risk management practices beyond the end of the project‟s lifetime (III.ii) 4.3 Number of „lessons learned‟ captured about reducing vulnerability of food security to drought and rainfall variability (IV.i) All outcomes: 1 - 3 4.4 Number of „lessons learned‟ disseminated through the Adaptation Learning Mechanism (ALM) platform and regional knowledge sharing efforts (IV.ii) 4.5 Food security deficits during periods characterised by climate extremes (e.g. drought or false start to wet season/extreme rainfall conditions), compared with deficits in previous years characterised by similar extremes (II.vi) OR 4.5 Food production or food security among project stakeholders (depending on data availability: predictability, ability to purchase food, or yields) (II.vi) Type Impact
Sustainability
Replicability
Replicability
Impact (development outcome)
23 23
4. Next steps
Gathering baselines, tracking results and reviewing experience with the framework to determine:
• Field-level feasibility for project/programme teams • Stakeholder feedback
• Fit of the framework with adaptation activities and
needs
• Donor and agency satisfaction on evaluation side • Integrating new approaches, tools
24 24
Contact
Xianfu Lu xianfu.lu@undp.org or Jennifer Frankel-Reed jennifer.frankel-reed@undp.org Climate Change Adaptation Team United Nations Development Programme (UNDP)
Adaptation Learning Mechanism Global knowledge sharing platform for adaptation www.adaptationlearning.net
25 25