EC WG/SOP-II(09)/Doc 4.1, Annex
WMO Monitoring and Evaluation Plan
WMO Strategic Planning Office
EC WG/SOP-II(09)/Doc 4.1, Annex, p. 2
Table of Contents
I. Introduction ................................................................................................................... 3
II. Objectives of the M&E System .................................................................................... 4
III. WMO Conceptual framework for Monitoring and Evaluation ...................................... 4
IV. WMO M&E Plan Guiding principles............................................................................ 5
V. WMO M&E System: Main features, roles and responsibilities..................................... 6
a) Performance measures.................................................................................................... 6
b) Evaluation ........................................................................................................................ 9
c) Reporting and use of information ................................................................................... 11
d) Quality assurance of performance data ......................................................................... 12
f) Management IT System ................................................................................................. 13
VI. WMO M&E System - Timelines, milestones and budget ...........................................14
EC WG/SOP-II(09)/Doc 4.1, Annex, p. 3
1. Cg-XV established RBM as the fundamental concept for managing the planning,
implementation and performance assessment of WMO‟s programme activities. The WMO RBM
framework consists of four major building blocks, namely, the WMO Strategic Plan (SP), the
WMO Operating Plan1, the Results-based Budget (RBB) and a WMO Monitoring and Evaluation
System (referred to hereafter as WMO M&E system).
2. A concept proposal for the design and implementation of a WMO M&E System was
developed in line with the pertinent decisions of EC-LVIII, Cg-XV and EC-LIX and presented to
EC-LX (2008). The proposal also incorporated relevant international best practices and lessons
learned, including from other UN organizations.
3. EC-LX expressed its general agreement with the proposal and requested the Secretariat to
develop a comprehensive M&E Plan outlining the development and implementation of the WMO
M&E System in a practicable, feasible and cost-effective way for presentation at its 61st session
taking the following decisions of the Council duly into account:
- The Plan should reflect a phased and pragmatic approach, for example, by initially
concentrating on a limited number of expected results;
- The Plan should take into account lessons learned and monitoring and evaluation tools
and methodologies available in a number of NMHSs.
- The M&E Plan should be based on a cost-benefit analysis.
- The implementation of a first-phase, simplified component of the M&E system should be
aligned with the preparation of the mid-term monitoring and performance evaluation
report requested from the Secretary-General for presentation to the Council at its 61st
session (June 2009);
- The development of monitoring and evaluation tools and methodologies should be
sufficiently flexible to ensure their adaptability by Secretariat departments, technical
commissions and regional associations.
4. Taking duly into account the requests of EC outlined above, this document outlines the M&E
Plan with a description of the main features of the WMO M&E System together with major
milestones and a timeline for its implementation. The M&E Plan aims at serving as a detailed
action plan for the implementation of the WMO M&E System, in order for the system to become
fully operational as of the Sixteenth Financial Period (2012-2015).
5. The WMO M&E Plan takes into account the findings of the “Review of M&E Practices and
Experiences in NMHSs” (Doc 4.3) and the “WMO Monitoring and Evaluation System – Cost –
Benefit Analysis” (Doc 4.2) which should be read in conjunction with this document.
The Secretariat Operating Plan for the financial period 2008-2011, which will be extended to a WMO
Operating Plan for the financial period 2012-2015.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 4
II. Objectives of the M&E System
6. It is recalled that the objectives of the WMO M&E System are:
- to enable senior management and programme managers of the Secretariat, the WMO
constituent bodies and the scientific steering committees of WMO Joint Programmes2 to
track progress of the Organization‟s activities, enhance performance and impact and
identify areas where corrective action is required with respect to activities and
- to enable the Secretariat to demonstrate organizational effectiveness, efficiency,
accountability and transparency externally and when reporting to WMO governing bodies;
- to facilitate dialogue and discussions at meetings of WMO constituent bodies, and with
partner organizations, donors as well as other stakeholders.
III. WMO Conceptual framework for Monitoring and Evaluation
7. The below diagram provides a schematic overview of the conceptual framework on which
the WMO M&E system is based. It also illustrates the iterative nature of monitoring and
evaluation at the level of activities, deliverables, results (outcomes) and impacts.
What applies in the following sections to the WMO constituent bodies also applies to the scientific steering
committees of the WMO Joint Programmes.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 5
IV. WMO M&E Plan Guiding principles
8. The WMO M&E Plan has been developed based on the following ten guiding principles
derived from guidance received from EC and experiences and lessons learned internationally
from the implementation of a managing-for-results approach, including in NMHSs:
a) The WMO M&E System is based on the performance measures as stipulated in the
WMO Strategic Plan and WMO Operating Plan (see WMO Conceptual Framework for
b) The WMO M&E System connects the results (outcomes) achieved to the use of
resources, implementation of activities and production of outputs;
c) Recognizing that performance information is likely to be most credible and accurate
when Secretariat departments and WMO constituent bodies are permitted to collect and
analyze the necessary performance information with a certain degree of flexibility to
best reflect the nature of their activities, the WMO M&E System will be based on a
bottom-up flexible approach3 but with uniform formats used for reporting purposes;
d) The WMO M&E System will evolve over time because the effective development and
use of performance measures require certain experience;
e) A result-oriented culture to underpin the WMO M&E System is critical and should
therefore be promoted acknowledging that culture change is a long-term endeavor4;
f) The WMO M&E System is both about learning and accountability. Learning is
consistent with a focus on program improvement and on identifying new approaches
that may be needed to address recurring or new needs;
g) The quality assurance of performance data is important while acknowledging that
information may never be perfect;
h) An integrated approach to monitoring and evaluation is key recognizing that an effective
performance management system requires both;
i) The WMO M&E System is consistent with the WMO Quality Management Framework
initiative and the ISO 9004:2000 Quality management systems - Guidelines for
j) Cost-efficient solutions for the implementation and operation of the WMO M&E System
are a priority in line with EC requests.
This principle is based on a growing recognition that no single approach to results-focused management and
budgeting is appropriate in all circumstances. The initial experiences of a number of countries has been that whatever
approach is taken must be flexible and adaptable. The OECD study “Implementing the Vision: Addressing Challenges
to Results-focused management and Budgeting”, Burt Perrin (2002), reports that Denmark, for example, through the
pilot projects of its performance management approach, learned that the approach must be adapted to fit each
Ministry. Similarly, rather than try to force programs into a pre-established framework and method for assessing
performance, it is essential to adjust the method to suit the circumstance rather than vice versa.
The experience of many different countries, such as Norway, Denmark, the United States and others, is that training,
guidance and the availability of technical assistance is required, over a period of time.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 6
V. WMO M&E System: Main features, roles and responsibilities
9. The main features of the WMO M&E System are described in the following sections, as are
the roles and responsibilities of the WMO constituent bodies and the Secretariat.
a) Performance measures
10. The WMO M&E System is based on the following performance measures:
- Performance metrics stipulated in the WMO Strategic Plan (Expected Results, Key
Performance Indicators (KPIs)) and WMO Operating Plan (Deliverables and Key
Performance Targets (KPTs)).
- Performance baselines – quantitative or qualitative – that provide data at the beginning
of, or just prior to, the monitoring period, and are used as a starting point to monitor future
performance. Baselines are the first critical measurement of the KPIs and KPTs.
- Performance targets which reflect the desired achievements (status of outcomes) at the
end of the monitoring period. Organizational performance during a given period is hence
measured on the status of the KPIs and KPTs as compared to the baseline situation (i.e.
targets versus baselines) illustrated in Figure 1 below.
11. The strategic and operational planning process in WMO is a „top down‟ process going from
Top Level Objectives to Activities. The detailed planning model can be illustrated as follows:
Top Level Objectives Strategic Thrusts
Expected Results and Key Performance Indicators (KPIs)
Deliverables and Key Performance Targets (KPTs)
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 7
12. It is recalled that the Expected Results reflect the desired outcomes to be achieved by WMO
as a whole (i.e. WMO Members, WMO constituent bodies, WMO Joint Programmes and the
Secretariat)5. The associated KPIs are the agreed upon measurements which will enable WMO,
as an Organization, to assess the achievement of the Expected Results. Achieving Expected
Results will be a complex process involving coordinated efforts of the main WMO components
(Members, WMO constituent bodies and the Secretariat), but also the participation of partners,
such as IOC/UNESCO, UNEP, UNDP, and international funding institutions.
13. The effectiveness of performance measurement at WMO depends, inter alia, on the quality
of performance metrics contained in the WMO Strategic and Operating Plans. Indicators must
overcome a number of measurement challenges, including the challenge of
attribution/contribution, the qualitative nature of some Expected Results, and the time frame
needed to achieve results. Good indicators are the cornerstones for measuring performance
and it is therefore imperative that they fulfil the SMART criteria6.
(ii) Roles and Responsibilities
14. Particular attention needs to be paid to ensuring an adequate quality of performance metrics
during the WMO strategic and operational planning processes.
When developing the WMO Operating Plan, the WMO constituent bodies and the
Secretariat will ensure that their work programmes are fully aligned with the WMO
Strategic Plan and that respective contributions to the Expected Results are clearly
distinguishable and measurable.
15. The establishment of baselines and targets is important because they are key components
of the measurement system without which any meaningful analysis of the achievement of results
The WMO constituent bodies and the Secretariat need to determine baselines and
targets, as appropriate, as an integral part of their contribution to the WMO Operating
16. Baselines and targets related to the KPIs (at the level of Expected Results) need to be
determined collectively during the strategic planning process.
See WMO Conceptual Framework for M&E, p.4.
Specific, measurable, achievable, reliable and time-bound.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 8
17. Monitoring is a continuous process concerned with overseeing programme implementation
and assessing the status of implementation in relation to the approved work plan and budget.
Regular monitoring enables identifying actual or potential problems as early as possible in order
to facilitate timely adjustments in implementation. Monitoring in a results-based context is also
concerned with assessing the status of performance, i.e. whether results have been achieved.
18. Monitoring functions in a reverse sense of the strategic and operational planning process in
WMO, i.e. as a „bottom-up‟ process. A simple model of the monitoring system is given below:
Resources Activities Deliverables Expected Results
19. For monitoring purposes, the basic data collection is performed at the level of the individual
activities. These activities are linked to the Deliverables, the achievement of which is measured
by the KPTs. A similar process is used to link the Deliverables to Expected Results, this time
measured by the KPIs.
20. The second focus of the monitoring is on the performance indicators, i.e. the KPTs and the
KPIs, to ensure that programme implementation is on the right track towards achieving results.
This set of data will allow programme managers to determine whether:
- a Deliverable has been achieved as a result of the set of Activities implemented;
- Key Outcomes and Expected Results have been achieved as a result of a set of
(ii) Roles and responsibilities
21. In the full implementation phase of the WMO M&E System7, the WMO constituent bodies
and the Secretariat would monitor the implementation of their respective work programmes as
outlined in the WMO Operating Plan as follows:
Prior to each financial period, the WMO constituent bodies and the Secretariat
departments will establish their respective monitoring approach to best suit and
support the implementation of their work programme. This approach would be
outlined in the respective contributions to the WMO Operating Plan and would
a) The 1 level monitoring to be conducted on a continuous (at least quarterly)
basis, concerned with the collection of data and analysis of the following elements:
- The tracking of the use of resources (post and non-post) within the allotted
budget allocations (this concerns in particular the Secretariat)
- Timeliness, quality and cost-efficiency of activities and outputs (meetings,
capacity building events, studies completed, persons trained, technical
[box continues on next page]
See section VI.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 9
The 2 level monitoring, to be conducted periodically (at least annually), concerned
with the collection of data and analysis of the following elements:
- Progress made or achievement of Deliverables (measured on the KPTs)
- Progress made towards achieving Expected Results as compared to the
baseline (measured on the PKIs)
22. The data to be collected on the KPIs (at the level of Expected Results) are in many cases to
be supplied by the Members. Where this is the case, the Secretariat will coordinate the
collection of relevant data on the KPIs, including through targeted surveys.
Note! The Expected Results are the higher level outcomes for WMO as a whole, including the
Secretariat, and their achievement is the ultimate yardstick for organizational
performance. Therefore, the responsibility for the assessment process of whether an
Expected Result has been achieved ultimately needs to reside with the Executive
Council and Congress.
23. In addition to providing information about the status of performance, i.e. whether results
have been achieved, evaluation provides more analytical information as to how the results were
achieved, or why they were not achieved.
24. Evaluations generally are episodic. There are a wide variety of potential evaluation
approaches, ranging from informal to highly sophisticated, using a wide-range of both
quantitative and qualitative methodological approaches. Evaluations can range from little or no
cost to very expensive and time consuming undertakings. They can be participative in nature or
carried out exclusively through external review. Evaluations can be conducted:
- at the end of a programme in order to make judgments about its effectiveness or impact
(ex post or summative evaluations)
- before a programme is implemented to assist in programme planning (ex ante
- while a programme is being implemented to inform on-going improvements (formative
25. Evaluation in a results-based M&E system generates evaluative information in real-time
while implementation is on-going. The WMO constituent bodies and the Secretariat can use
such evaluative information to focus on the broad strategy and design issues “are we doing the
right things?” (relevance), on operational and implementation issues “are we doing things right?”
(effectiveness, sustainability and cost-efficiency) and whether there are better ways of
approaching the problem “what are we learning?” (alternatives, good practices and lessons
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 10
26. The WMO M&E system comprises two types of evaluation:
- self-evaluations (to be conducted by the WMO constituent bodies and the Secretariat),
- independent evaluations (to be undertaken by expert groups of the technical
commissions, particular groups of the regional associations, or individual external or
internal evaluation/technical experts).
27. Both self-evaluations and independent evaluations will be conducted periodically in case
concerns arise during implementation of the WMO Operating Plan which would warrant a more
in-depth review of programme implementation to support decision-making. An evaluation could
be triggered when:
- There is a divergence between planned and actual performance;
- The linkages between implementation (activities, outputs) and Deliverables and
Expected Results is unclear;
- Making resource allocation decisions;
- There is conflicting evidence of results;
- There is a need to validate results-information or provide an impartial assessment of
programme performance (independent evaluation).
(ii) Roles and responsibilities
28. In the full implementation phase of the WMO M&E System8, the WMO constituent bodies
and the Secretariat would conduct self-evaluations on a biennial basis of the implementation of
their work programmes as contained in the WMO Operating Plan as follows:
At the beginning of each financial period the WMO constituent bodies and the
Secretariat departments will establish their respective evaluation approach as part of
their contributions to the WMO Operating Plan and include a description of self-
evaluations to be conducted on a biennial basis. Self-evaluations should address
issues of relevance, effectiveness, efficiency and sustainability of the achievement of
Deliverables and Expected Results and should, in particular, look into:
- the achievement of Deliverables and Expected Results compared to
- unintended outcomes
- the satisfaction of Members
- external factors having affected implementation
- challenges encountered during implementation
- corrective actions taken and eventual adjustment of strategies
- what worked well and why
- lessons learned
29. Independent evaluations will be carried out upon request by the EC and Congress, possible
based on recommendations by the Secretary-General and the WMO constituent bodies. Such
evaluations could cover, for example, a project, programme, specific work areas of the technical
commissions, or work programmes in particular regions. It is suggested that one such major
independent evaluation to be undertaken per year.
See section VI.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 11
30. In addition, in the full implementation phase of the WMO M&E System, one independent
evaluation of an Expected Result would be systematically scheduled on an annual basis.
Assuming that the total number of Expected Results is around eight9, this would ensure that all
Expected Results are evaluated once every eight years. Any decisions to modify Expected
Results from one financial period to the next should be preceded by an independent evaluation
to support decision-making.
An Evaluation Plan will be prepared by the Secretariat at the beginning of each
financial period and submitted to Congress, detailing the sequence and scope of
independent evaluations of the Expected Results within a financial period (four in
The scope of the remaining four independent evaluations to be conducted during a
financial period will be guided by the decisions of EC or Congress, possible based on
recommendations by the Secretary-General and the WMO constituent bodies.
c) Reporting and use of information
31. Using M&E information is key to an iterative management process, in which implementation
decisions are based on real-time monitoring information and learning from past experience
(evaluative information)10. M&E furthermore becomes a critical tool for demonstrating results as
part of accountability to stakeholders.
(ii) Roles and responsibilities
32. Taking due account of Congress and EC decisions regarding the periodic reporting on
implementation and performance, the following outlines the reporting requirements for the WMO
constituent bodies and the Secretariat.
33. Reporting of monitoring information would take place on an annual basis as follows:
Monitoring reports will be submitted by the WMO constituent bodies and the
Secretariat on an annual basis to the sessions of the Executive Council.
34. To facilitate the reporting of monitoring information, an “Implementation Monitoring Report
Template (IMRT)” will be developed by the Secretariat for use by the WMO constituent bodies
and the Secretariat Departments. The format of the template will be kept as simple as possible
whilst ensuring that it adequately serves its intended purpose. The Secretariat will compile the
information contained in the IMRTs and produce a summary overview for the sessions of
As suggested in the new WMO Strategic Plan.
See “WMO Monitoring and Evaluation System - Cost and Benefit Analysis” for more detail.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 12
35. Self-evaluations by WMO constituent bodies and the Secretariat need to coincide with the
preparation of input into the WMO Monitoring and Performance Evaluation Report at mid-term
(second year) and at the end of each financial period (fourth year).
Self-evaluation reports will be submitted by the WMO constituent bodies and the
Secretariat on a biennial basis as inputs into the WMO Monitoring and Performance
Evaluation Report (at mid-term and end of the financial period) to be submitted to the
relevant sessions of EC and Congress.
36. To facilitate the conduct and reporting of self-evaluations, a “Performance Evaluation Report
Template (PERT)” will be developed by the Secretariat. The format of the template will be kept
as simple as possible whilst ensuring that it adequately serves its intended purpose.
37. As regards independent evaluations, a summary of their results will form an integral part of
the WMO Performance Monitoring and Evaluation Reports.
A summary of the results of independent evaluations will be prepared by the
Secretariat and included in the biennial WMO Performance Monitoring and Evaluation
Report to be submitted to EC and Congress.
38. The Performance Monitoring and Evaluation Reports will be submitted to EC and Congress
at mid-term and at the end of each financial period in accordance with the decisions of
Congress. Such reports will draw on the monitoring and self-evaluation reports prepared by the
WMO constituent bodies and the Secretariat and the independent evaluations conducted during
the period under review.
The Secretariat will compile the WMO Performance Monitoring and Evaluation
Reports at mid-term and at the end of each financial period for submission to EC and
39. The reports of independent evaluations will be made available electronically to Members
unless EC or the Secretary-General decides, on a case by case basis, on another mode of
40. The WMO constituent bodies and the Secretariat will ensure that information generated by
monitoring and evaluation is used on a continuous basis to enhance performance of respective
work programmes and to support effective coordination between programmes.
d) Quality assurance of performance data
41. The quality of performance data is important. If the data that is reported do not reflect actual
performance it would be misleading to use it to support decision-making. Information might
never be perfect but the aim should be to reduce uncertainly to the extent possible.
42. The WMO constituent bodies, the Secretariat and Members should ensure that performance
data is provided objectively and to the best of their knowledge. Sufficient documentation should
be compiled in support of the performance data which has been reported for eventual
independent evaluation and audit.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 13
43. The independent evaluation mechanism as described in Section 5b above as well as the
external and internal audit functions indirectly serves as a form of performance data validation
e) M&E institutional support
44. One key lesson learned from experience with result-based management systems, including
M&E systems, both in national environments and in the context of the UN system is that central
support is important in order to assist, advice and ensure a coherent implementation of a results-
oriented performance management approach.
45. Institutional support for M&E will be provided, as appropriate, by the Secretariat to the WMO
constituent bodies and internally to Secretariat Departments. The support will include promotion
of a results-based performance culture, M&E training and capacity building, advice on
monitoring and evaluation, preparation of guidelines, policies, procedures and templates and
preparation of monitoring and performance reports for internal use and for submission to EC and
46. To ensure effective communication and feed back on performance information, a network of
M&E focal points will be established from Secretariat departments and the WMO constituent
bodies. The network will, in particular, share information, discuss problems, find solutions,
coordinate input, discuss methodologies, identify data collection constraints, ensure validity of
data, share experiences, etc..
f) Management IT System
47. A Management Information System (MIT system) is required for the collection, analysis,
storing, sharing and dissemination of performance data.
48. The most cost-efficient solution would be to use the Microsoft Excel spreadsheet system,
and to post completed spreadsheets in an ftp folder to which Executive Management, Directors
of Secretariat Departments and the Presidents of technical commissions and regional
associations would have access. A second advantage of an Excel spreadsheet solution would
be the ease with which the spreadsheets can be filled in and accessed around the world. The
disadvantage would be the missing link to the financial information in the Oracle-based ERP
system used within the Secretariat.
49. It is proposed to opt for the Excel spreadsheet solution and to develop and test it during the
Pilot Phase 2010-2011. When more experience is gained with the use of this IT solution, it
would be determined whether it effectively and efficiently serves the needs of both providers and
users of performance information. Based on such a review, an eventual further development of
the IT solution could be envisaged during the full implementation phase (as of 2012)11.
Such as the development of a non-complicated web-based software, either from scratch or by customizing an
existing software available commercially.
EC WG/SOP-II(09)/Doc. 4.1, Annex, p. 14
VI. WMO M&E System – Tasks, timelines and budget
50. Table 1 in the document EC-WG/SOP-II(09)/INF.4.2 (Cost-benefit analysis of the WMO
M&E System) provides an overview of the required tasks, related timelines and estimated
budget for non-staff resources12 required for implementing and operating the WMO M&E
51. Taking duly into account the decision of EC-LX regarding a phased approach, three phases
are suggested for the implementation of the WMO M&E System as follows:
- a Preparatory Phase (2009) concerned with the preparation of the Pilot Phase for two
selected Expected Results.
- a Pilot Phase (2010-2011) in which monitoring and self-evaluations by WMO
constituent bodies and the Secretariat would commence for the Expected Results
initially selected. This Phase would thus be concerned with the testing of the suitability
of the M&E methodologies, processes, procedures and guidelines, including the
involvement of the WMO constituent bodies in the M&E process as well as setting up of
a Management Information System to facilitate the storing, sharing and dissemination
of performance data and M&E capacity building. An evaluation of the Pilot Phase
would be conducted in 2011 to identify eventual adjustments to be made to the M&E
System before the Full Implementation Phase.
- a Full Implementation Phase (as of 2012) covering all Expected Results taking duly
into account experiences gained and lessons learned during the Pilot Phase.
52. For the Pilot Phase it is suggested to select one expected result related to a scientific
technical work area and one related to a policy area having well defined Key Performance
Indicators. Such a combination would provide the best basis for ensuring that the initial M&E
methodologies, processes, procedures and guidelines are broadly tested and, after eventual
adjustments, applicable to all Expected Results in the Full Implementation Phase. For practical
reasons, a second proposed criterion for the selection of Expected Results is that they each
should mainly fall under the responsibility of one Department only.
53. On the basis of the above, it is suggested to select the following Expected Results for the
- Expected Result 5: Development and implementation of the new WMO Information
- Expected Result 8: Broader use of weather-, climate- and water-related outputs for
decision-making and implementation by Members and partner organizations.
54. A detailed M&E framework for Expected Result 5 consisting of a description of the
deliverables, indicators (based on those in the Secretariat Operating Plan 2008-2011 with
eventual fine-tuning as necessary), baselines, targets, M&E methodologies, processes,
procedures and guidelines, as well as the involvement of relevant WMO constituent bodies, as
appropriate, in the M&E process, would be prepared and submitted to EC-LXI. The M&E
framework for Expected Result 8 would be prepared during the 2nd half of 2009.
For more details on the estimated budget please see “WMO M&E System – Cost-benefit Analysis”.
EC WG/SOP-II(09)/Doc 4.1, Annex, p. 15