UNICEF, Guidelines for the 2007 Country Office Annual Reports,

Results Based Management at country level: Systemic issues that prevent good UNDAF results and the use of UNDAF results information A paper presented to the Working group on programming policies Alexander MacKenzie Consultant 20 March, 2008 2nd Draft – Not for quotation Table of contents Table of contents.............................................................................................................. i 1.0 The Study: Background, Purpose and Methodology, ................................................. 1 2.0 Definitions and a conceptual framework ................................................................... 3 3.0 RBM at country level: Situation Analysis ................................................................... 5 Immediate causes........................................................................................................ 6 UNDAF annual reviews and reports don‘t have a results-focus, and the results matrix is not live. .................................................................................................. 6 There isn‘t enough routine internal or external demand for information about UNDAF results..................................................................................................... 6 Different RBM Terminologies and Approaches ......................................................... 8 Roles & responsibilities for UNDAF results monitoring and reporting are not operational ........................................................................................................... 9 Underlying causes ..................................................................................................... 11 The UNDAF, results matrix, and M&E plan are not owned sufficiently at all levels . 11 Root causes............................................................................................................... 14 Agency RBM systems do not focus on UNDAF results........................................... 15 Agency accountability frameworks do not focus on UNDAF results ........................ 17 Corporate leadership and commitment for managing for UNDAF results................ 20 4.0 Conclusion and preliminary recommendations for the Road Map ........................... 21 Annexes Annex A: Mapping of Agency RBM Systems ................................................................. 29 Annex B: Principles and a Framework for Results-Based Management in an Organization .................................................................................................................. 49 Annex C: References .................................................................................................... 51 i 1.0 The Study: Background, Purpose and Methodology, Background At country level, the UN system is struggling to use Results-Based Management (RBM) to support implementation of the United Nations Development Assistance Framework (UNDAF). A range of recent studies and evaluations show that UNDAF Results Matrices (RM) are of mixed quality and do not live up to expectations. The Delivering as One pilot countries are facing the challenge of how to make different agency RBM systems work together to manage for results at country level. The major weaknesses in UNDAF results matrices have been identified as:  Outcomes that are too broad, not strategic, and whose contribution to national priorities is unclear;  Outputs that are not linked to those accountable for them;  Results chains with poor internal logic;  Indicators that don‘t help to measure whether results – particularly outcomes – are being achieved and a lack of baselines and targets; and  Poor use and monitoring of risks and assumptions. Most of these concerns have to do with the formulation of high quality results. But a more important issue is the use of results information. While UNCTs are struggling to formulate SMART1 results – they appear to be struggling even more with how to use results information to manage for UNDAF results. As the UNEG study of RBM across 21 organizations concluded, ―…monitoring for results and use of performance information are, in particular, the weakest components of RBM‖2. Longhurst3 likens the UNDAF to a ―barometer of UN reform, with the results matrix as a serious business plan for implementation‖, but – like others - finds that the quality of results is poor. He suggests that as UN staff become more familiar with RBM in their work, the situation will improve. But can it improve without understanding more about the different ways that RBM is understood and practiced at country level? To answer this question, the United Nations Development Group‘s working group on programming policy (WGPP) identified a critical need to examine the systemic aspects that affect RBM at country level. 1 SMART results are: Specific, Measurable, Achievable, Relevant, and Timebound. UNDG, Common Country Assessment and United Nations Development Assistance Framework, Guidelines for UN Country Teams on preparing a CCA and UNDAF, United Nations, February 2007. 24. The consultant has also heard the ―R‖ interpreted as ―rights-based‖ in recent inter-agency workshops. 2 UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007, 5. 3 Longhurst, R., Review of the Role and Quality of the United Nations Development Assistance Frameworks, Overseas Development Institute, May 2006, 22. 1 Purpose of the study The purpose of this study is to identify and examine the systemic issues that affect the use of RBM at country level4. A second part of the study will propose a road map for harmonizing and strengthening the RBM approaches of the UN system. The findings of this study and recommendations contained in the road map will help to strengthen common country programming processes, including development of common guidance and training materials. It is envisaged that they will feed into policy formulation at the wider UN system level. Methodology This study is based mainly on a desk review of major studies and evaluations about RBM in both the inter-agency arena, as well as for 13 organizations (see Annex A). Assumptions & limitations A key assumption of this study is that analysis of the systemic issues that prevent more effective RBM at country level cannot be separated from analysis of the UNDAF and the workings of the UNCT. Nor can it be separated from its links to agency country programmes and projects, and their related RBM systems. They are intertwined and mutually dependent5. Given the large number of studies and evaluations about agency RBM systems, this study has limited itself to agency issues which have, or suggest, wider inter-agency concerns. Specific agency issues and recommendations are amply addressed in the various evaluations and studies6, and are not repeated here. Outline of the report This study contains a situation analysis and it proposes some preliminary recommendations for the road map. Following this introduction:  Part 2 establishes the definitions of RBM used in this report, and offers a conceptual framework for understanding what is needed to manage for results at country level;  Part 3 uses this conceptual framework to present a situation analysis of RBM;  Part 4 offers some preliminary recommendations for the road map. 4 UNDG, Terms of Reference – Study of Inter-Agency Results-Based Management, Task Team 1 of the Working Group on Programming Policies (WGPP), December 2007. 5 This seems intuitive, but recent inter-agency reports about RBM in the UN system appear to de-link principles, lessons and good practices in RBM from the UNDAF. The Mayne report on best practices in RBM in the UN system does not mention the UNDAF once. The 2006 JIU report on RBM in the context of reform, also does not mention the UNDAF. 6 The literature suggests that in many organizations a project approach still dominates, and that mobilization of resources and delivery of outputs are still more powerful drivers than achievement of outcomes. The literature also highlighted that some agencies continue to have a management culture that is hierarchical, centralized and rigid. For some agencies this makes them risk averse and slow to seize opportunities. Often the lack of decision-making authority and adequate financial resources at country level make it very difficult for some agencies to contribute to the UNDAF, let alone engage in managing for UNDAF results. 2 2.0 Definitions and a conceptual framework This study uses the harmonized RBM terminology approved by the UNDG in 20037. The UNDG defines Results-Based Management (RBM) as: ―A management strategy by which an organization ensures that its processes, products and services contribute to the achievement of desired results (outputs, outcomes and impacts). RBM rests on clearly defined accountability for results, and requires monitoring and self-assessment of progress towards results, and reporting on performance.‖ The literature review shows that, while there are numerous definitions of RBM in the UN system, they all vary slightly on the same theme. As stated in the ILO RBM guidebook8, ―…there is a common agreement that the implementation of RBM entails applying lessons learned from one programming cycle to the next programming cycle in order to improve performance‖. Or, as Mayne has summarized: ―The essence of RBM is managing so that intended outcomes are achieved.‖ 9 The practice of RBM at country level asks UN staff, along with their implementing partners, to learn and adapt from their experiences in order to improve performance. Overall, the literature demonstrates that most organizations are succeeding to institutionalize RBP (results-based planning), and that they are moving quickly towards results-based reporting. But agencies are having a more difficult time with the use of results information for management decision-making at country level. And the ―M‖ or management element is essential for the full practice of RBM. The UNEG study10 found that RBM has not developed into a management tool, but it perceived by management and program managers as a technical reporting exercise. Several of the agency evaluations concurred11. For a UN organization to manage for results, Mayne12 identifies six (6) key elements: 1. Foster senior-level leadership in results-based management 2. Promote and support a culture of results, particularly:  Informed demand for results information;  Supportive organizational systems, procedures and incentives;  A results-oriented accountability regime; and  A capacity to learn and adapt. 3. Build results frameworks with ownership at all levels 7 8 UNDG, Approved harmonized RBM Terminology, UNDG, 2003. ILO, RBM Guide Book, ILO, October 2006. 4. 9 Mayne, J., Best Practices in Results-Based Management: A review of experience, Volume 1: Main Report, UN Secretariat, July 2007, p 15. 10 UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007, para 21. 11 For example, the evaluation of RBM at UNDP found ―little evidence that results are being used systematically to inform adjustments to the country portfolios.‖ Poate, D. et al., Ibid., x. 12 Mayne, J. Ibid., 3-4. Note that the details of these elements correlate closely with the benchmarks identified by the JIU in 2004 for the successful implementation of RBM in UN organizations. JIU, Overview of the series of reports on Managing for Results in the United Nations System, JIU/REP/2004/5, JIU, 2004. 3 4. Measure sensibly and develop user-friendly RBM information systems 5. Use results information for learning and managing, as well as for reporting and accountability 6. Build an adaptive and RBM regime through regular review and updating of frameworks. A more detailed description of these elements is provided in Annex B. They have been adapted into a conceptual framework that seeks to explain what is required for the UN system to manage for results at country level. It begins with a desired situation at the top, and lays out the critical elements or causes at immediate, underlying and root levels. It is not intended to be comprehensive. Inter-agency discussion would certainly add-to or revise the causes and their order. Rather it is meant to provoke debate about the major systemic problems that prevent the use of RBM at the country level for UNDAF achievement. Figure 1: A conceptual framework – Key elements required to manage for results at country level13 UNDAF results are SMART & results information is used to inform decision-making by the UNCT and stakeholders There are regular progress reviews of the UNDAF and high quality resultbased reports << the UNDAF is live and adapted >> There are clear roles & responsibilities for UNDAF monitoring and reporting There is demand for UNDAF results information There is a common and applied RBM terminology & approach Immediate Causes UNDAF results are owned at all levels with: - a clear results framework with concrete performance expectations - an operational strategy and plan for measuring performance - senior country management capacity for results management Underlying Causes Supportive organisational RBM systems, incentives, and procedures A results-oriented accountability regime that balances Country and Corporate results Corporate management leadership & commitment to a country focus for RBM Root Causes 13 Adapated from: Mayne, J., Best Practices in Results-Based Management: A review of experience, Volume 1: Main Report, UN Secretariat, July 2007, p 3-4. 4 3.0 RBM at country level: Situation Analysis It is helpful to begin any situation analysis with a sense of what we want14. So what do we want from the UNDAF and from the practice of results-based management at country level? The RBM definition, the conceptual framework (above), and the UNDAF guidelines help to provide an answer: We want SMART, strategic results, that make a value-added contribution to the achievement of MD-MDG-related national priorities. And we also want that results information is used to influence decisions about UN country programming and improve performance. What do we have? Problem statement The literature suggests that the problem is two-fold: (1) UNDAF results are not yet SMART enough, and (2) Results information is not being generated and used effectively by UNCTs to manage for the achievement of UNDAF outcomes. Before proceeding it is important to emphasise the positive. All the external evaluations reviewed found that agencies have made great efforts to institutionalize effective RBM systems and tools. Moreover, as the Dalberg evaluation of UNFPA points out, the twin tasks of institutionalizing and harmonizing RBM have ―created significant complexity and new requirements for development organizations‖. In short, UN agencies have taken on, in a relatively short period, a Hurculean task of creating ―cultures‖ of results in their organizations. Paradoxically, the force and energy that agencies are applying to this task may also be making it more difficult to strengthen the inter-agency use of RBM to manage for UNDAF outcomes. It is not possible to speak of one UN RBM system. There are many systems. And this points to three (3) key issues that underlie most of this report and its preliminary recommendations:  What is the strategic results framework that drives agency RBM systems?  Who are the primary, intended users of results information from agency RBM systems?  How well do agency RBM systems help the UNCT generate and use coherent results information for UNDAF achievement? The following analysis attempts to understand these issues. 14 This is confirmed in the literature by Mayne and others who argue that institutionalizing RBM requires ―A clear understanding of results and what they are used for‖. Mayne, J. Ibid. 5 Immediate causes UNDAF annual reviews and reports don’t have a results-focus, and the resultsmatrix is not live. SMART-er results and the use of results information for decision-making depend on regular progress reviews against planned results and high quality results reporting. Results frameworks must be live tools, open to adaptation. However in many countries this is not happening for the UNDAF. The results-focus of the UNDAF annual review and Resident Coordinator Annual Report (RCAR) has been strengthened. The guidelines for UNDAF annual reviews have a clear results focus. Their purpose is to assess progress towards UNDAF results, to inform agency annual work instruments and to adjust the UNDAF monitoring & evaluation (M&E) plan15. Anecdotal evidence suggests that this is not very common, and that results information from agencies may not be rolling-up in a coherent and usable way to the UNCT for consideration of the UNDAF as a whole16. This is supported by the recent evaluation of One UN pilots that highlighted the need for consolidated monitoring and reporting systems17. The RCAR is a vehicle for reporting about UNDAF performance. But given the lack of results information, reports tend to focus more on coordination processes18. The RCAR guidelines do not ask RCs and UNCTs to reflect on whether, or how well, results information is being used to better align agency and partner resources for the achievement of UNDAF outcomes. In practice, both the UNDAF AR and RCAR appear to have a reporting focus, primarily to an audience outside the country – ―headquarters and regional directors‘ teams‖ 19. One indicator of this problem is that many UNDAF RMs are static. Using RBM, the results matrix (RM) should be a live tool, updated regularly to reflect results information, major lessons, and decisions about programme strategy and design. In practice, the RM is often seen as a one-off product necessary for programme planning and reporting, but not very essential for day-to-day programme management. There isn’t enough routine internal or external demand for information about UNDAF results One major reason why UNDAF reviews and reports lack a results-focus is that there isn‘t enough demand for information about UNDAF performance. This is a major issue 15 UNDG, Guidelines for the UNDAF Annual Review, UNDG, 2007. None of the documents examined the quality and results focus of the UNDAF ARs and RCARs in detail. This finding is based on prior field visits and discussions with UN COs. It may need further validation. 17 CEB High Level Committee on Programmes, Evaluation of the Delivering as One Pilots – Progress Report on Evaluability Assessments, paper by UNEG, CEB/2008/HLCP-XV/CRP.3 3 March 2008. para 14. 18 UNDG-CPSG, Consolidated CPSG Comments on the 2006 Synthesis of Resident Coordinator Annual Reports (SRCAR), 2007. 1. 19 UNDG, 2007 Guidelines for the completion of the Resident Coordinator Annual Report (RCAR), 2007, UNDG. 1. 16 6 identified by Balogun in the M&E issues paper about the UNDAF guidelines20. It is important to consider internal demand from the UN system, as well as external demand from national authorities and other stakeholders. Internal The review of agency RBM systems reveals 3 things: First, agency RBM systems are focused mainly on meeting demands about corporate or organizational priorities. Second, the audience for this results information is mainly outside the country, typically headquarters and governing bodies. Certainly, RCs and UNCTs represent a constituency for results information about the UNDAF, but they too exist within, and must feed, their agency RBM systems. This is supported by a striking finding from the UNDAF evaluability study: outside of the UNCT, there is no other current demand for evaluation of the UNDAF‖21. Third, there has been a much more focus on results-based planning, and efforts to harmonize planning approaches among UN agencies, than on the use of results information for decision-making. RBM is understood mainly as planning system – less as a reporting system – and almost not at all as a system to help with day-to-day decisions about programme management22. So high quality results information isn‘t really needed, or only needed periodically to meet outside reporting requirements. Beyond the UNCT, responsibility for the quality of UNDAF progress reviews and reports appears to rest with the Regional Director‘s Teams (RDT). They have a central role to ―provide leadership, strategic guidance and support to RCs/UNCTs for the achievement of country level results‖23. However it is not clear from existing studies and reports how well RDTs are able to perform this core function, and this needs further investigation. Certainly RDTs and their regional peer support groups (PSGs) have the potential to demand more coherent information about UNDAF performance, and encourage the use of results information for UNCT decision making. The RC and UNCT performance appraisal system may offer an entry point for this change. External Nearly all the organizations studied work through programmes and projects of cooperation at country level. Across the agency literature, there is a message that results from development cooperation must contribute to the achievement of national priorities. For example, ILO‘s report to its governing body24 states that ―ownership of results is a major source of motivation and legitimacy‖ and there needs to be greater involvement of constituents, particularly at country level, within an RBM framework. 20 Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. 3-5. 21 Balogun, P., UNEG Study on the Evaluability of the UN Development Assistance Framework, UNEG, Dec 2006, 5. 22 UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007, 5-6 23 UNDG, Generic Terms of Reference for RDT, Draft 5, March 6, 2008. 24 ILO, Preview of the Programme and Budget proposals for 2008-09 and related questions (a) Strategy for continued improvement of results-based management in the ILO. GB.297/PFA/1/1. Report to the ILO Governing Body, 297th Session, Geneva, November 2006, 26. 7 However, in general, many agencies do not include national authorities as the first, or even second audience for results information. And both agency and inter-agency RBM systems do not emphasise greater engagement with national planning and results/performance systems25. Agency RBM systems convey a message that results should contribute first and foremost to national priorities, but results information isn‘t flowing in that direction. Overall, the impression is that RBM systems are seen as agency ―things‖ – corporate requirements – rather than systems to help generate information that could lead to better programming decisions. This is fed by a perception that the introduction of RBM is intended primarily to satisfy donors. The UNFPA manual provides a good example. The text suggests that the primary drivers for RBM are the donors, and that improvements in performance, management and accountability are ―necessary to increase the confidence of donors and other partners‖26. This is not unique to UNFPA. Most of the agency guidance has this or a similar message: results information is primarily for an audience outside the country. The 2007 TCPR27 establishes that the RC, supported by the UNCT, must now report to national authorities on progress made against the results agreed in the UNDAF. Whether this creates greater external demand for results information is not known. A more important issue is whether the UNCT is in a position to respond to this demand when it comes. Different RBM Terminologies and Approaches Terminology across the agencies shows a high level of harmonization. There appears to be broad agreement about:  The definition of outcomes and outputs,  Causality in a results chain,  The importance of indicators, risks and assumptions; and  That achievement of outcomes requires actions by those beyond the control of programmes and projects. There continue to be some discrepancies between UNDG and agency terminology. These are shown in table 1. See Annex A for a more complete description of agency RBM systems and terminology for the 12 agencies. 25 26 Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007, 39. UNFPA, Results-Based Management Orientation Guide, See 1.2: Why is results-based management necessary, June 2001. 2-3. 27 UN General Assembly, Triennial comprehensive policy review of operational activities for development of the United Nations system, A/C.2/62/L.63, 19 December, 2007. 8 Table 1: Key discrepancies in RBM Terminology28 UNDG Goal (aligned with MDGs/ National priorities) Impact FAO Strategic objective (corporate) ILO UNESCO UNHCR Goals (corporate) Outcome Activity Objectives Intermediate outcomes (―Goals of the Organization‘s constituents‖) Immediate outcomes Results Interventions Objective There is a mostly common understanding of outcome, but very few of the agency RBM manuals distinguish between UNDAF outcomes and agency outcomes. More importantly, none of the agency RBM manuals makes absolutely clear that UNDAF outcomes are the collective strategic results for UN system cooperation at country level, intended to support achievement of national priorities. Thus, wherever there are messages about collecting and using outcome information to influence management decisions, these messages are, generally, not placed in the context of UNCT cooperation to help manage for UNDAF outcomes. Secondly, none of the agency RBM manuals address the emerging trend for agency outcomes to be shared between two or more UN agencies with their implementing partners. This poses some challenges and opportunities for joint monitoring, use of results information to influence decisions about agency resources, and attribution during reporting. There is no agency nor inter-agency guidance for how to deal with these concerns. The harmonized terminology was agreed in 2003 and efforts to communicate it began in 2004. Four years is not a long time to institutionalize a terminology across so many agencies. And many staff have spent their careers using one or another planning system. It takes time to make this kind of institutional change. Ongoing disagreements between agencies at the working level over terminology are a hindrance. It may be creating a reluctance to include government or other stakeholders in inter-agency meetings. But the problem is decreasing over time. A more systemic issue has to do with roles and responsibilities. Roles & responsibilities for UNDAF results monitoring and reporting are not operational Internal Roles and responsibilities for monitoring, review and reporting about UNDAF results are getting clearer. Part 4 and Annex 6 of the 2007 UNDAF guidelines29 highlight the 28 See Annex A for more detail. the development and implementation of RBM systems is still a work in progress. This information is current as at March 2008, but should not be viewed as final or conclusive. Some agencies have several manuals and technical notes and this table may not represent the most current one. 29 UNDG, Common Country Assessment and United Nations Development Assistance Framework, Guidelines for UN Country Teams on preparing a CCA and UNDAF, United Nations, February 2007. 9 importance of inter-agency UNDAF outcome groups to use the RM and M&E plan for joint monitoring and reporting with programme partners. But generally, UNDAF outcome groups are not yet operational. Functioning UNDAF outcome groups would help to rollup usable results information about the UNDAF. Often it is left to an under-staffed RC coordination office to collect results information piecemeal from the agencies. This issue was highlighted during the updating of the 2004 guidelines30 and again in the M&E issues paper31. UNDAF outcome groups are meant to use the M&E framework as an operational monitoring tool and to report on a regular basis to the UNCT about progress. This seems to be happening only rarely, and when it does, without sufficient results information. It still appears that the pressure for agency reporting to HQs and governing bodies takes precedence. Having guidelines that stipulate the need for operational UNDAF outcome groups is not enough. Not when the demands of agency RBM systems still require the lion‘s share of staff time. External From the external perspective, the role and contributions of national and other partners to progress reviews and results reporting is, at best, unclear. Nearly all the agency RBM manuals emphasise the importance of stakeholder involvement. The UNHCR manual is a good example. It positions ―participatory planning involving all stakeholders‖ as a key element of its RBM system.32 However the manuals tend to emphasise stakeholder involvement in planning, less in reporting, and not at all in the use of results information33. The 2007 UNDAF guidelines say that UNDAF outcome groups ―should‖ include government and civil society stakeholders. But it is unclear if this is happening. Field visits and anecdotal information suggest that routine stakeholder involvement is rare. As Balogun argues: ―…creation of an UNDAF Outcome group imposes additional transaction costs on other partners, with little prospect of substantive benefits, since it is not the forum or mechanism which is intended to produce information and decisions relevant to their results and decision making systems. Hence, it isn‘t designed to meet their information demands.‖34 30 MacKenzie, A., Issues and recommendations for the revision of the 2004 CCA-UNDAF Guidelines, Working Paper presented to the WGPP, October, 2006. 31 Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. 32 UNHCR, Results-Based Management in UNHCR, Working Draft June 2006. 10. 33 The consultant could find no guidance in any agency RBM manual for how to involve stakeholders in using results information to influence programme design and delivery. 34 Balogun, P. Ibid., 5. 10 Summary: Immediate causes This section argues that SMART-er UNDAF results and use of results information by the UNCT and stakeholders to manage for UNDAF outcomes depends first and foremost on progress reviews and high quality reports that focus on UNDAF results. This would help to make the UNDAF a ―live‖ management tool. The lack of regular UNDAF progress reviews and results reporting has 3 main causes:  Some ongoing confusion over results terminology and systems; But more importantly:  Low demand for UNDAF results information (internally among UN agencies and externally from country stakeholders);  Un-clear roles and responsibilities for UNDAF monitoring and reporting. To understand the reasons behind these, it is important to look at a range of underlying causes, related mainly to the ownership of UNDAF results. Underlying causes The UNDAF, results matrix, and M&E plan are not owned sufficiently at all levels Ownership of the UNDAF, results matrix, and M&E plan is central to the use of RBM at country level. However a number of concerns suggest that agency ownership is lacking: Too many parallel results frameworks and monitoring plans A number of studies point out a mismatch between UNDAF results and results in agency programme and project documents. Reflecting on the ExCom agencies, Longhurst identified some discontinuity in the flow from UNDAF results, to results in Country Programme Documents (CPDs) and Country Programme Action Plans (CPAPs). He noted that the UNDAF was supposed to make ―heavy planning, strategising and dialogue unnecessary in the preparation of the CDP and CPAP‖. But in practice ―this does not work out‖.35 He sees this as symptomatic of ―skewed incentives between what ExCom agencies [at country level] do for their own HQs and what they do in the name of internal [country level] harmonization.‖36 Results agreed by the ExCom agencies in the UNDAF are often ‗tweaked‘ in CPDs for consumption by their Executive Boards. This retrofitting of results to a corporate results framework was noted by Poate, et al37. This is understandable, given the clear accountability of agencies to their governing bodies. But these tweaked results are then incorporated into CPAPs or other project instruments, and are different from their starting version in the UNDAF. 35 Longhurst, R., Review of the Role and Quality of the United Nations Development Assistance Frameworks, Overseas Development Institute, May 2006, 46. 36 Longhurst, R., Ibid., 20. 37 Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007. 43. 11 For the specialized agencies and non-resident agencies, the challenge is how to incorporate their country level results into the UNDAF when they follow a 1 or 2 year project cycle. A number of evaluations and reports noted that SAs and NRAs often don‘t have sufficient decision-making authority or resources at country level to participate fully in the UNDAF38. And this is combined with a similar adherence to corporate level strategic frameworks as the ExComs. At country level, the result is not one, but several results frameworks, sometimes using different terminology, and often with different indicators. This may help explain why some UNDAF RMs and M&E plans are incomplete. As Longhurst noted, ―…the UNCT ran out of steam when it came to completing the results matrix‖39. It also suggests that the UNDAF results matrix and M&E plans aren‘t perceived as being all that important for agency monitoring and reporting purposes. The UNDAF M&E plan is not operational The WGPP-TT1 has reported that ―M&E plans are often developed as an afterthought40‘. Key weaknesses in M&E plans include too many indicators that are poorly defined or unrelated to the results in the UNDAF. Many agencies have monitoring and evaluation frameworks parallel with the UNDAF M&E plan. In concept, these plans should nest, however they often do not. This creates confusion and competition for staff time and resources. The UNDAF M&E plan is not driving UNDAF monitoring. The 2007 UNDAF guidelines require inter-agency UNDAF monitoring and reporting, but offer no advice or tools to help make it operational - particularly for how to pay for it. Funding, staff time, and incentives to give priority to monitoring and reporting about UNDAF results seems to be in short supply. RCs and UNCTs are mostly left to their own devices to make it work at country level. The M&E issues paper suggests that one way forward is to ―explore opportunities for strengthening the treatment of UNDAF needs in the M&E guidelines of individual agencies‖41. There are no or few performance expectations for the UNDAF UNDAF M&E plans suffer from a lack of baseline data and poor targets. It is also not clear that country staff are sufficiently aware of how what they‘re doing on a day-to-day basis contributes to UNDAF results. The UNEG study argued that alignment of corporate with country level results is unclear42. This implies that there is no recurring sight line between agency programmes and projects and the UNDAF outcomes. This makes it difficult to establish and monitor performance expectations for the UNDAF, and it may 38 For example, see FAO, The Challenge of Renewal, Report of the Independent External Evaluation of the Food and Agriculture Organization of the United Nations (FAO), Submitted to the Council Committee for the Independent External Evaluation of FAO (CC-IEE), September 2007. para 816. 39 Longhurst, R., Review of the Role and Quality of the United Nations Development Assistance Frameworks, Overseas Development Institute, May 2006, 15. 40 WGPP-Task Team 1, Results based management in UNDAFs, Issues Note, October 2007, 7. 41 Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. 5. 42 UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007, para 19. 12 explain why so few country level staff have their inter-agency monitoring and reporting responsibilities included in performance appraisal instruments. Senior managers don‘t have sufficient understanding of how to apply RBM The literature suggests that senior managers at the country level don‘t have sufficient understanding about RBM – particularly how to use results information, with stakeholders, to influence programme strategy and design. From the Dalberg review: ―Country level managers face challenges in implementing RBM on a day to day basis. Results management seems to plays its most significant role in goal setting and reporting, but not on a day-today operations basis. There is a need to strengthen and/or generalize the use results for guiding Country office operations.”43 Few of the agency RBM manuals help with this. The manuals focus on the planning elements: definition of results, the causal results chain, the importance of measurement and of indicators. But they are mostly silent on how results information from agency RBM systems should be used to influence programme decision-making in the interagency context of the UNDAF, and with stakeholders. Results are not often linked to a human rights-based analysis Most agency RBM manuals and guidelines establish that results are identified from a situation analysis. However, apart from the UNIFEM44 and UNICEF45 materials, no other agencies make the explicit link between a human-rights based analysis and the formulation of results. Recent inter-agency trainings on a Human Rights Based Approach to programming and Results Based Management (HRBA-RBM) have succeeded in making this link, and demonstrating that results which flow from a human rights based-analysis can be SMART-er, than results which do not46. This is because a HRBA, comprising causality analysis, role analysis and capacity gap analysis makes explicit:  The reasons why some people have been left behind in national development;  The specific changes needed for human rights to be realized (both outcomes and processes);  Those responsible for making the changes; and  The new capacities they need for succeeding. This means that results statements benefit from a clear subject of change (the who?), as well as the substance of change (the what?), that are linked to human rights standards. 43 44 45 Dalberg, Assessing Results-Based Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006, 21. UNIFEM, Results-Based Management in UNIFEM: Essential Guide, UNIFEM, February 2005. 4. UNICEF, Understanding Results-Based Programme Planning and Management, Tools to reinforce good programming practice, UNICEF, May 2005. 12. 46 For example, see recent UNSSC workshop facilitation reports from inter-agency HRBA-RBM trainings in Angola (July 2007), Nigeria (July 2007), and Turin (January 2008). 13 Summary: Underlying causes Ownership of the UNDAF, results matrix, and M&E plan is central to the use of RBM at country level. However the literature review points to a number of concerns that suggest that agency ownership is still not sufficient:  There are still too many competing results frameworks and monitoring plans at country level;  The UNDAF M&E plan is not operational;  There are few performance expectations for UNDAF results; and  Senior managers at country level aren‘t practiced at using results information to used to influence programme decisionmaking in the inter-agency context of the Box 1: A Weaker Culture of Results UNDAF and with stakeholders. As UN staff become more familiar with the UNDG approved terminology and systems, resultsbased planning will continue to improve and results will become SMART-er. But without a commitment to use UNDAF results information for management purposes, the UNCT‘s use of RBM at country level will remain mostly mechanical. Overall, this situation suggests that results information about the UNDAF is generally not seen by country staff as being central to good management and delivery. This points to the issues of organizational systems and incentives. These root causes are examined in the next section. An organization with a weaker culture of results might, for example,  Gather results information, but limit its use mainly to reporting  Acknowledge the need to learn, but not provide the time or structured occasions to learn  Undergo change only with great effort  Claims it is results-focused, but discourages challenge and questioning the status quo  Talk about the importance of results, but frown on risk taking and mistakes  Talk about the importance of results, but value following process and  Focus on delivering outputs Source: Mayne Root causes The external evaluations make clear that UN agencies are struggling with their own RBM systems and that most have weaker cultures of results47 (see box 1). RBM systems are still viewed by many country level staff as externally imposed things that need feeding, with a heavy emphasis on planning and reporting and much less on managing for results. The UNEG study pointed to the emergence of ―multi-layer RBM systems and structures in some organizations, with a distinction between RBM in the field and RBM in headquarters‖48. This distinction between RBM in the field and in headquarters makes it very difficult for country staff to focus on managing for UNDAF outcomes. To understand this we need to look at: 47 48 See the Dalberg evaluations, the Poate et al. Evaluation of UNDP, as well as the UNEG study para 21. UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007. paras 14, 29-30. The UNEG study found that while a majority of organizations have introduced RBM, only a third had established comprehensive corporate RBM frameworks, covering all aspects of RBM, including roles and responsibilities 14    Agency RBM systems, incentives and procedures; The accountability frameworks that agencies use; and Corporate management leadership and commitment to managing for UNDAF outcomes. Agency RBM systems do not focus on UNDAF results RBM Guidelines and Manuals Generally there were two distinct trends in the agency RBM guidelines and manuals: First, few agency RBM manuals addressed the issue of how to use results information to improve management decision-making about results and the use of resources. Most materials amply reviewed results-based planning, the logic of a SMART results chain, the critical role of indicators, and the importance of risks and assumptions. But, overall there was very little on how to use results information. The UNFPA and UNICEF RBM manuals stood out as having a substantial section on how to use performance information for decision-making and reporting. The UNFPA manual emphasizes the consideration of strategic trade-offs between ―programme results, reach, and resources‖49. However, the guidance is at a very high and theoretical level, with no examples from the field for how results information actually led to changes in programme design. The UNICEF RBM manual contains clear messages about keeping ―all financial resources and staff focused on the… results agreed with programme partners‖ and the use of its Annual Management Plan (AMP) which links results in the Annual Work Plans (AWPs) to annual office priorities, staff assignments, and performance review50. But, overall there was very little on how to use results information at the country level. And this leads a second major point. Second, few agency RBM manuals and systems address the practice of RBM within the wider UNCT environment, with a focus on managing for UNDAF outcomes. Few of the manuals addressed RBM in the context of the UNCT and UNDAF51. Here, the UNICEF manual stood out as having a substantial section on how UNICEF-supported results fit into the overall framework of the UNDAF, and the importance of joint decision-making within the UNCT to ―get the UNDAF right‖ and the possibilities for joint projects and programmes.52 But overall, the reach of agency RBM systems to the UNDAF seems to extend mainly to the planning elements. Like the UNDAF guidelines, agency RBM manuals and guidelines are mainly silent about how to operationalise the UNDAF monitoring and evaluation. And how to use results information, in the inter-agency context, to influence decisions about the use of UNCT resources. Annual Reports 49 50 UNFPA, Results-Based Management Orientation Guide, June 2001. section 3, 3.4. UNICEF, Understanding Results-Based Programme Planning and Management, Tools to reinforce good programming practice, Updated May 2005. 26-28. 51 A word search of agency materials shows almost no mention of the UNDAF, as the results framework at country level. 52 Ibid., In addition the UNICEF RBM manual offers a number of worked, but realistic country examples to help clarify how the contributions of multiple partners are necessary for the achievement of outcomes. 13. 15 Agency guidance materials for annual reports suggests that a culture of results is emerging, but the focus is primarily on programme and project results, and their contribution to organizational priorities53. There is not much consideration of how results information from the agencies is contributing to a better understanding of progress towards UNDAF outcomes overall. Several agency guidelines for annual reporting have no mention of the links between agency-supported results and UNDAF outcomes: OHCHR; UNFPA; UNDP (CORR description); and WFP. UNCHR reports54 are to address the specific objectives and activities of UNHCR within the agreed wider UN country operation. But it goes no further. Overall, it appears that annual reports are viewed primarily as a reporting tool and data source for regional offices and headquarters, for onward reporting to governing bodies55. Box 2: The results section in the UNICEF COAR The UNICEF guidance for  Title of the programme component (per CPD) preparation of the Country Office  Code for the main MTSP focus area addressed Annual Report (COAR) provides an  Codes for the main MTSP key results area interesting example56. It is very much  Description of the programme purpose, results in line with good RBM practice, AND planned, and how it relates to the UNDAF, it asks the CO to reflect on a number SWAps, PRSP/National development plan  The resources used from different budget of important issues related to sources achievements and shortfall against UNDAF results and the performance The actual results achieved, as follows: of the UNCT. But, the main results  A critical assessment of progress against the section of the report loses some of results planned and progress towards the higherlevel results in the CPAP this focus. The flow is described in  the most critical constraints affecting box 2. Results information is performance and any lessons learned immediately classified in terms of  A summary of monitoring, study and evaluation the corporate plan priorities. The key methods which were undertaken, how the results question of whether results are of these M&E efforts were used  Key partnerships and interagency collaboration, contributing to or influencing including UN joint programme/project initiatives if something higher, is mainly in applicable. relation to organizational plans. The focus is not about whether UNICEF Source: UNICEF, Guidelines for the 2007 supported results are actually Country Office Annual Reports contributing to the outcomes in the UNDAF. CO staff are asked to describe the relationship with the UNDAF and national plans, but it goes no further than a narrative of the plan. There seems to be an implicit assumption that so long as agency results are planned to contribute to the UNDAF, then it is enough. The weight of evidence about results is classified and directed primarily at corporate results. It must be emphasized that this is not unique to UNICEF. 53 54 Guidelines were available for 6 agencies: OHCHR, UNDP, UNFPA, UNHCR, UNICEF, and WFP. See: OHCHR, Guidelines for 2007 Internal Annual Reports – Field Offices, OHCHR, 30/08/2007; WFP, Annual Reporting Exercise, User Reference Manual, WFP, 2007; UNHCR, Instructions and Guidelines to UNHCR Field Offices and Headquarters Units on Reporting on 2007, Implementation in 2008 and Planning for 2009, UNHCR, 2007. 55 The evaluation of UNDP found that the ROAR [now Country Office Results Report (CORR)] ―has become primarily a tool for reporting to senior management – of little use for country office or regional management. See Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007. ix. 56 UNICEF, Guidelines for the 2007 Country Office Annual Reports, CF/EXD/2007-009, UNICEF, 16 November 2007. 16 IT systems The UNEG study reported that in most organizations RBM is supported partially by electronic tools. But the systems are ―not integrated to form a comprehensive RBM supporting platform‖57. Overall, the impression is of stand-alone applications not widely used by managers and staff to manage for outcomes. For example the evaluation of RBM at UNDP found that IT systems have been designed mainly to meet the demand for data for reporting to the Executive Board rather than to manage for outcomes UNDP58. But it is important to note that the evaluators are referring to outcomes in the UNDP strategic plan. If this situation is common to most agencies, IT systems and platforms are being developed to roll-up results information with a focus on corporate strategic results. This will make it harder to get a view of how agency results are contributing to UNDAF outcomes and country priorities. Agency accountability frameworks do not focus on UNDAF results Mayne argues that ―incentives are more important than capacities in institutionalizing RBM59‖. A DFID report on multi-lateral organizations, has argued that UN agencies need to ―…amend their internal incentive structures in favour of results‖60. But which results? Is the UNDAF the strategic results framework for the UN at country level, in response to national priorities? A range of inter-agency policy documents and instruments say yes, particularly the 2007 TCPR resolution, the Resident Coordinator System Mutual Accountability Framework, and the 2007 CCA and UNDAF guidelines61. But the current focus of agency RBM systems would suggest otherwise. When it comes to RBM at the country level, the problem is not the lack of a framework, but rather which framework drives the RBM system and creates demand for information about results. For RBM to be used for the achievement of UNDAF outcomes, agency accountability frameworks must place the emphasis on achieving UNDAF outcomes. However, most agency accountability frameworks, emphasise corporate results. There is some variation in this. Some agencies have a strict line of accountability for corporate results to headquarters and governing bodies. Others use a blended accountability that makes room for both corporate results as well as for country results. UNHCR is typical of the former. Accountability is defined entirely at the organizational level in ―a clear line responsibility…which runs from ECOSOC and the UNHCR Executive Committee, through the High Commissioner to representatives of country 57 58 UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007. paras 27-28. Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007, xii. Mayne, J., Ibid., 14. 60 Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007, 40. 61 Most notably: TCPR 2007, para 86 (A/C.2/62/L.63); TCPR 2004, paras 49-50 (A/RES/59/250); Resident Coordinator System Mutual Accountability Framework, 2007; CCA and UNDAF guidelines, 2007, para 61; Report of CEB Retreat, Oct 2004, paras 15,17. 59 17 offices. There is no mention of country level accountability for results either to the UNCT or to national stakeholders 62. UNDP is an example of the latter. UNDP identifies organizational accountability for corporate results to its Executive Board, as well as programmatic accountability for country level results. The convergence of these two is intended to yield a single integrated system for strategic planning. But there is a tug-of-war. The UNDP strategic . plan makes clear that ―UNDP operations are anchored in the development plans and priorities of programme countries. The plan also states that ―…results-based operational plans at the country office are ―derived from the strategic plan, [and] drive the allocation of resources‖63. Across all agencies, the emphasis is clearly weighted towards accountability for corporate results to governing bodies. The Dalberg assessment mapped RBM tools in UNFPA. The original diagram is presented in Figure 264. The MYFF and UNDAF inform the formulation and planning of the country programme. But the key point here is that the agency strategic framework and the UNDAF do not equally inform the formulation of agency programmes and projects. In the figure below, the arrows are clearly centered on the MYFF. The UNDAF is on the periphery. There are no in-flow or feedback arrows to the UNDAF. It is critical to point out that this pattern is not unique to UNFPA. This diagram was chosen because it best illustrates the core of the problem that seems to prevent greater use of RBM at country level. This was best summarized by Longhurst who highlighted the existence of ―excessive, overlapping planning and double/contradictory directives‖ about the importance of country priorities vis-à-vis agency‘s priorities65. In addition the Dalberg studies and other evaluations have encouraged agencies to develop standardized, attributable project level results and indicators. This would allow for ―aggregation of key indicators to a corporate level and enable comparison across countries, regions and over time‖66. But no mention is made of the need to aggregate for UNDAF performance measurement. There is a risk that this standardization will only further cement the flow of results information away from country users and uses. 62 UNHCR, Results-Based Management in UNHCR, Working Draft June 2006. 13. UNDP, Strategic Plan, 2008-2011 Accelerating global progress on human development, DP/2007/43, UNDP, 2007. para 47& 129. 64 Dalberg, Assessing Results Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006. 12. 65 Longhurst, R., Review of the Role and Quality of the United Nations Development Assistance Frameworks, Overseas Development Institute, May 2006, 49. 66 Dalberg, Assessing Results-Based Management at UNICEF, Danish Ministry of Foreign Affairs, April 2007. 18-20. 63 18 Figure 2: Key performance management elements at UNFPA Accountability for outcomes – especially shared outcomes - is unclear A related issue here concerns accountability for outcomes, particularly when outcomes are ―owned‖ by two or more agencies and their implementing partners. The WGPP has adopted a stance where agency accountability rests primarily at the output level. But the use of RBM requires a focus on managing for outcomes. At country level, this often means managing for shared agency and UNDAF outcomes. However, there is no interagency consensus on how to do this. The point is made by the UNDG‘s Task Team 2 in its report on lessons from the ‗One UN‘ pilots in 200767. The team highlighted the issue that ―…the UNDAF is conceived as a framework and was not originally intended to include implementation‖, and that the ‗one programme‘ is helping to operationalise the UNDAF. But for most UNCTs, there is little help to make the UNDAF operational. Most agency RBM guidelines are silent about how to manage for shared agency and UNDAF outcomes. This is also fuelled by concerns over attribution and reporting. The literature on accountability for outcomes recognizes this as a challenge, and suggests that organizations move towards a system based on accountability for influencing outcomes, rather than achieving them outright. UNFPA is reported to be moving towards this approach68. 67 UNDG, Final Report: Task Team 2 Lessons Learned on the One Programme, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. 4-5. 68 Mayne, J., Ibid., 15-16; Poate, D. et al. Ibid., 92-95. 19 From the preceding discussion, it seems that the UNDAF is viewed mainly as a planning reference, but it does not figure prominently in decision making about organizational results and resources. As such, there are few incentives for SMART-er results and better results information. The message seems to be that, where the UNDAF is concerned, these are nice things but hardly essential. The UNDP evaluation69 addressed this issue head-on: UN agencies… ―face a potential tension between having a corporate headquarters vision of the organization and the need for its country offices to design programmes reflecting national priorities. This tension is highlighted even more in a results-based approach to planning and managing. Efforts to set clear and measurable intended results at the corporate level for [the agency] could conflict with efforts at the country level to set clear results that reflect national priorities. The challenge is to find the right balance.‖ In order to find this balance, agencies may have to consider how to adapt their strategic results frameworks, and related systems and procedures, to meet both country and corporate information needs. This was a major recommendation of the evaluation of RBM at UNDP70: ―The current practice of setting corporate outcome-level objectives and indicators within the strategic plan should end. Instead, outcome objectives and indicators should be set at the country programme level, where they should be linked to UNDAF outcomes….‖ Corporate leadership and commitment for managing for UNDAF results There are numerous examples of high level statements by the leadership of UN organizations about the need for greater harmonization and effectiveness at country level. The summary of results from the 2006 RC and UNCT performance appraisal system, including the 180° assessments, demonstrates a clear commitment to collectively assess country performance against planned results in the UNDAF71. The evaluations and studies reviewed for this study also indicate a high or growing level of commitment to RBM within each agency. But agency RBM systems appear to be focused primarily on generating results information about corporate strategic results, aimed primarily for an audience outside the country. This suggests that information about UNDAF performance, is seen to be relatively less important. Once indicator of this situation is the difficulty to agree on an accountability framework and job description for the Resident Coordinator (RC) system. The 2007 TCPR requests UN agencies to use the UNDAF as ―the common programming tool‖ at the country level. The draft proposal for the RC job description72 makes clear that the UNDAF is the main strategic framework for most of the UNS operational activities for development, that 69 Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007, 91. Poate, D. et al., Ibid., UNDP, 2007. xiv. 71 UNDG, Summary Overview of the Resident Coordinator and UN Country Team Performance Appraisal System, 2006 Summary, UNDG, 2007. 72 UNDG, Proposal on RC Job Description, Confidential DRAFT Feb 08. 70 20 UNDAF results must make a substantive and measurable contribution to the achievement of national priorities, and that the RC will: ―Undertake annualized reporting on the progress of the UNCT against the respective commitments to the UNDAF results, based on a comprehensive accountability framework contained in the results matrix and operationalized through a Monitoring and Evaluation Framework and Plan, included in the UNDAF.‖ Clear, unambiguous leadership and accountability of the RC and UNCT for UNDAF results will help to drive demand for information about UNDAF performance. The challenge here is how to do this when most of the agencies that make up the UNCT have their own accountability framework, and purpose-built RBM systems, geared mainly towards the information needs of their headquarters and governing bodies. This tension is obvious in the Resident Coordinator Mutual Accountability Framework73. Two overarching principles of the framework emphasize (1) accountability for development results [UNDAF results] shared by the RC and UNCT members, as well as (2) responsibility of UN system organizations for their respective results and resources, consistent with previous arrangements. From the preceding analysis, the balance is clearly tilted towards organizational results and reporting systems. This discussion is made more challenging because the RC system is managed by UNDP, and efforts to establish a ―firewall‖ between the RC‘s office and UNDP are just beginning. 4.0 Conclusion and preliminary recommendations for the Road Map This study examined the systemic issues that affect the use of results-based management (RBM) at country level. The problem is that UNDAF results are not yet SMART enough, and results information is not being generated and used effectively by UNCTs to manage for the achievement of UNDAF outcomes. At the immediate level, UNCTs are having difficulty to produce sufficient, regular evidence about how well the UNDAF is going. This is because the UNDAF results matrix and M&E plans are not very operational. They are often seen as a one-off products necessary for programme planning, but not very essential for day-to-day programme management. This isn‘t likely to change until UNDAF performance is seen by the UNCT and UN staff as central to agency and individual performance. This is difficult when the agencies have their own accountability framework, and purpose-built RBM systems, geared mainly towards their corporate strategic plan and the information needs of their headquarters and governing bodies. Agency RBM systems aren’t helping to generate coherent information about UNDAF performance because they aren’t designed to do it. 73 UNDG, Resident Coordinator System Mutual Accountability Framework, UNDG, 2007. 21 For RBM to be used more effectively at country level, agency strategic plans and accountability frameworks must place greater emphasis on managing for UNDAF outcomes. This kind of systemic change will help to create incentives for SMART-er UNDAF results and demand for information about UNDAF performance. The detailed causes from this situation analysis are listed below with some preliminary recommendations for the road map. Immediate Causes 1. UNDAF annual reviews and reports don’t have a results-focus, and the results matrix is not live The guidelines for UNDAF ARs and the RCAR have a results focus, but there appears to be little inter-agency support nor an oversight mechanism to help UNCTs manage for the achievement of results. Do most UNCTs have meetings in which they regularly review progress towards agency and UNDAF outcomes? More importantly do they ever discuss together, and with national authorities, how to adjust their available resources to better achieve results. To support this, agency results information needs to start rolling-up in a coherent and usable way for use by the UNCT. Recommendations:  Clarify whether the RCAR is to focus on UNDAF results or coordination processes.  Pursue the introduction and piloting of one annual UN progress report (whether the RCAR or other). It should use the UNDAF M&E plan as its frame of reference, operationalise the role of UNDAF outcome groups, and help to aggregate results information from contributing agencies. The Standard Progress Report (SPR), which is a common tool for the ExCom agencies, may provide a useful format. The SPR is also recommended for reporting on joint programmes Efforts have been made at country level to introduce this approach and a format, but with variable results74. It is made difficult by the mandatory reporting obligations of agencies to their headquarters and governing bodies, and by the different reporting systems that agencies use to capture country level data. The key challenge is how to generate a coherent and usable picture of UNDAF performance while also satisfying corporate information needs, and without creating an additional reporting burden for UN staff and their partners. It is critical that this report, focusing on UNDAF results, not be done in parallel with existing agency annual reports. The key issues were raised by Task Team 2 and are repeated here for emphasis: 74 See the discussion in UNDG, Issues paper: Common Annual Reporting by Pilot UNCTs, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. 22 Box 3: Key issues for common reporting:  A minimum common (central) purpose to the report agreed by agencies – allowing it to meet agreed reporting standards and requirements but potentially to be supplemented through annexes or some other means to meet any agency specific requirements without having to report separately A clear indication of how it could meet government requirements on reporting: since these requirements may vary in different countries, will there need to be some flexibility? A clear view on the role of government systems in reporting – if the UN‘s programme is nationally owned, then it would be logical to use national systems to report, if they are well enough developed. If they are not, then part of the UN‘s role is to support their development The role of any common report for the purposes of donor reporting UNDG, Issues paper: Common Annual Reporting by Pilot UNCTs, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. 4-5     Responsibility for the quality of UNDAF progress reviews and reports appears to rest with the Regional Director‘s Teams (RDT) and their regional peer support groups (PSGs). They have the potential to demand more coherent information about UNDAF performance, and to encourage and support UNCTs to use results information for UNCT decision making. Consult with that group to see what more can be done to help UNCTs on an ongoing basis to manage for UNDAF outcomes. This would mean coaching and support to help UNCTs: - Monitor financial commitments, per the UNDAF; - Achieve outputs and track that they are indeed contributing to agency outcomes; - Monitor UNDAF outcome indicators75 - Engage more regularly and effectively with national authorities to discuss UNDAF performance and how to better align UNCT and government resources for greater effectiveness. It is important to note that regular coaching and support from RDTs or PSGs will likely require additional resources  The accountabilities of agencies at country level to report to the UNCT about agency results need to be made clearer. For example, the accountability framework establishes that ―The RC will have the responsibility to monitor implementation of the UNDAF and to report on UN agencies‘ progress against their commitments76‖. But are agencies accountable for sharing results information with the RCO. Annex 6 in the 2007 UNDAF guidelines do not establish any agency accountability for this. 75 76 Adapted from Poate, D., Ibid, 93. UNDG-RCIG, Principles for Enhancing the Leadership Role of the Resident Coordinator for UN Operational Activities for Development and Accountability Framework of the Resident Coordinator System, March 2006. 6. 23 2. There isn’t enough routine internal or external demand for information about UNDAF results Internal Demand A move towards one annual UN progress report would help generate greater demand and focus for information about UNDAF performance. But the lack of internal demand derives mainly from agency RBM systems that are focused on generating results information about corporate priorities. This is addressed below. External Demand Recommendations:  Efforts are needed to create greater ownership of the UNDAF among stakeholders and they need to play a stronger role in UNDAF outcome groups. The M&E issues paper has recommended that UNDAF results systems be better aligned with those of governments to produce information that is relevant for their results and decision-making systems77. This should be pursued. Situations where DevInfo has been adopted by governments may serve as an example of this. However, this analysis also makes clear that alignment of UNDAF results systems may not be enough. Agency RBM systems will also have to become more relevant to country results and decision-making systems  Agency RBM manuals and guidelines should establish clearly that country stakeholders – especially national authorities – are the intended audience for country results information. 3. Different RBM Terminologies and Approaches Recommendation: There are still some discrepancies between UNDG and agency RBM terminologies and approaches. Agencies should update their existing RBM manuals and guidelines to:  Address any remaining differences from the UNDG-approved terminology.  Deliver some top-line message about the importance of using results information to manage for UNDAF outcomes.  Distinguish between UNDAF outcomes and agency outcomes, as per the 2007 UNDAF guidelines.  Address the emerging trend for agency outcomes to be shared between two or more UN agencies with their implementing partners. This should be 77 Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. 4-5. 24 accompanied by guidelines and good practice examples for joint monitoring, and joint use of results information. 4. Roles and Responsibilities for UNDAF results monitoring and reporting are not operational The UNDAF guidelines are clear about the importance of operational inter-agency UNDAF outcome groups. The functioning of these outcome groups depends more on whether the UNDAF is owned or not, and the root causes addressed below. Underlying Causes 5. The UNDAF, results matrix, and M&E plan are not owned sufficiently at all levels A number of concerns are raised:  There are still too many competing results frameworks and monitoring plans at country level;  The UNDAF M&E plan is not operational;  There are few performance expectations for UNDAF results; and These are related. The 2007 guidelines have attempted to address the issue of different and competing results frameworks by stipulating that results in the UNDAF must be identical to results in agency programme and project documents. Inter-agency training at the country level, led by UNDGO and UNSSC (Staff College), deliver this message continuously. However the ―skewed incentives‖ identified by Longhurst between country needs and the demands of headquarters continue. So long as agencies at country level have to ―interpret‖ UNDAF results in separate documents for approval by their governing bodies, there will be a risk of several parallel and competing results frameworks, parallel M&E plans, and few incentives to establish clear baselines and targets for the UNDAF results matrix. A proposal was made in 2005 by the UNDG to radically simplify the process for the ExCom agencies with a single consolidated programme document. This is still the best avenue for addressing the proliferation of separate results frameworks. Recommendation:  The TT should discuss with the wider WGPP about how to revive the proposal for radical simplification, and make it inclusive of the specialized and non-resident agencies.  More immediately, agencies should instruct their country offices to ensure that staff responsibilities for UNDAF outcome monitoring and reporting are included in their performance appraisal instruments. Another concern is that:  Senior managers don‘t have sufficient understanding about how to apply RBM. 25 The external evaluations of agency RBM systems have all recommended greater investments in RBM training for county offices. Interagency training could follow the model used by Action 2, where an inter-agency task force develops a common learning package and learning and training support is provided by UNSSC (Staff College). The training could be adapted to different contexts and audiences. The RC Induction Workshop and the combined orientation for senior agency representatives are ideal opportunities to provide a more detailed orientation on RBM. Recommendation:  There should be inter-agency RBM training, and the focus should shift from an emphasis on planning, to an emphasis on the use of results information to assist UNCT decision-making. The training package should link HRBA with RBM, based on the existing training package available from Staff College.  Agency RBM manuals and guidelines should all make an explicit link between a human rights based analysis and the formulation of results. Root causes 6. Agency RBM systems do not focus on UNDAF and country results Agency RBM systems need to be re-oriented towards country results, country users and uses. The M&E issues paper has suggested that UN agencies must consider how to strengthen the treatment of UNDAF needs in their systems and guidelines78. A more ambitious, post-Paris reorientation of RBM systems would see governments included as a key user of results information and greater alignment with national results systems. Recommendation:  Agency RBM manuals need to provide more guidance and examples for how to use results information to improve management decision-making. This should be placed in the context of managing for UNDAF outcomes and country results.  Until one annual UN progress report is introduced, agency guidelines for annual reports need to be reviewed and re-oriented towards generating results information for the UNCT and stakeholders. Their focus needs to be on whether agency-supported results are actually contributing to UNDAF outcomes, and to the achievement of national priorities.  IT systems and platforms appear to focus on generating information for reporting to headquarters and governing bodies. These should be re-examined to see if they can also focus on consolidating results information about UNDAF performance. 78 Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. 5. 26 7. Agency accountability frameworks don’t focus on UNDAF and country results Agency programmes and projects are not driven by UNDAF priorities. They are much more influenced by corporate priorities. There is a pressing need to re-orient agency accountability frameworks to focus more on UNDAF performance for a country audience. This was highlighted by the UNDG‘s Task Team 2 which, in its review of country experience with ‗one programme‘ in 2007 found that the UNDAF ―is not an operational document and therefore does not include clear accountability agreements‖79. Agencies will always maintain accountability to their governing bodies, but this needs to be balanced with accountability for UNDAF and country results. This is in line with recommendations from several agency evaluations. The Dalberg study at UNFPA urged that the corporate results framework ―take into consideration‖ the UNDAF outcomes defined at country level, and provide ―clearer evidence of the organization‘s attributable contribution to development outcomes.‖ 80 The external audit of RBM at WFP recommended the incorporation of joint outcomes with implementing partners into the RBM framework81. And the evaluation of RBM in UNDP has urged that outcomes and indicators for the strategic plan should be set at the country programme level. To do this, UN agencies will also have to resolve their concerns about accountability for shared outcomes. Recommendation:  The TT should encourage the CPSG to initiate a discussion amongst the agencies, funds, and programmes about the need to distinguish between, and balance, corporate results and UNDAF/ country results82. It should be positioned as part of the UNDG‘s ongoing response to the Paris Declaration.  UN agencies should move towards a system based on accountability for influencing outcomes, rather than achieving them outright.  Each agency will have to figure out how to adjust its IT systems to balance the needs for UNDAF results information at country level with corporate results information at HQ level. These recommendations are preliminary but they imply far-reaching changes to agency strategic plans and RBM systems. This is no small matter, particularly when several agencies have just recently issued new multi-year strategic plans. 79 UNDG, Final Report: Task Team 2 Lessons Learned on the One Programme, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. 5. 80 Dalberg, Assessing Results Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006, 4 and 26. 81 WFP, Review of the Results-Based Management Framework for monitoring and reporting results: Report by the WFP External Auditor, WFP/EB.A/2006/6-D/1, WFP, 5 May 2006. 82 Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007, xiii and . 27 However, the strategic plans of UNFPA and UNIFEM for 2008-201183 may offer a useful way forward. They define a development results framework, focused on country level outcomes, as well as a management results framework, focused on organizational performance. Similarly, the UNICEF strategic plan (MTSP) includes measurable outcomes that provide strategic direction to country offices and key performance Indicators which track UNICEF‘s performance across a series of financial, operational and human resource outputs. 8. Corporate leadership and commitment to managing for results The external evaluations have all recommended ―explicit efforts by senior management to foster country office buy-in to the use of results management approaches tools‖84. These messages are needed, but the focus should be on managing for UNDAF outcomes. This would also focus greater attention on leadership for results at the country level, and create greater demand for information about UNDAF results. The accountability framework and job description for the RC needs to be finalized, communicated and made operational. Shifts in agency RBM systems and accountability frameworks towards managing for UNDAF outcomes (see above) will help make the RC job doable. Without these shifts, agency systems will continue to push results information to users outside the country. And RCs will continue to rely on moral suasion and arm twisting to get information about UNDAF performance. 83 UNIFEM, Strategic Plan 2008-2011, DP/2007/45, Sept, 2007; UNFPA, Strategic Plan, 2008-2011: Accelerating Progress and National Ownership for the ICPD Programme of Action, DP/FPA/2007/17, July 2007. 84 Seeee Dalberg, Assessing Results Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006, 29. 28 Annexes Annex A: Mapping of Agency RBM Systems Notes: This study is based on a desk review of major studies and evaluations about RBM in both the inter-agency arena, as well as for 13 organizations: ILO, FAO, UNAIDS, UNDP, UNESCO, UNFPA, UNHCR, UNICEF, UNIDO, UNIFEM, WFP, UN Secretariat, and OHCHR. The type of information in this Annex and its presentation vary greatly. This is because of the wide variety of sources that were used, including:  External evaluations and studies;  Internal studies and reports; and  Both agency and inter-agency guidance materials. Efforts were made to highlight and collate the most illuminating pieces of information about agency directions in RBM. The external evaluations often provide the most compelling pieces of information. But not all agencies have undergone one, and this may make the information appear lop-sided. As this study demonstrates, the development and implementation of RBM systems is still a work in progress. This information is current as at March 2008, but should not be viewed as final or conclusive. It should be considered a working document for this situation analysis and for the road map. Instances where ―UNDG definition‖ appears does not mean that agencies have adopted the UNDG definition verbatim, but that the meanings are essentially the same. 29 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG85 Results Based Management (RBM) A management strategy by which an organization ensures that its processes, products and services contribute to the achievement of desired results (outputs, outcomes and impacts). RBM rests on clearly defined accountability for results, and requires monitoring and selfassessment of progress towards results, and reporting on performance. UNDP86 87 The objective of RBM is to ―provide a coherent framework for strategic planning and management based on learning and accountability in a decentralised environment. Introducing a results-based approach aims to improve management effectiveness and accountability by ―defining realistic expected results, monitoring progress toward the achievement of expected results, integrating lessons learned into management decisions and reporting on performance‖. UNFPA88 89 90 Results-Based Management (RBM) is a participatory and team-based approach to management designed to improve programme and management effectiveness, efficiency and accountability, that focuses on achieving defined results. UNICEF 91 Results Based Programme Planning ensures that the sum of interventions is sufficient to achieve the expected result. Results Based Programme Management ensures that all available financial and human resources continue to support the planned results. WFP 92 93 RBM introduces a structured management approach designed to keep an organization clearly focused on its expected results throughout the management process. Communicated as: Results-Based Management is a common sense idea: plan, measure, and manage what you do with a clear eye on the results you want to achieve. Results Framework/ Chain The logic that explains how results are to be achieved, including causal relationships and underlying assumptions. The results framework is the application of the logframe approach at a more strategic level, across an entire organisation, for a country programme, a programme component within a country programme, or even a project. Planning outline Goal The higher-order objective to which a development intervention is intended to contribute. UNDG definition ** ** Please note that “UNDG definition” does not mean that agencies have adopted the UNDG definition verbatim, but that the meanings are essentially the same UNDG definition UNDG definition UNDG definition The Strategic Plan has a goal for each of the four focus areas. These goals are essentially ―the higher order objective to which UNDP development interventions are intended to contribute‖ and are based on helping countries to achieve the MDGs. A basic condition of well being for individuals, families and communities. It reflects the long-term results expected from a programme. It is the highest level of result to which UNFPA contributes, together with the efforts of other partners. A Strategic Result (or goal, intended impact) describes the expected change in the lives of people, especially children and women. It provides direction for the overall programme. A Strategic Result would, for instance, directly relate to key national priorities aligned with the Millennium Declaration and The highest level result to which a WFP operation is intended to contribute. It is measured by impact indicators. 85 86 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNDP, RBM in UNDP: Technical Note, Knowing the What and the How 87 UNDP User Guide, Programme and Operations Policies and Procedures – Results Management and Accountability, 2/15/2008. 88 UNFPA, Results-Based Management Orientation Guide, June 2001. 89 Dalberg, Assessing Results Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006. 90 UNFPA, Strategic Plan, 2008-2011: Accelerating Progress and National Ownership for the ICPD Programme of Action, DP/FPA/2007/17, July 2007. 91 UNICEF, Understanding Results-Based Programme Planning and Management, Tools to reinforce good programming practice, UNICEF, May 2005. 92 WFP, RBM Orientation Guide, WFP, October 2003. 93 WFP, Mainstreaming Results-Based Management at the World Food Programme – The Way Forward Road Map, GDE Consulting, January 2006. 30 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG85 UNDP86 87 Also… At the country level, the highest level goal is the national goal or priority, to which UNDAF outcomes are related. UNFPA88 89 90 UNICEF 91 the MDGs. Also communicated as ―child level results‖ WFP 92 93 Impact: Positive and negative long-term effects on identifiable population groups produced by a development intervention Equated with National priority/ Goal in the results chain Communicated as: A measurable, achievable, sustainable change in the lives of people Outcome The intended or achieved short-term and medium-term effects of an intervention‘s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact. Communicated as institutional or behavioural change at national or sub-national levels, to be achieved at the end of the UNDAF cycle Outputs The products and services which result from the completion of activities within a development intervention. UNDG definition na UNDG definition Equated with above UNDG definition UNDG definition ―As a rule of thumb, an Outcome statement should ideally communicate a change in the behaviour or quality of life for people – however modest that change may be‖ UNDG Definition While UNFPA uses the harmonized UNDG terminology at country level and in its strategic plan, the RBM manual from 2001 still refers to purpose at the outcome level. This is a minor matter, but may lead to some confusion among field staff. UNDG definition Also communicated as: - programme results - institutional change, quality, coverage of services, behaviour change UNDG definition UNDG definition UNDG definition UNDG definition Also communicated as: - results of completed work plans and activities - products or change directly attributable to project funds and management UNDG definition UNDG definition Activity Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs. UNDG definition UNDG definition UNDG definition 31 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG94 Results Based Management (RBM) A management strategy by which an organization ensures that its processes, products and services contribute to the achievement of desired results (outputs, outcomes and impacts). RBM rests on clearly defined accountability for results, and requires monitoring and selfassessment of progress towards results, and reporting on performance. FAO95. RBM is a management approach focused on achieving results; a broad management strategy aimed at changing the way agencies operate, with improving performance (achieving results) as the central orientation ILO96 97. The central principle of results-based management is that an organization must manage and measure its performance against the real-world outcomes to which it intends to contribute UNAIDS UNDG definition UNESCO98 RBM is a broad management strategy aimed at changing the way institutions operate, by improving performance, programmatic focus and delivery. It reflects the way an organization applies processes and resources to achieve interventions targeted at commonly agreed results. Results-based management is a participatory and team-based approach to programme planning and focuses on achieving defined and measurable results and impact. It is designed to improve programme delivery and strengthen management effectiveness, efficiency and accountability. Results Framework The logic that explains how results are to be achieved, including causal relationships and underlying assumptions. The results framework is the application of the logframe approach at a more strategic level, across an entire organisation, for a country programme, a programme component within a country programme, or even a project. Planning outline/ Results Chain Goal The higher-order objective to which a development intervention is intended to contribute. Impact: Positive and negative long-term 94 95 Strategic objectives The Strategic Framework establishes 12 strategic objectives with associated strategies na A three-level hierarchy: 1. Mandate and vision, UNDG definition UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 JIU, Implementation of Results-Based Management in the United Nations Organizations, Part I, JIU/REP/2004/6. 96 ILO, Preview of the Programme and Budget proposals for 2008-09 and related questions (a) Strategy for continued improvement of results-based management in the ILO. GB.297/PFA/1/1. Report to the ILO Governing Body, 297th Session, Geneva, November 2006 97 ILO, RBM Guide Book, DRAFT, ILO 2006. 98 UNESCO, Results-Based Programming, Management and Monitoring (RBM) at UNESCO, Guiding Principles, Bureau of Strategic Planning, Jan 2008, 6. 32 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG94 effects on identifiable population groups produced by a development intervention Equated with National priority/ Goal in the results chain Communicated as: A measurable, achievable, sustainable change in the lives of people FAO95. ILO96 97. Clarifies the Organization‘s aspirations, mandate and comparative advantages. The Constitution of the Organization establishes its vision and mandate, 2. intermediate outcomes (equated with long-term changes or impacts) of constituents and the global community, Goals of the Organization‘s constituents. Determines priorities for ILO action UNAIDS UNESCO98 Outcome The intended or achieved short-term and medium-term effects of an intervention‘s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact. Communicated as institutional or behavioural change at national or sub-national levels, to be achieved at the end of the UNDAF cycle Objectives For each strategic objective the MTP proposes programme entities The programmes fall into three categories: Technical Projects (up to 6 years), Continuing Programme Activities and Technical Service Agreements The programme entities have objectives expressed in terms of relevance to the strategic objectives and of benefits to the users, with quantified targets when possible. The objectives have outcome indicators. 3. immediate outcomes, for which the ILO is accountable and which are the basis for assessing the performance of the Office. - These are real-world results to which the Office‘s contribution is direct and verifiable. There remain measurement - Indicators and targets are developed around immediate outcomes to measure progress towards achieving them UNDG definition Results are finally the effects of outputs on a group of beneficiaries. In the UNESCO results chain, these C5 expected ―results‖ closely correlate with Agency outcomes Outputs The products and services which result from the completion of activities within a development intervention. Each of the programme entities has outputs (similar to the UNDG definition). identifying the major outputs that will allow the stated objective to be achieved, not only in terms of the product but also in terms of the period of time in which they are to be produced, thus establishing milestones for monitoring purposes. Outputs are divided, to the extent possible, into the following categories: - information products and systems The concrete products and services that are directly and immediately borne out of organizational activities What the Office and its staff produce and deliver UNDG definition The completion of interventions leads to the production of outputs. 33 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG94 FAO95. and databases - studies and analyses - methodologies and guidelines - international undertakings, agreements/conventions and standards - training courses and training materials - coordination and information exchange - other ILO96 97. UNAIDS UNESCO98 Activity Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs. The work that organizations engage in on a daily basis to produce results What the Office and its staff produce and deliver UNDG definition Interventions describe what we do in order to produce the changes expected RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG99 Results Based Management (RBM) A management strategy by which an organization ensures that its processes, products and services contribute to the achievement of desired results (outputs, outcomes and impacts). RBM rests on clearly defined accountability for results, and requires monitoring and selfassessment of progress towards results, and reporting on performance. UNHCR100 UNHCR views results-based management as a management philosophy that emphasizes the achievement of results -- the impact to be achieved -- as the essential task of management. UNHCR uses RBM to help ensure that all organisational processes support the achievement of the right results in terms of protection and solutions for persons of concern as effectively and efficiently as possible. For UNHCR, RBM involves, at a minimum, four key steps: 1. Defining a strategic framework that provide a focus for action; UNIDO 101 UNDG definition UNIFEM 102 UNDG definition UN Secretariat103 UNDG definition Implementation of results-based management is not yet fully realized, pending endorsement by the GA of the RBM Framework, to be proposed by the SG in early 2008 under Governance and Oversight (A/62/_/Add._). Definitions and systems pending the endorsement by the General Assembly of the proposed RBM Framework. OHCHR OHCHR works within the secretariat-wide framework for RBM UNDG definition 99 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNHCR, Effective Planning - Guidelines for UNHCR Teams, OMS Working Draft, June 1999. UNIDO, Interim Report of the External Auditor, IDB.33/8–PBC.23/8, March 2007. 102 UNIFEM, Results-Based Management in UNIFEM: Essential Guide, February 2005. 103 UN Secretariat, http://www.un.org/Depts/oios/mecd/mecd_glossary/index.htm 100 101 34 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG99 UNHCR100 2. Specifying the expected results that contribute to these goals and aligning programmes, processes, people and resources behind them; 3. Engaging in ongoing monitoring and performance/ impact assessment using standards and indicators, and integrating lessons learned into future planning; and, 4. Ensuring accountability of individuals, teams and partners based on continuous feedback to improve performance. UNIDO 101 UNIFEM 102 UN Secretariat103 OHCHR Results Framework The logic that explains how results are to be achieved, including causal relationships and underlying assumptions. The results framework is the application of the logframe approach at a more strategic level, across an entire organisation, for a country programme, a programme component within a country programme, or even a project. Planning outline/ Results Chain Goal The higher-order objective to which a development intervention is intended to contribute. UNDG definition Conceptual framework indicates hierarchy of objectives and classic logframe elements. UNDG definition Logical framework UNDG definition Logical framework UNDG definition The desired result of UNHCR programmes in terms of overall solutions; goals are established at the organizational and programme levels. UNDG definition UNDG definition UNDG definition UNDG definition Impact: Positive and negative long-term effects on identifiable population groups produced by a development intervention Equated with National priority/ Goal in the results chain Communicated as: A measurable, achievable, sustainable change in the lives of people UNDG definition UNDG definition UNDG definition UNDG definition UNDG definition 35 RBM terminology Key questions: 1. Are there any major discrepancies between the UNDG harmonized approach and agency approaches? UNDG99 Outcome The intended or achieved short-term and medium-term effects of an intervention‘s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact. Communicated as institutional or behavioural change at national or sub-national levels, to be achieved at the end of the UNDAF cycle Outputs The products and services which result from the completion of activities within a development intervention. UNHCR100 A hierarchy of objectives model. Objective A statement of desired result, or specific accomplishment often established at the sectoral level UNIDO 101 UNDG definition UNIFEM 102 UNDG definition UN Secretariat103 ―outcome‖ is used as a synonym of an accomplishment or a result OHCHR ―outcome‖ is used as a synonym of an accomplishment or a result Defined deliverables which enable objectives and impacts to be achieved; UNDG definition The products, services, and capacities which result from the completion of activities within a development intervention. UNIFEM adds ―capacities‖ to the UNDG harmonized definition. UNDG definition UNDG definition Activity Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs. An activity is a specific action or series of actions which provide the means for achieving an objective. UNDG definition UNDG definition UNDG definition UNDG definition 36 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG104 1. Is there a RBM policy/ strategy in the agency, and when was it established? UNDP105 106 Yes. A formalized results-based management approach introduced in 1998. UNFPA 107 108 RBM was officially introduced in UNFPA in Dec. 2000 with a policy statement issued by the Executive Director. UNICEF 109 110 RBM was conceptualized in 1998 HRBA Executive Directive. It has been progressively introduced in the Medium Term Strategic Plans (20022005 and 2006 – 2009) and elaborated in the a 2003 RBM Guide which was updated in 2005 The Medium Term Strategic Plan (MTSP) - The MTSP sets out the vision and overall strategy for UNICEF over a four-year period. The current MTSP includes measurable outcomes that provide strategic direction to country offices and Key Performance Indicators which track UNICEF‘s performance across a series of financial, operational and human resource outputs. Furthermore, UNICEF has developed systems to monitor these indicators at a country and global level. Dalberg, 1. WFP Yes. WFP has been implementing RBM since 1997. But a dedicated RBM Division was officially established in WFP in February 2003 by the Executive Director 2. What is the strategic results framework for the RBM system? United Nations Development Assistance Framework The UNDP Strategic Plan, 2008-2011 is the basis for the accountability framework A hierarchy of 3 tiers of accountability: a) organizational accountability to the UNDP Executive Board b) Programmatic accountability for overall development outcomes rests with the government, with joint accountability between the government and UNDP for the portion of programmes/outcomes funded by UNDP c) Staff accountability for expected ethical and professional conduct and to their managers to deliver agreed performance results and budgets The results framework and RBM systems is built on a single instrument: the SRF/ now Strategic Plan. The SRF is used for strategic planning and forms the basis for performance assessment through the results-oriented annual report (ROAR)/ now Country Office Results Report (CORR). The MYFF is an organization-wide, results-oriented framework for UNFPA‘s performance. It defines the organizational goals that UNFPA contributes to and some of the outputs that will lead towards these goals; it does not include the purpose level of results. The MYFF is designed to be as simple a tool as possible for planning, monitoring and reporting on organizational-level results. The Strategic Plan 2006-2009 and Biennial Management Plan 20082009 ―…are results oriented and incorporate the organization‘s vision of RBM‖. Road Map p5. 3. What are the key elements of the RBM system? MD/MDGs and other internationally agreed treaty obligations and development goals National priorities/ goals UNDAF Results Matrix UNDAF M&E plan UNDAF Annual Review & Report (RC 104 105 106 UNFPA‘s strategic planning and performance management mechanisms are centered on the 4- year MYFF, which lays out the organization‘s overall strategic direction and priorities, and links them to programme resource commitments. The MYFF UNICEF‘s results-based programme planning and management components include results chain with: MTSP (5 focus areas); 20 key result areas- executive board; 57 organizational targets; 32 operational targets: Human Resources, Administration, Finance, Supply, Evaluation, Audit (2-5 per group); 16 WFP‘s RBM Components consist of: (i)Strategic planning (ii) Programme/Project planning, (iii) Biennial Management Planning ( Budgeting) (iv) Performance Measurement and Review (v) Performance Management and Reporting (vi) Evaluation UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNDP, Strategic Plan, 2008-2011 Accelerating global progress on human development, DP/2007/43, UNDP, 2007. section VI para 120. UNDP User Guide, Programme and Operations Policies and Procedures – Results Management and Accountability, 2/15/2008. 107 UNFPA, Results-Based Management Orientation Guide, June 2001 108 Dalberg, Assessing Results Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006. 109 UNICEF, Understanding Results-Based Programme Planning and Management, Tools to reinforce good programming practice, UNICEF, May 2005. 110 Dalberg, Assessing Results-Based Management at UNICEF, Danish Ministry of Foreign Affairs, April 2007. 37 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG104 Annual Report) UNDAF Evaluation UNDP105 106 UNFPA 107 108 and UNDAF inform the formulation and planning of the Country Program Documents (CPDs), which in turn shape the Country Action Plan (CPAP) that has a clear indication of resource allocation to various programme activities at the Country Office level (specified in the Annual Work Plan). UNICEF 109 110 functions. WFP 4. Who is the main audience for results information? Audience: UNCT, National Authorities UNDG for results related mainly to coordination (?) Executive Board Headquarters National Authorities Executive Board Headquarters National authorities ―Country level managers face challenges in implementing RBM on a day to day basis. Results management seems to plays its most significant role in goal setting and reporting, but not on a day-today operations basis. There is a need to strengthen and/or generalize the use results for guiding Country office operations‖. Dalberg p.21 Executive Board Headquarters National authorities Executive Board Headquarters National authorities EB sessions and informal consultations are the main channels of interaction and communication of WFP programmes which offer opportunities for defining strategic objectives, expected results and feedback. Annual Performance Report provides an aggregated picture Standard project reports (SPRs) and Annual performance reports (APRs) are the primary instruments to transparently communicate results with stakeholders. 38 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG111 1. Is there a RBM policy/ strategy in the agency, and when was it established? FAO112 Yes, with the adoption of the FAO Strategic Framework by the FAO Conference in 1999 ILO113 114 115 ILO has progressively introduced RBM to its strategic planning and budgeting since the 2000-2001 biennium. The Office submitted a ―roadmap‖ on RBM to its Governing Body in November 2006. Progress against the road map was reported on in a report to the ILO Governing Body in November 2007. UNAIDS The PCB at its meeting in June 2005 endorsed the strategies and approaches set forth in the Unified Budget and Workplan for 2006-2007 which included strengthened results based orientation and an improved platform for results-based management. UNESCO 116 In November 1999, UNESCO launched a comprehensive reform programme with RBM as a pillar. Since then, a results-based approach is systematically applied to the design and implementation of UNESCO activities at all levels. UNESCO's ―Guiding Principles to Results Based Programming, Management and Monitoring (RBM) at UNESCO‖ are designed to assist UNESCO staff in applying RBM and are updated on a regular basis. The Medium Term Strategy (6 years) and Biennial Programme and Budget (2 years) together constitute the programmatic and conceptual framework for all of UNESCO‘s action. 2. What is the strategic results framework for the RBM system? United Nations Development Assistance Framework The Strategic Framework for the period: 2000-1015, approved by the Conference in 1999. The Strategic Framework establishes the overall definition of those areas in which Members of the Organization require FAO's services. It does this in the form of 12 strategic objectives with associated strategies, which will become the basis of all programme planning within the Organization The Medium-Term Plan (MTP) is intended to be the principal vehicle for programme formulation and prioritization - consistent with the objectives of the Strategic Framework. The Strategic Policy Framework (SPF), The UNAIDS 2008-2009 Unified Budget and Work plan (UBW) 3. What are the key elements of the RBM system? The MTP sets the parameters of a six-year work plan (revised and updated each A simplified logical hierarchy has been progressively put in place (from a higher-level statement of mandate The UNAIDS 2008-2009 Unified Budget and Work plan (UBW) follows the RBM structure. UNESCO has three main sets of institutional planning documents: 111 112 113 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 FAO, Strategic Framework, Section III. Implementation programme for the Strategic Framework, FAO, 1999, Section III. ILO, Preview of the Programme and Budget proposals for 2008-09 and related questions (a) Strategy for continued improvement of results-based management in the ILO. GB.297/PFA/1/1. Report to the ILO Governing Body, 297th Session, Geneva, November 2006 114 ILO, RBM Guide Book, DRAFT, ILO 2006 115 ILO, Results-based management (a) Progress report and review of the programming cycle, Progress report submitted to Governing Body, GB.300/PFA/9/, November 2007. 116 UNESCO, Results-Based Programming, Management and Monitoring (RBM) at UNESCO, Guiding Principles, Bureau of Strategic Planning, Jan 2008 39 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG111 UNDAF Results Matrix UNDAF M&E plan UNDAF Annual Review & Report (RC Annual Report) UNDAF Evaluation FAO112 biennium), including the main objectives for each programme, outcome indicators, outputs, related timeframes, and broad estimates of resources required Programme of Work and Budget (PWB) sets out the delivery details for each programme area and the financial allocations required for each biennium. Results-Based budgeting (RBB) was introduced in FAO in the 2001 programme The Programme Implementation Report is intended to inform each session of the Conference on the performance and achievements of the Organization on outputs approved in the PWB and on progress towards the larger goals set out in the MTP. ―FAO staff correctly point out that excessive management time spent on planning takes time away from implementation and quality assurance. The planning emphasizes the micro-level at the expense of macro strategy and without making the essential connections. As a result, the system, however sound its design, remains essentially dysfunctional.‖117. The MTP is the document where outcomes and outcome indicators are intended to be set. The indicators included in previous MTPs were not conducive to outcome ILO113 114 115 and vision, and building downwards to the intermediate and immediate outcomes, the latter being the basis for assessing the performance of the Office: real-world results to which the Office‘s contribution is direct and verifiable. At country level, Decent Work Country Programmes (DWCP) constitute ILO‘s main programming mechanism. They are organised around a targeted number of country programme priorities and outcomes valid for a two to four year period. They define expected outcomes in countries that respond to both UN frameworks and constituents‘ needs and priorities. The DWCPs are also used as an instrument to actively involve ILO‘s constituents in the programming process, in alignment with UNDAF. ―The ILO‘s DWCPs are an important asset in linking RBM in the ILO with the rest of the UN system. Partly as a result, country commitments to decent work and productive employment are increasingly included in national poverty reduction and development plans. The ILO‘s guidance on DWCPs is being updated to take into account the latest experience on linking DWCPs to UNDAF and similar UN frameworks. RBM terminology is now more closely aligned with that used in other parts of the UN system. Monitoring and evaluation are integral to management decision- UNAIDS The main components are as follows: Unified and coordinated action on jointly established priorities, in accordance with the UNAIDS Technical Support Division of Labour and taking into account the comparative advantage of each member of the Joint Programme; A Budget focused on joint priorities and results, maximizing the impact of available resources; and A Workplan that provides a framework for joint implementation, translated into clear and accountable operational plans at country level. It includes a Performance Monitoring and Evaluation Framework designed to support results-based management, promote transparency, strengthen accountability, improve reporting, and reflect links between collective and individual levels of effort. UNESCO 116 The Medium-Term Strategy (C/4) - a 6-years rolling document determining the corporate strategy of the organization that can be revised by the General Conference The biennial Programme and Budget (C/5 document) 3 consecutive biennial programme and budget documents are produced for each MTS. These are meant to ensure a seamless transition between UNESCO‘s medium-term and biennial programme priorities and guaranteeing alignment between specific programme activities and medium-term objectives. There are three programming levels Level: 1 Major Programme Level: 2 Main Line of Action Level: 3 Activity Main lines of action focus on the presentation of expected results and performance indicators and, where necessary, of benchmark targets. The work plans (available in SISTER). At UNESCO the term ―work plan‖ refers to a planning document for an activity 117 FAO, The Challenge of Renewal, Report of the Independent External Evaluation of the Food and Agriculture Organization of the United Nations (FAO), Submitted to the Council Committee for the Independent External Evaluation of FAO (CC-IEE), September 2007. 40 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG111 FAO112 measurements. The MTP has become increasingly a collection of projects, rather than offering a coherent programme base. Secondly, managers concede that they focus mostly on inputs and outputs, not on monitoring outcomes or results (see further below). External evaluation para 1102 The biggest weakness of the MTP as currently used is that it fails to achieve its main purpose of providing effective and transparent linkages between ends and means. Para 1103 ILO113 114 115 making at all levels of the organization and evaluation is also envisioned as a means to support organizational learning. UNAIDS UNESCO 116 4. Who is the main audience for results information? Audience: UNCT, National Authorities Audience: FAO Governing Body Main audience: ILO Governing Body\ The Governing Body can make RBM more effective precisely by emphasizing outcomes over activities, and by reviewing performance in relation to outcomes. 41 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG118 1. Is there a RBM policy/ strategy in the agency, and when was it established? UNHCR119 UNHCR embraced resultsbased management (RBM) as an essential means of ensuring that UNHCR is a performancedriven organization, whose operations achieve the right results in the most effective and efficient manner possible. A UNHCR RBM Conceptual Framework was prepared in June 2006 that explains the Organization‘s concept of and approach to results-based management, further outlines how RBM will be implemented throughout the Organization. UNHCR‘s strategic planning framework involves the resultsbased management principle of cascading strategic objectives from a corporate level to the regional level and from there to sub-regional and country levels. Global strategic objectives 20072009 and HQ Corporate Plan. The corporate plan addresses a range of topics including: global situational/ context analysis and its implications for UNHCR, strategic direction and global strategic objectives, strategic priorities within operations, priorities in management and organisational development, and resource implications. Regional plans elaborate on the corporate plan and global UNIDO DG bulletin on RBM of 13 June 2007 provides strategic guidance and asks for formal RBM policy to be developed UNIFEM Yes. Initial efforts towards establishing RBM began in 1998. In 2005, the Executive Director issued a guide that clarifies basic RBM concepts from a human rights perspective for application in UNIFEM's programming, monitoring, reporting and evaluation. UN Secretariat Definitions and systems pending the endorsement by the General Assembly of the proposed RBM Framework. OHCHR120 OHCHR works within the UN secretariat framework for RBM 2. What is the strategic results framework for the RBM system? United Nations Development Assistance Framework NA from available materials The Strategic Plan, 2008-2011, which includes: 1. A development results framework, focused on country level outcomes, and 2. A management results framework focused on overall organizational performance. The Plan of Action defines OHCHR‘s overarching goal The High Commissioner‘s Strategic Management Plan (SMP) for 2008-09 Related Strategic Plans of all branches and field offices for 2008-09 118 119 UNDG Results-Based Management Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNHCR, Results-Based Management in UNHCR, Working Draft June 2006. 120 OHCHR, Guidelines for the preparation of OHCHR‘s Strategic Plans for 2008-09, 2008. 42 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG118 UNHCR119 strategic objectives and are intended to provide guidance on planning at the country and subregional levels as well as resource allocation. Regional plans are approved by the Executive Office. Country Operations Plans (COP)/ Sub-regional plans: reflects all aspects of a country operation, including protection, assistance, and any proposed durable solutions, as well as resource requirements. In addition to outlining the strategic direction for the operation in question, the COP also includes operational aspects. The COPs are reviewed and approved by the Operations Review Board as part of the annual review and resource allocation process UNIDO UNIFEM UN Secretariat OHCHR120 3. What are the key elements of the RBM system? UNDAF Results Matrix UNDAF M&E plan UNDAF Annual Review & Report (RC Annual Report) UNDAF Evaluation The elements of UNHCR‘s RBM system include planning, programming, budgeting, monitoring and evaluation; delegation of authority and accountability; staff performance management; and knowledge management processes. These are described in the UNHCR RBM Conceptual Framework Document. UNHCR is developing a new RBM software application Focus - designed to support the planning of and reporting on UNHCR field operations within the context of regional and corporate strategic plans. RBM in UNHCR involves four key steps: 1. Defining a strategic goals that provide a focus for action; 2. Specifying expected NA from available materials UNIFEM‘s RBM has evolved through the different cycles of the Strategy and Business Plan/MYFF/Strategic Plan. The Strategic Plan for 20082011 has a Development Results Framework and Management Results Framework. The Plan of Action defines The High Commissioner‘s Strategic Management Plan (SMP) for 2008-09 Related Strategic Plans are developed by all branches and field offices for 2008-09 Strategic plans contain a results matrix. The Results Matrix indicates the key strategic improvements in the human rights situations in countries which could be attributable to OHCHR (although not necessarily solely). These expected key results should contribute to the achievement of the objectives and goal set by the Plan of Action. At the Field level, they should also 43 Annex: Main Components of the RBM System Key questions: 1. Is there a RBM policy/ strategy in the agency, and when was it established? 2. What is the strategic results framework for the RBM system? 3. What are the key elements of the RBM system? 4. Who is the main audience for results information? UNDG118 UNHCR119 programme results that contribute to these goals, and aligning programmes, processes, people and resources behind them. 3. Undertaking ongoing monitoring and performance/impact assessments (using standards and indicators) and using lessons learned in future planning; 4. Ensuring individuals, teams and partners are accountable and using continuous feedback to improve performance. UNIDO UNIFEM UN Secretariat OHCHR120 contribute to the relevant UNDAFs. 4. Who is the main audience for results information? Audience: UNCT, National Authorities NA from available materials Executive Board Headquarters National authorities High Commissioner The General Assembly and funding partners 44 Progress Reviews & Annual Reporting Key questions: 1. Are links to UNDAF and UNDAF results clear? 2. Is there an emphasis on use of results information to manage, not just on planning and reporting about results? 3. Who is the main audience for the reviews and reports? UNDG121 UNDP Country Office Results Report: The report concentrates on performance towards the intended outcomes of the country programme Annual CPAP Review A basis for assessing the performance of ongoing and recently completed projects and aligned initiatives, and a basis for the preparation of the CORR Each CPAP outcome has an Outcome Board responsible for the monitoring progress on achievement, and the extent to which lessons are being fed back into programming. But the evaluation of UNDP found that the ROAR [now CORR] ―has become primarily a tool for reporting to senior management – of little use for country office or regional management – and is viewed as a data source for the annual MYFF report to the ExB‖ (see UNDP RBM evaluation p. ix) No mention of the links between agency-supported results and UNDAF outcomes UNFPA At the country level, the Country Office Results Oriented Annual Report (COAR) is the key report that contributes to the MYFF. It reports performance of the country office according to the outcomes and outputs outlined in the MYFF, Country Program and Annual Work Plan. It is the primary tool to measure UNFPA contribution to overall development goals (SRF). It also reports on the country office performance on management (MRF). The COAR is targeted at SPO for the development of the MYFF and at GDs for assessing the CO and region‘s performance. Main audience is HQ and the ExB (see Dalberg) No mention of the links between agency-supported results and UNDAF outcomes UNICEF COAR guidelines in line with good RBM practice COs asked to reflect on a issues related to achievements and shortfall against UNDAF results and the performance of the UNCT. Main results section classified in terms of the corporate plan priorities. No assessment of the effectiveness of the COAR as a vehicle for reporting on UNDAF results in the Dalberg report. WFP SPRs have been introduced that are results-oriented. Annual Performance Reports have been produced to provide a results story for the agency as whole. Main audience: HQ and national authorities The external auditors review of RBM system in 2006 encouraged: ―structured procedures to obtain systematic feedback from recipient and donor countries on their level of satisfaction with the way WFP is reporting on its results. Rec2 confirmation of the extent to which the framework meets the needs of Stakeholders (para 61) No mention of the links between agency-supported results and UNDAF outcomes 121 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 45 Progress Reviews & Annual Reporting Key questions: 1. Are links to UNDAF and UNDAF results clear? 2. Is there an emphasis on use of results information to manage, not just on planning and reporting about results? 3. Who is the main audience for the reviews and reports? UNDG122 FAO na ILO na UNAIDS na UNESCO na Progress Reviews & Annual Reporting Key questions: 1. Are links to UNDAF and UNDAF results clear? 2. Is there an emphasis on use of results information to manage, not just on planning and reporting about results? 3. Who is the main audience for the reviews and reports? UNDG123 UNHCR124 UNCHR reports are to ―address the specific objectives and activities of UNHCR within the agreed wider UN country operation.‖ No mention of the links between agency-supported results and UNDAF outcomes UNIDO na UNIFEM na UN Secretariat Definitions and systems pending the endorsement by the General Assembly of the proposed RBM Framework. OHCHR125 Internal Annual Report primarily an instrument for accountability and provides information about progress against plans Also serve as a basis for external reporting No mention of the links between agency-supported results and UNDAF outcomes 122 123 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 124 UNHCR, Instructions and Guidelines to UNHCR Field Offices and Headquarters Units on Reporting on 2007, Implementation in 2008 and Planning for 2009, UNHCR, 2007. 125 OHCHR, Guidelines for 2007 Internal Annual Reports – Field Offices, OHCHR, 30/08/2007. 46 IT Platforms Key question: 1. Do IT platforms for the planning, implementation, monitoring, and reporting about programmes and projects assist management for results – especially outcomes? UNDG126 DevInfo – on a voluntary basis at country level. UNDP There are strong internal perceptions within UNDP that financial administration and management systems have improved. However, there is little evidence that these systems have led to an increased focus on managing for outcomes. Atlas and PRINCE2 both deal with information at the project level, and the project is at the core of their designs. Results systems have been designed mainly to meet the demand for data for reporting to the Executive Board rather than to manage outcomes. Yet UNDP has not developed a system for reporting on its contribution towards development results. UNFPA The i-Track system allows all units (CO and HQ) to fill their annual reports online. The information is collected (COAR and other unit‘s annual reports) and is used to produce the MYFF report. The Atlas ERP system manages enterprise information relating to budgets and commitments, funding contributions, human resources, programming information and increasingly, partner interactions and development outcomes and outputs. UNICEF UNICEF is also improving ERP and IT systems to capture data on performance management software [COGNOS-based] and Human Resource Information System / SAP (Systems Applications and Products software) systems o Use of DevInfo (development information software) to further capture/review data for making RBM more widely used WFP WFP can be categorized as being in the ―Exploration‖ phase. Much work needs to be done to define a cost model, to adapt this model into the SAP system as part of the WINGSII upgrade, and to develop Resourcesto-Results reports for managers. IT Platforms Key question: 1. Do IT platforms for the planning, implementation, monitoring, and reporting about programmes and projects assist management for results – especially outcomes? UNDG127 FAO PIRES – Programme Planning, Implementation Reporting and Evaluation Support System, is a webbased system used by HQ and decentralized offices to plan, implement, monitor and report on a results basis Unclear from available documents, however the external evaluation argues that ILO IRIS (an ERP system) "Integrated Resource Information System" (IRIS) introduced in 2005. It contains modules and functions to facilitate results-based strategic management. UNAIDS UNESCO Platforms are: SISTER FABS STEPS No indication from available documents whether SISTER emphasizes management for UNESCO results (agency outcomes) 126 127 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 47 IT Platforms Key question: 1. Do IT platforms for the planning, implementation, monitoring, and reporting about programmes and projects assist management for results – especially outcomes? UNDG128 None. UNHCR129 MSRP and proGres contribute significantly to the implementation of Results-Based Management within UNHCR The key component of UNHCR‘s information technology toolset for results-based management will be the Operations Management Support Software. This tool will support the comprehensive recording of objectives at all levels from the strategic through to projects within operations, the definition of performance indicators and performance budgets and the recording of progress in achieving results so as to facilitate reporting. The Operations Management Support Software will be developed in 2006, piloted in 2007 and rolled out world wide in 2008. UNIDO NA from available materials UNIFEM Same as UNDP. UN Secretariat 128 129 UNDG, Approved Harmonized RBM Terminology, UNDG, June 2003 http://www.undg.org/index.cfm?P=224 UNHCR, Results-Based Management in UNHCR, Working Draft June 2006. 20. 48 Annex B: Principles and a Framework for Results-Based Management in an Organization130 Principle 1. Foster senior-level leadership in results-based management Effective leadership is essential if results-based management is to succeed and requires: 1.1 Demonstrated senior management leadership and commitment 1.2 A senior management capacity for results management Principle 2. Promote and support a culture of results Fostering an appropriate organizational culture of results is critical and requires: 2.1 Informed demand for results information 2.2 Supportive organizational systems, incentives, procedures and practices 2.3 A results-oriented accountability regime 2.4 A capacity to learn and adapt 2.5 Results measurement and results-based management capacity 2.6 Clear role of and responsibilities for results-based management Principle 3. Build results frameworks with ownership at all levels The organization needs to set out the overall and specific strategic results its programmes are collectively and individually intended to achieve and how best to structure itself to achieve them, namely: 3.1 A strategic results framework, outlining organizational objectives and strategies and major risks, aligned with the organization‘s programmes. 3.2 Results frameworks for programmes showing objectives, strategies and resources used, risks faced and the logic behind the programme design. 3.3 Reasonably clear and concrete performance expectations for programmes. 3.4 A strategy for measuring key results, including a manageable set of performance indicators for programmes and complementary evaluations. 3.5 Ownership by managers and staff of results frameworks that are relevant and useful. Principle 4. Measure sensibly and develop user-friendly RBM information systems The organization needs to gather and analyse credible information on performance through: 4.1 Measuring results and costs using both ongoing monitoring and evaluation, and assessing actual results and costs in light of the performance expectations. 4.2 Assessing the contribution and influence made by the programmes to the observed results. 4.3 Building cost-effective, user-friendly and relevant RBM information systems. Principle 5. Use results information for learning and managing, as well as for 130 Mayne, J., Best Practices in Results-Based Management: A review of experience, Volume 1: Main Report, UN Secretariat, July 2007. 3-4. 49 reporting and accountability Realizing the benefits from results-based management requires: 5.1 Using performance information to inform and improve programme performance and budgets. 5.2 Identifying and using best practices to improve performance. 5.3 Credible performance reporting internally and externally, telling a coherent performance story. Principle 6. Build an adaptive and RBM regime through regular review and update Implementing RBM is an ongoing learning process: 6.1 Regularly review and update all aspects of the RBM regime—frameworks, indicators, expectations, measurement strategies, systems and use—as to continued relevance, usefulness and cost. 50 Annex C: References Balogun, P., Issue Paper on Fundamental Revision of M&E Aspects of UNDAF Guidelines, Working paper submitted to the WGPP, January 2008. Balogun, P.UNEG Study on the Evaluability of the UN Development Assistance Framework, UNEG, December 2006, 5. CEB High Level Committee on Programmes, Evaluation of the Delivering as One Pilots – Progress Report on Evaluability Assessments, paper by UNEG, CEB/2008/HLCP-XV/CRP.3 3 March 2008. Dalberg, Assessing Results-Based Management at UNICEF, Danish Ministry of Foreign Affairs, April 2007. Dalberg, Assessing Results-Based Management at UNFPA, Danish Ministry of Foreign Affairs, June 15 2006 FAO, Strategic Framework, Section III. Implementation programme for the Strategic Framework, FAO, 1999, Section III. FAO, The Challenge of Renewal, Report of the Independent External Evaluation of the Food and Agriculture Organization of the United Nations (FAO), Submitted to the Council Committee for the Independent External Evaluation of FAO (CC-IEE), September 2007. ILO, Results-based management (a) Progress report and review of the programming cycle, Progress report submitted to Governing Body, GB.300/PFA/9/, November 2007. ILO, RBM Guide Book, ILO, October 2006. ILO, Preview of the Programme and Budget proposals for 2008-09 and related questions (a) Strategy for continued improvement of results-based management in the ILO. GB.297/PFA/1/1. Report to the ILO Governing Body, 297th Session, Geneva, November 2006 JIU, Implementation of Results-Based Management in the United Nations Organizations, Part I, JIU/REP/2004/6. JIU, Overview of the series of reports on Managing for Results in the United Nations System, JIU/REP/2004/5, JIU, 2004. Longhurst, R., Review of the Role and Quality of the United Nations Development Assistance Frameworks, Overseas Development Institute, May 2006. MacKenzie, A., Issues and recommendations for the revision of the 2004 CCA-UNDAF Guidelines, Working Paper presented to the WGPP, October, 2006. Mayne, J., Best Practices in Results-Based Management: A review of experience, Volume 1: Main Report, UN Secretariat, July 2007. OHCHR, Guidelines for 2007 Internal Annual Reports – Field Offices, OHCHR, 30/08/2007. OHCHR, Guidelines for the preparation of OHCHR‘s Strategic Plans for 2008-09, 2008. Poate, D. et al., Evaluation of Results-Based Management in UNDP, UNDP, 2007. UNDG, 2007 Guidelines for the completion of the Resident Coordinator Annual Report (RCAR), UNDG, 2007. UNDG, Approved harmonized RBM Terminology, UNDG, 2003. UNDG, Issues paper: Common Annual Reporting by Pilot UNCTs, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. UNDG, Common Country Assessment and United Nations Development Assistance Framework, Guidelines for UN Country Teams on preparing a CCA and UNDAF, United Nations, February 2007. UNDG, Final Report: Task Team 2 Lessons Learned on the One Programme, Task Team 2 of the Working Group on Programming Policies (WGPP), January 2008. 51 UNDG, Generic Terms of Reference for RDT, Draft 5, March 6, 2008. UNDG, Guidelines for the UNDAF Annual Review, UNDG, 2007. UNDG, Principles for Enhancing the Leadership Role of the Resident Coordinator for UN Operational Activities for Development and Accountability Framework of the Resident Coordinator System, March 2006. UNDG, Resident Coordinator System Mutual Accountability Framework, UNDG, 2007. UNDG, Summary Overview of the Resident Coordinator and UN Country Team Performance Appraisal System, 2006 Summary, UNDG, 2007. UNDG, Terms of Reference – Study of Inter-Agency Results-Based Management, Task Team 1 of the Working Group on Programming Policies (WGPP), December 2007. UNDG, UN agency RBM mapping, working document of the Task Team 1 of the Working Group on Programming Policies (WGPP), January 2008. UNDG-CPSG, Consolidated CPSG Comments on the 2006 Synthesis of Resident Coordinator Annual Reports (SRCAR), 2007. UNDP, Strategic Plan, 2008-2011 Accelerating global progress on human development, DP/2007/43, UNDP, 2007. UNDP User Guide, Programme and Operations Policies and Procedures – Results Management and Accountability, 2/15/2008. UNDP, RBM in UNDP: Technical Note, Knowing the What and the How UNEG, The Role of Evaluation in Results-Based Management (RBM), UNEG, Feb 2007, UNESCO, Results-Based Programming, Management and Monitoring (RBM) at UNESCO, Guiding Principles, Bureau of Strategic Planning, Jan 2008. UNFPA, Strategic Plan, 2008-2011: Accelerating Progress and National Ownership for the ICPD Programme of Action, DP/FPA/2007/17, July 2007. UNFPA, Results-Based Management Orientation Guide, June 2001 UNGA (General Assembly), Triennial comprehensive policy review of operational activities for development of the United Nations system, A/C.2/62/L.63, 19 December, 2007. UNHCR, Results-Based Management in UNHCR, Working Draft June 2006. UNHCR, Effective Planning - Guidelines for UNHCR Teams, OMS Working Draft, June 1999. UNHCR, Instructions and Guidelines to UNHCR Field Offices and Headquarters Units on Reporting on 2007, Implementation in 2008 and Planning for 2009, UNHCR, 2007. UNICEF, Understanding Results-Based Programme Planning and Management, Tools to reinforce good programming practice, UNICEF, May 2005. UNICEF, Guidelines for the 2007 Country Office Annual Reports, CF/EXD/2007-009, UNICEF, 16 November 2007. UNIDO, Interim Report of the External Auditor, IDB.33/8–PBC.23/8, March 2007. Para 12. UNIFEM, Results-Based Management in UNIFEM: Essential Guide, February 2005. WFP, Mainstreaming Results-Based Management at the World Food Programme – The Way Forward Road Map, GDE Consulting, January 2006. WFP, RBM Orientation Guide, WFP, October 2003. 52