Evaluation Policy by liaoqinmei

VIEWS: 6 PAGES: 36

									         National Evaluation
       Capacity Development
                                  A conceptual framework



                                        Marco Segone*,
                        Systemic Management, UNICEF Evaluation Office,
                                and former Vice President, IOCE
                                 E-mail: msegone@unicef.org




* The opinions expressed are those of the author and do not necessarily reflect the policies or views of UNICEF.
 Based on the book published by UNICEF in
partnership with key international institutions


                     Authors:
                     40 global evaluation leaders
                     Partnership:
                     UNICEF, WB, UNDP, WFP,
                     UNIFEM, ILO, IDEAS, IOCE,
                     DevInfo
Webinars are available at www.mymande.org



             1




         2
                       UN Monthly webinars:
Saraswathi Menon            The Role of the United Nations in Fostering 22nd June 2010
                            National Ownership and Capacities in
Finbar O’Brien              Evaluation

Michael Quinn Patton        Future trends in evaluation.                  1st July 2010

Marco Segone                Moving from policies to results by
                            developing national capacities for country-
                            led monitoring and evaluation systems

Caroline Heider             Evaluating policies and their results         16th September 2010

Craig Russon                The role of policy analysis in over-coming
                            the implementation challenge

Belen Sanz Luque            Human Rights and gender in evaluation         22nd October 2010

Florence E. Etta

Jean Quesnel                The professionalization of                    23rd November 2010
                            evaluation
Linda G. Morra Imas

Robert Picciotto

Oscar A. Garcia             Joint evaluation of the role and contribution 14th December 2010
                            of the UN system in South Africa. Lessons
Angela Bester               learned
                        UN Monthly webinars:
Michael Bamberger            Institutionalizing impact evaluation. A key January 2011
                             element in strengthening country-led
                             monitoring and evaluation systems

Hallie Preskill              Exploring effective strategies for facilitating   February 2011
                             evaluation capacity development

Alexey Kuzmin                Use of evaluation training in evaluation
                             capacity building

Michael Quinn Patton         Utilization-focused evaluations                   March 2011

David Fetterman              Empowerment evaluation                            April 2011

Rita O’Sullivan (TBC)        Collaborative evaluation. Creating
                             environments to engage stakeholders in
                             evaluation

Indran Naidoo                The monitoring and evaluation in South            May 2011
                             Africa. Many purposes, multiple system

Manuel Fernando Castro       Building a results-based management and           June 2011
                             evaluation system in Colombia
Diego Dorado (TBC)

Velayuthan Sivagnanasothy    National monitoring and evaluation system         July 2011
                             in Sri Lanka. Experiences, good practices,
                             challenges and the way forward

David Rider Smith            Policies, institutions and personalities.         September 2011
                             Lessons from Uganda’s experience in
                             monitoring and evaluation
Webinars on Emerging Practices on Development Evaluation:
                   Speakers                                                   Title                                     When
Zenda Ofir, Former President, African Evaluation
Association                                           Developing Countries Perspective on Emerging    13 October 2010
                                                      Practices in Development Evaluation
Shiva Kumar, Independent Consultant from India



Fred Carden, International Development                Good Practices in Evaluating Policy Influence   16 November 2010
Research Center (IDRC)



Heather Creech, Director of Global Connectivity
at the International Institute for Sustainable        Evaluating Networks and Partnerships            7 December 2010
Development (IISD)

Jared Raynor, Consultant of TCC group

Peter Morgan, Independent Consultant                  Evaluating Capacity Development                 January 2011



Charles Lusthaus, Universalia, Associate              Evaluating Organizational Performance           February 2011
Professor in the Department of Administration
and Policy Studies, McGill University; & partner in
Universalia Management Group; Faculty Advisor
to the Centre for Educational Leadership, McGill
University

Steve Rochlin, Director and AccountAbility's US       Evaluating Innovation                           March 2011
Representative


Alastair Bradstock, Business Development              Evaluating Sustainable Development              April 2011
Director of the International Institute for
Environment and Development (IIED)
Webinars are available at www.mymande.org



             1




         2
WHY
        Management for DevResults and
              Paris Declaration


 National ownership and capacity development:
      the key ingredients to development



Managing for    Mutual      Harmoni     Alignment   Ownership
results         accounta    zation
                bility




               Paris Declaration on Aid Effectiveness
Partner countries exercise effective leadership
over their development policies and strategies
                    Paris Declaration Commitment

    Partner countries exercise leadership in developing and
     implementing their national development strategies
    Donors respect partner country leadership and help strengthen
     their capacity to exercise it.


                   Implications to the M&E Function



     Strengthen   and use country-led M&E systems
     Institutionalize M&E systems
     M&E capacity development to ensure international standards are met
… in line with the UN General Assembly …


• national Governments have the primary responsibility for
  coordinating external assistance and evaluating its
  contribution to national priorities
• mandated the United Nations system to promote national
  ownership and capacity development, and to make
  system-wide progress in harmonizing evaluation practices




   UNICEF
WHAT
A Systemic approach to Capacity Development




                Individual Level
                (skills, knowledge, experience)




   UNICEF
   Individual Level
Demand side
• Capacity to strategically plan evaluations, and to identify the key evaluation
  questions
• Capacity to manage evaluation for independence and credibility
• Capacity to use evaluation


Supply side:
• Behavioural independence
   • Independence of mind & integrity
   • Knowledge and respect of evaluation standards
   • Agreed evaluation processes & products

• Professional competences
   • Formal education (Masters)
   • Specialized training
   • Professional Conferences and meeting
   • On the job training (country-led evaluations)
   • Community of Practices and networking
A Systemic approach to Capacity Development



                                          Institutional Level
                                          (policies, procedures, frameworks)




                Individual Level
                (skills, knowledge, experience)




   UNICEF
Institutional Level
 • Evaluation culture
   •   Set of values and attitudes supporting evaluative (critical) thinking within an
       organization
   •   Institutional commitment to learning from evaluation, support evidence-based
       policy debate and demand for accountability.
   •   Individual more self-directed learners and use information to act; take higher risks
       but also develop a greater sense of personal accountability and responsibility;
       consult, coach, and support each other more.
   •   Protective culture (Remove repercussions on careers)
   •   Understanding of the foundations and principles of Monitoring and Evaluation
 • Institutionalizing independence, credibility and utility
   •   Evaluation policies
 • Work programme and budget
   •   Independence & adequacy of budget
 • Conduct of evaluations
   •   Institutional endorsement of standards
   •   In-built Quality Assurance systems
A Systemic approach to Capacity Development



                                                                 Institutional Level
                                                                 (policies, procedures, frameworks)
             Enabling
             Environment
             (policies, legislation,
             power relations,
             social norms)




                                       Individual Level
                                       (skills, knowledge, experience)




    UNICEF
Enabling Environment
• Public administration committed to manage for
results and accountability
   • Transparency
   • Results-based public budgeting
   • Evidence-based policy making

• Strong civil society
   • rights holders able to demand for and monitor quality of
   public services

• Strong national evaluation association
   • foster indigenous demand for M&E, and strengthen
   indigenous supply
   • set national evaluation standards and norms
Lessons learned and Good practices in
Capacity Development (1/2)
 Capacity development is underpinned by the
 fundamental characteristic of national ownership

  • Taking a capacity development response to scale requires linking
    to national and local plans, processes, budgets and systems

  • A comprehensive capacity development response must link to and
    draw from relevant national reforms to be sustained




   UNICEF
Lessons learned and Good practices in
Capacity Development (2/2)
Capacity development is not about a technical fix. It is
about transformations and must address how best to
manage change


   • A capacity development response can and often must show both
     short- and long-term gains, to ensure continued political
     commitment and resource support

   • Capacity traps’ more often pertinent to the “soft side” such as
     power relations, vested interests, access, ethic and attitudes



   UNICEF
HOW
Greatest influence on capacity development.
4 core issues


                         National Capacities




  Institutional
  Arrangements




   UNICEF
        Availability of Resources (human and financial ) and Competencies
                 Institutional settings

Institutional   Diagnosis of country-led
settings        M&E systems

                • Functional clarity
                • Effective HR management
                • Robust coordination mechanism
                • Etc

                Examples
                • Malaysia institutionalized RBM
                System

                •Sri Lanka national M&E System
                (to be presented in this panel)

                • Maldives national framework for
                aid effectiveness (in design
                stage).
UNICEF

                •Results-based public budgeting
                  Institutional settings

Institutional   Technical support to
settings        country-led M&E
                systems based on
                identified
                bottlenecks

                • Systems support
                • Technology (DevInfo)
                • Methodologies (MICS)



                Examples

                •DevInfo-based M&E
                systems
                (PRS, SWAP, MDG)
                •Country-led Evaluations
UNICEF
                  Institutional settings

Institutional   Local context-
settings        specific M&E
                standards and
                norms

                Examples

                • UNEG evaluation
                standards
                •AfrEA Evaluation
                standards
                • Niger evaluation
                principles




UNICEF
Greatest influence on capacity development.
4 core issues


                         National Capacities




  Institutional        Leadership
  Arrangements




   UNICEF
        Availability of Resources (human and financial ) and Competencies
                         Leadership

Leadership   Identify and support local
             leaders, especially in the
             initial stage
             • Identification & support to
             champions
             and change agents

             Examples
             M&E Depts in Governments
             • Sri Lanka’s Ministry of Planning
             •India’s National Planning
             Commission; and National
             Commission for the Protection of
             Child Rights:
             • Ministry of Finance, Malaysia

             National Evaluation associations
             • Community of Evaluators
UNICEF       •Malaysian Evaluation Society
             (MES)
             •Sri Lankan Evaluation Association.
Greatest influence on capacity development.
4 core issues


                         National Capacities




  Institutional        Leadership          Knowledge
  Arrangements




   UNICEF
        Availability of Resources (human and financial ) and Competencies
                        Knowledge

Knowledge   Domestic knowledge
            Networks

            • Seed/support a pool of
            local talent/national
            Expertise

            Examples
            •   National Evaluation
                Associations

            •   National and local
                Centers of Excellence
                TESA Project: Teaching
                Evaluation In South Asia

            •   Evaluation trainings
                using local institutions -
UNICEF          India and Bangladesh
                with support from
                UNICEF
                         Knowledge

Knowledge   South-South
            learning solutions

            •Linking to regional networks
            and institutions
            • Institutional twining
            arrangements

            Examples
            Regional Evaluation Associations:
            •  Community of Evaluators

            Facilitate Peer to peer support
            •   Malaysia and India to
                Thailand
            •   Sri Lanka to Maldives




UNICEF
                         Knowledge

Knowledge   KM systems in support
            to evaluation

            • Document good practices
            • Facilitate access to knowledge
            • Dynamic and human-based
            KM


            Examples

            • Books
            • Videos
            • Webinars
            • CoP
            • Website
            • Wiki

UNICEF
Greatest influence on capacity development.
4 core issues


                         National Capacities




  Institutional        Leadership          Knowledge          Accountability
  Arrangements




   UNICEF
        Availability of Resources (human and financial ) and Competencies
                      Accountability
Accountability Right-holders feedback
                mechanisms and
                systems
                • Developing M&E capacities
                of non-Governments
                stakeholders
                •Promoting participatory
                monitoring
                processes and instruments
                • Advocacy on evidence-
                literacy
                •Facilitate access to
                evidence
                Examples
                •   Civil society networks
                    (Child-friendly monitoring
                    network in Brazil)
                •Child-friendly journalists



UNICEF
                      Accountability
Accountability Duty-bearers capacity             Partners
                and policy-makers
                demand for M&E
                systems
                • Strengthen the demand for
                evidence by Parliaments          UN
                and Governments                  National and international
                •Develop M&E capacities of       NGOs
                Public administration

                Examples
                • Asia – Pacific Regional
                Agenda for Action on use of
                data for evidence make
                decision making
                •Parliamentary Committees
                for children's rights (Brazil)
                • Ministry of Finance to
                develop and monitor policy
                framework for RBM in
UNICEF
                Malaysia.
 We need a mind shift to do things
 differently...

We should always aim at strengthening National
  ownership and leadership, and not undermining it:
• Selecting topics of mutual interest
• Implement evaluations jointly with Governments, as a
  first step towards country-led evaluation
• Hiring local experts to the maximum extent possible
• Not assuming there are weak evaluation capacities,
  even if none is immediately apparent
• Co-ordinating with other key international stakeholders
Invest in the country-led process, even if it may
  require additional efforts and less control
Thank you

								
To top