Evaluation of the Capacity Building for EFA Cap EFA extra by alicejenny

VIEWS: 49 PAGES: 88

									                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report




                                                                           Internal Oversight Service
                                                                                      Evaluation Section

                                                                                              IOS/EVS/PI/86
                                                                                            Original: English




                                                              Evaluation of the
                                         Capacity Building for EFA (Cap-EFA)
            Extra-budgetary Programme for Technical Support to Member States




                                                     Juho Uusihakala Consulting and Advisory Services
                                                                                   in association with
                                                                      PEDIUM Education Consultants
                                                                                       Capacity Trust
                                                                                      Ms Ana Juremo


                                                                                                      April 2008



The views and opinions expressed in this document are those of the authors and do not necessarily represent the
views of UNESCO or IOS. The designations employed and the presentation of material throughout this document
do not imply the expression of any opinion whatsoever on the part of UNESCO concerning the legal status of any
country, territory, city or area of its authorities, or concerning its frontiers or boundaries.
                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                     Final Report



List of Contents
List of Abbreviations .................................................................................................................... i
Acknowledgement ......................................................................................................................iii
Executive Summary .................................................................................................................. iv
1 Background to the Evaluation.............................................................................................. 1
  1.1     Objective of the Evaluation ......................................................................................... 1
  1.2     Methodology ............................................................................................................... 1
  1.3     Structure of the report ................................................................................................. 3
  1.4     Limitations to the evaluation........................................................................................ 3
2 Contextualising Cap-EFA .................................................................................................... 4
  2.1     UNESCO Mandate and Core Functions ...................................................................... 4
  2.2     EFA in UNESCO Programme and Budgets................................................................. 5
  2.3     Importance and challenges of extra-budgetary funding (EXBF) ................................ 10
  2.4     Evaluation of UNESCO’s Support to National Planning for EFA................................ 13
  2.5     Consolidated findings of the Evaluations of UNESCO’s Major Programme I Institutes
  and Centres .......................................................................................................................... 14
  2.6     A Review of UNESCO’s Capacity-Building Function ................................................. 15
  2.7     Internal Review of UNESCO’s Capacity Development Programmes ......................... 16
  2.8     Challenges in the Field Offices.................................................................................. 17
  2.9     UNESS to ensure relevance and effectiveness of UNESCO’s activities.................... 17
  2.10 Summary of identified challenges and responses ..................................................... 18
3 Capacity Building for Education for All – programme description....................................... 19
  3.1     Memorandum of Understanding (MOU) .................................................................... 19
  3.2     Capacity Building for EFA (2005) .............................................................................. 19
4 Assessment of Cap-EFA Implementation 2006/2007 ........................................................ 24
  4.1     Cap-EFA in Literacy.................................................................................................. 28
  4.2     Cap-EFA in Teacher Training.................................................................................... 29
  4.3     UNESS ..................................................................................................................... 30
  4.4     Relevance to national priorities ................................................................................. 30
  4.5     Effectiveness ............................................................................................................ 31
  4.6     Efficiency, project and programme management ...................................................... 32
  4.7     Sustainability and chances for scaling-up ................................................................. 34
  4.8     Comparative advantage and added value through family approach .......................... 35
  4.9     UNESCO’s key functions .......................................................................................... 37
  4.10 Upstream or downstream?........................................................................................ 39
5 Conclusions....................................................................................................................... 40
  5.1     Relevance, effectiveness, efficiency and sustainability of Cap-EFA financed activities .
           ................................................................................................................................. 40
  5.2     Cap-EFA response to identified challenges............................................................... 41
6 Recommendations ............................................................................................................ 45
  Annex 1 Terms of Reference ................................................................................................ 48
  Annex 2 List of documents consulted.................................................................................... 54
  Annex 3 List of people interviewed........................................................................................ 58
  Annex 4 LIFE Logframe ........................................................................................................ 61
  Annex 5 TTISSA Logframe ................................................................................................... 62
  Annex 6 Project proposal assessment criteria....................................................................... 63
  Annex 7 Guidance Note to Mid-term evaluations .................................................................. 64
  Annex 8 Cap-EFA budget per country and expenditure component 2006 – 2008 ................. 68
  Annex 9 Cap-EFA cooperating partners and co-funders ....................................................... 70
  Annex 10: Summary of key findings, conclusions and recommendations.............................. 72
  Annex 11: Management Response Table ............................................................................. 74
                 Evaluation of Capacity Building for Education for All (Cap-EFA)
                                          Final Report



List of Abbreviations
BREDA     Regional Bureau for Education in Africa
Cap-EFA   Capacity Building for Education for All
CB        Capacity Building
CCA       Common Country Assessment
CLC       Community Learning Centre
DFA       Dakar Framework for Action
DFID      UK Department for International Development
DG        Director General
DoET      Department of Education and Training
ECDPM     European Centre for Development Policy Management
EDI       Education Development Index
EFA       Education for All
EMIS      Education Management Information System
ERC       Extra-budgetary Funding Sources
ESDP      Education Sector Development Plan
ESMIS     Education Sector Management Information System
EXB       Extra-budgetary
EXBF      Extra-budgetary Funding
FIT       Funds-in-Trust
FITOCA    Funds-in-trust Overhead Cost Account
FO        Field Office
FTI       Fast Track Initiative
GPI       Gender Parity Index
HQ        Head Quarters
HRD       Human Resources Development
ICT       Information and Communication Technology
IGA       Income Generating Activities
IICBA     UNESCO International Institute for Capacity Building in Africa
IIEP      International Institute for Educational Planning
ILO       International Labour Organisation
INGO      International Non-governmental Organisation
IRA       International Reading Association
JICA      Japan International Cooperation Agency
LAMP      Literacy Assessment and Monitoring Programme
LDC       Least Developed Country
LFA       Logical Framework Approach
LIFE      Literacy Initiative for Empowerment
LL-MIS    Lifelong Learning Management Information System



                                                                                   i
                Evaluation of Capacity Building for Education for All (Cap-EFA)
                                         Final Report

LPO       Lead Project Officer
MLA       Main Line of Action
MOET      Ministry of Education and Training
MoEVT     Ministry of Education and Vocational Training
MOU       Memorandum of Understanding
MS        Member State
MTE       Mid-term Evaluation
NC        Nordic Countries
NFE       Non-formal Education
NFE-MIS   Non-formal Education Management Information System
NQFT      National Qualification Framework for Teachers
IOS       Internal Oversight Service (of UNESCO)
PRSP      Poverty Reduction Strategy Paper
PSC       Programme Support Cost
RB        Regional Bureau
RBP       Regular Budget Programme
SSF       Sectoral Strategic Frameworks
STR       Student Teacher Ratio
TA        Technical Assistance
TODAA     Table on Delegated Authority and Accountability
TOR       Terms of Reference
TTI       Teacher Training Institutes
TTISSA    Teacher Training Initiative in Sub-Saharan Africa
UIL       UNESCO Institute for Lifelong Learning
UIS       UNESCO Institute of Statistics
UNESCO    United Nations Educational, Scientific and Cultural Organisation
UNESS     UNESCO National Education Support Strategy
UNDAF     United Nations Development Assistance Framework
UNDP      United Nations Development Programme
UNFPA     United Nations Population Fund
UNGEI     United Nations Girls Education Initiative
UNICEF    United Nations Children’s Fund
UNLD      United Nations Literacy Decade
WB        World Bank




                                                                                  ii
                  Evaluation of Capacity Building for Education for All (Cap-EFA)
                                           Final Report



Acknowledgements

The evaluation team acknowledges the assistance of all the people in UNESCO, in the field visit
countries and in other agencies who gave their time and professional assistance during the
work. We are especially grateful to Ms May Pettigrew in the IOS and Mr Stein Osttveit and Ms
Janne Kjaerskaard Perrier in EO/ED for their continued support to this assignment




                                                                                             iii
                   Evaluation of Capacity Building for Education for All (Cap-EFA)
                                            Final Report




Executive Summary
Capacity Building for Education for All is an extra-budgetary programme to support capacity
building activities for EFA in selected Member States. The programme is pool-funded through a
special account as provided in Memorandum of Understanding that was signed between
UNESCO and five Nordic Countries in 2003. Recently Switzerland has joined the pool. In
accordance with the programme document from 2005, the programme has supported literacy
and teacher training activities in 11 member states with a total programme budget of
approximately USD 11.8 million over two years.

The objective of the evaluation of Capacity Building for Education for All (Cap-EFA) programme
was to improve UNESCO’s performance in education by assessing the utility of the current
Memorandum of Understanding (MOU) to the realisation of Education for All, and to help Cap-
EFA donors and UNESCO decide on future forms of cooperation. More specifically, the Terms
of Reference required an assessment of how, and to what degree the Cap-EFA programme and
its funding modality (the MOU) have contributed to relevant parts of UNESCO education
programme.

The evaluation was conducted over the period December 2007 to March 2008. It concentrated
on the assessment of the current implementation period of the programme between mid-2006 to
mid-2008 at late mid-term point of the programme implementation. Except for some months, the
implementation period coincides with the Programme and Budget 2006-2007 (33 C/5), which is
the last biennium in UNESCO’s Mid-term Strategy 2002-2007 (31 C/4).

The evaluation is based on extensive consultation of relevant background and programme
documentation, field visits to three programme countries (Angola, Tanzania and Vietnam),
Bangkok Regional Bureau and interviews with relevant stakeholders in Paris and in Europe.
The evaluation is essentially a programme level evaluation. The progress in projects at country
level is assessed in mid-term evaluations (MTE) that were carried out in all 11 programme
countries concurrently with this Programme Evaluation.. These are evaluations of the individual
CAP/EFA Programme projects most of which carried out either by external consultants or field
office staff. The findings of the mid-term evaluations are included in this report mainly at a
synthesised programme level to complement discussion of the field visit findings and
information obtained through interviews.

In order to appreciate the role of the Cap-EFA programme and its contribution to UNESCO’s
education programme, the evaluation begins with a contextualisation of the Cap-EFA
programme and description of its relevant operational framework. UNESCO is referred to as a
custodian for EFA with a mandate to coordinate the partners. It is also, as a specialised UN
agency, expected to serve the Member States as the lead technical agency and resource on
EFA. This role is to be fulfilled mainly through UNESCO’s five key functions: 1) idea laboratory,
2) standard setter, 3) clearing house, 4) capacity builder and 5) catalyst for international
cooperation. These functions are primarily enforced by UNESCO field offices (FO’s) with the
help from Head Quarters and UNESCO Institutes and Centres, which all are expected to be
centres of excellence in their respective field of expertise. The evaluation took also note of
UNESCO, and the Cap-EFA programme, prioritisation of upstream policy advice to the Member
States.

A number of challenges were identified in interviews, management documents and in previous
evaluations and studies for UNESCO to effectively deliver on its vast and important mandate.
Similarly, the evaluation identified UNESCO’s responses to these challenges. The identified
challenges and responses (in shaded columns) are summarised in the table below.



                                                                                               iv
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report


    Key Challenge                            Document or other source which identifies the                                                                                   Action taken by
                                             key challenge                                                                                                                   UNESCO which
                                                                                                                                                                             mitigates the
                                                                                                                                                                             challenge




                                                                                                                                                                                                     Capacity Building
                                                 External Auditors




                                                                                          Eval. of Institutes
                                                                                      2




                                                                                                                External Review




                                                                                                                                                                      Action Plan for
                                                                     Eval. of support




                                                                                                                                                    Training course
                                                                                                                                  Internal Review
                                                                     to nat. planning




                                                                                                                                                                                                     Policy work for
                                                                                          and Centres3




                                                                                                                                                                 6




                                                                                                                                                                           EXBF
                                                                                                                                                    for FO Staff




                                                                                                                                                                                        UNESS7
                                                        1
                                    33 C/5




                                                                                                                of CB4

                                                                                                                                        5
                                                 report




                                                                                                                                  of CB
    Limited financial resources              X                                X                     X                   X                X                                    X                  X
    Regular Programme
    alignment with extra-                    X            X                   X                     X                                                                         X                  X
    budgetary funding
    Prioritisation/ unclear focus            X            X                   X                     X                   X                X                 X                  X                  X            X
    Professional capacities/
                                                                              X                     X                   X                X                 X                                                  X
    technical expertise
    Planning to match needs,
    normative role and available                          X                   X                     X                                                                         X                  X
    resources
    Internal cooperation and
                                                                              X                     X                   X                X                 X                                                  X
    coordination
    Lack of standard definition
                                                                                                                        X                X                                                                    X
    of key concepts



Conclusions on relevance, effectiveness, efficiency and sustainability

Assessment against the “traditional” OECD/DAC defined parameters for development
cooperation and evaluations - relevance, effectiveness, efficiency and sustainability – was to a
large extent based on the 11 mid-term evaluations that were carried out in all Cap-EGA
programme countries. On the basis of the MTEs, the evaluation found Cap-EFA
implementation successful.

There are 11 countries with a project underway under the CAP/EFA Programme of which seven
focus upon literacy and 4 focus upon teacher training. These Cap-EFA projects were found
relevant and justifiable to national needs and priorities. Case based field visits were made to
Vietnam, Tanzania and Angola. The small number of site visits means that the generalisability
of the results cannot be validated. However it was clear in each of these cases that there were
also needs at a more upstream policy level that would perhaps have fitted better UNESCO’s
mandate.

1
  Report by the External Auditor on the performance audits undertaken in the 2004/2005, Draft action
plan proposed by the DG on management of extra budgetary funds. 174 EX/INF.4 Rev., Paris 30 March
2006
2
  Evaluation of UNESCO Support to National Planning for EFA, Education for Change 2006.
IOS/EVS/P1/53.
3
  Consolidated Findings of the Evaluations of UNESCO’s Major Programme I Institutes and Centres, Nick
Davis, February 2006. IOS/EVS/PI/58 UNESCO,.
4
  Review of UNESCO’s Capacity-Building Function, Stiles Associates Inc. IOS/EVS/P1/71, UNESCO
2007
5
  Capacity Development for Achieving EFA Goals: UNESCO Strategy; Review of UNESCO’s Capacity
Development Programmes, Final Review Report, internal working document, February 2008.
6
  Training Course Evaluation based on written and oral comments by the participants as recorded by the
organisers of the training course, Professional Upgrading of UNESCO Education Sector Personnel:
Training Course on Education Policy Analysis – A joint undertaking by ED/EO, ED/SFS and IIEP, 21 Jan
to 1 Feb 2008. Participants included 28 staff members, of whom 21 were from Field Offices (FOs).
7
  http://unesdoc.unesco.org/images/0014/001485/148566E.pdf(UNESCO Education Sector Strategies)


                                                                                                                                                                                                                         v
                    Evaluation of Capacity Building for Education for All (Cap-EFA)
                                             Final Report

The mid-term evaluations found Cap-EFA effective in delivering for results and in developing
policies and programmes. This was confirmed in the field visits. However in these cases there
were some, reservations with regard to quality (Vietnam) and the small number of beneficiaries
(Tanzania). In Angola the evaluation found problems in both of these aspects of effectiveness.

The mid-term evaluations deemed Cap-EFA also efficient. The main challenge identified in the
field visits was the heavy administration load caused by UNESCO rules and procedures in
project implementation. It has put a substantial burden on UNESCO staff and in some cases
also on the cooperating partners.

On sustainability, the mid-term evaluations unanimously believed that the benefits from the
programme will last after the completion of the extra-budgetary support. During the field visits,
the sustainability and scaling up was not deemed as foregone conclusion. The sustainability
and chances for scaling up piloted activities depends on the financing from 1) Member State
national budget and/or 2) other development partners. The cooperation with especially the
multi- and bilateral development partners was found limited. Despite limited cooperation and
especially co-funding, the evaluators, through their field visits, found UNESCO to be well
perceived by the development partners and Member State governments in the countries visited.
UNESCO has played an active role in coordination in the education sector, especially within the
UN system.

The evaluation believes that the challenges that were identified in the field visits, especially
linked to relevance, effectiveness and sustainability, could be addressed and mitigated during
planning phase with the help of a more rigorous appraisal (or ex-ante evaluation) process to
ensure the objectives, needs, implementation, expected outcomes and monitoring are clearly
defined and operationable (or operationalised?)..

Cap-EFA response to the identified challenges

The challenges that are included in the above table are those extracted by the evaluation
through its analysis of previous evaluations and relevant documentation. They are not
mentioned or explicitly referred to in key Cap-EFA documents - the Memorandum of
Understanding or in the Capacity Building for EFA programme document (2005). Yet the
evaluation found Cap-EFA to be a deliberate and successful response to most of them. The
summary of the responses from Cap-EFA to the identified challenges is presented in the table
below.

             Challenge                                   Cap-EFA Response
   Limited financial resources         Added USD 11 million.
   Regular Programme alignment         Aligned with the priorities stated in the 33 C/5 through
   with extra-budgetary funding        MOU that enables pooled and harmonised support.
   Prioritisation/ unclear focus       Thematic focus on prioritised literacy and teacher
                                       training, emphasis on LDCs less evident as well as
                                       exclusiveness on upstream policy advice.
   Professional capacities/            Added human resources (683 person months).
   technical expertise                 However, it is not certain if this has increased
                                       professional capacities for policy advice.
   Planning to match needs,            UNESS
   normative role and available
   resources
   Internal cooperation and            Family approach (collaboration between the different
   coordination                        parts of the organisation including institutes in carrying
                                       out an initiative).
   Lack of standard definition of      Not identified.
   key concepts



                                                                                                    vi
                   Evaluation of Capacity Building for Education for All (Cap-EFA)
                                            Final Report



Additional resources, harmonisation and alignment
Cap-EFA has played a major role in the implementation of UNESCO’s education programme.
The available USD 11 million over two years is a substantial amount of additional financing in
UNESCO’s education budget. Moreover, as the Cap-EFA funds are allocated to two programme
areas (literacy and teacher training) that are relatively prioritised also in UNESCO’s regular
budget (i.e. limited fungibility), the evaluation concludes that Cap-EFA has, to a considerable
extent, enabled and supported implementation of “the programmatic core to of UNESCO’s
response to EFA at country level”, as described in 33 C/5.

The programme has enabled a substantial, 659 person-months increase in human resources in
the field offices, in addition to the mobilisation of a vast number of international and national
experts. This contributes greatly to the challenge of limited resources in the FOs. Cap-EFA has
also enabled project implementation in countries where UNESCO does not have a permanent
office, which is a challenging constellation from quality assurance and management back-
stopping point of view.

The Memorandum of Understanding between UNESCO and the Nordic Countries (and
Switzerland) has allowed integrated planning and programming and full coherence of regular
budget and extra-budgetary funds, which has been identified as a major concern with the extra-
budgetary funding. In UNESCO’s Education Sector, the MOU is referred to as a model for
thematic and flexible funding modalities within the upcoming Education Sector strategy for
extra-budgetary funding.

The MOU is also referred to as the most harmonised and aligned way of EXBF support to the
regular programme and budget in terms of implementation and management modalities. The
identified benefits include for example simplified administrative procedures, joint donor meetings
and consolidated reporting. The alignment goes, however, only half way as Cap-EFA planning
and reporting has not followed UNESCO standard procedures.

Cap-EFA has introduced management tools, formats for country reporting, and proactive
financial monitoring, which function well and are appreciated by the Field Offices. The use of
standardised reporting formats is not extended to the consolidated reports, which suffer from
the programme’s unclear objective setting and lack of clear programme logic, which the
reporting should follow. The consolidated reports should include more detailed financial
information on what has been financed, and make sure that the narrative reporting follows the
finances. More detailed financial information is available through the regular financial
monitoring system.

Prioritisation
Cap-EFA has three prioritisations: needs-based orientation to favour the Least Developed
Countries (LDCs) as per the MOU, thematic prioritisation to literacy and teacher training in
accordance with the Cap-EFA programme document (2005) and upstream capacity building as
per the invitation letter to the FOs to seek financing from Cap-EFA.

The majority of the countries benefiting from Cap-EFA are among the LDCs, but the
prioritisation was not unequivocal and in relative terms more non-LDCs were selected from the
eligible countries. This is probably due to the selection criteria, which emphasised technical
quality of the proposals higher than the needs in the country. The evaluation saw little added
value seen in the “competitive” selection process for the pilot countries. Selection should be
based on needs, not on technical merit of proposals, which can easily lead to supporting
countries and offices that least need that support.

The support from Cap-EFA has been channelled directly to UNESCO’s key priorities, literacy
and teacher training, greatly promoting thematic prioritisation of the education programme.


                                                                                               vii
                    Evaluation of Capacity Building for Education for All (Cap-EFA)
                                             Final Report

There has been discussion of whether the top-down thematic prioritisation of literacy and
teacher training has left other relevant and potentially more needed projects without support.
UNESCO’s mandate covers the whole EFA and the needs in the Member States for EFA are
vast and go beyond literacy and teacher training. In addition to the identified needs in the
Member States, the most defining factor in selecting substance areas for UNESCO support
should be the availability of the required expertise in UNESCO (capacity to deliver), and
suitability to UNESCO’s mandate.

The programme emphasis on upstream policy advice is considered ambiguous8. In addition to
the ministry level cooperating partners, all projects engage in down-stream activities of e.g.
training of teachers or illiterates together with mid-stream actors such as regional
administrations, NGOs and teacher training institutes. While this is important, it does not
necessarily strengthen the image of UNESCO as an upstream policy adviser to the ministries
and policy makers in the Member States. The evaluation was not able to conclude on the
reasons for the relative “downstreamism”.

Overall, the evaluation has found the implementation of the programme is largely successful;
the projects are, to a great extent, in accordance with the identified needs in the selected
countries, and they are in line with UNESCO strategies and policies. However it has also
identified that the strategic orientation of the programme should be sharpened towards more
upstream policy advice; that there is a need to focus support on countries with the biggest need
for support and while the programme should cover the whole UNESCO EFA mandate, it needs
to give preference, in its selection of initiatives to those themes and areas where UNESCO’s
capacity to deliver has been identified.

Technical expertise, internal cooperation and coordination
Through the UNESCO family approach (and its encouragement of co-operation among the
different parts of the organisation, including institutes, in progressing and initiative??), Cap-EFA
has greatly promoted service provision from within UNESCO on the basis of needs identified by
the field offices (FOs). The family approach has been generally well received in the FOs but
they also report on challenges in identification, coordination and availability of experts from
within the family. Another identified problem is linked to the short duration of services the
UNESCO family members can provide. To ensure effectiveness and impact of the services, the
field office staff should be able to go back to family members as needed in order to follow up on
the services. Clearly sufficient resources for this would need to be ensured.

On the service provider side, the institutes and regional bureaux are struggling with limited
resources to meet the requests from the field offices, which may come in short notice and/or
with unclear definition of roles and expectations. The evaluation concludes that the family
approach should be further strengthened through improved communication and closer
engagement of the family members in the institutes, centres and HQ in planning and monitoring
of the projects. More active participation throughout the project cycle would help ensure that the
required services are available and the experts are better aware of the expectations.

The evaluation was not able to accomplish the extent to which Cap-EFA has strengthened the
upstream technical advisory and capacity building role of the field offices.

Planning
Cap-EFA has been instrumental for UNESCO to pilot and roll-out UNESCO National Education
Support Strategies (UNESS), which are hoped to greatly improve the relevance, effectiveness
and sustainability of UNESCO’s activities at the country level. Planned implementation of
UNESS will provide UNESCO with strategic planning documents that 1) ensure relevance of

8
 This conclusion needs to be seen against the background of no clear definition for upstream and
downstream functions and activities.


                                                                                                   viii
                   Evaluation of Capacity Building for Education for All (Cap-EFA)
                                            Final Report

UNESCO’s activities to countries’ needs and priorities, 2) increase cooperation and
harmonisation with development partners, especially within UN-system, and 3) improve
effectiveness of UNESCO’s activities. The fulfilment of these objectives remains to be seen as
overwhelming majority of the documents are still only found in draft form.

Definition of key concepts and objectives related to the Programme
Implementation of the Cap-EFA programme has suffered from a number of inconsistencies and
areas of lack of clarity, some of which go beyond Cap-EFA and should be addressed to
UNESCO as a whole. Others, however are programme specific.

The objectives in the programme document (2005) are largely coherent with UNESCO
Programme and Budget (33 C/5) and the Strategy and Vision for LIFE. They are in the main
also coherent with TTISSA objectives as well. Where there is a clear discrepancy in this respect
in TTISSA, this is explained by the fact that there has been a revision of TTISSA objectives and
expected results and these have been made after the Cap-EFA programme and its activities
were planned. The main problem, however, is identified in the definition of problems to be
addressed that is not at a level that can be realistically addressed by the Cap-EFA programme.
This has led to unclear programme intervention logic with a mismatch between the challenges
and the available resources, missing hierarchy of objectives and lack of quantified indicators.
The evaluation was not able to clearly identify which objectives are promoted and with what
level of budgetary allocations. While contribution of Cap-EFA to LIFE and TTISSA is
acknowledged by all, it was not possible to quantify that added value in more detail.

Consolidated Cap-EFA programme reporting has been negatively affected by both unclear
objective setting and lack of definition of programme logics and conceptual frames.

UNESCO Medium-term strategy (31 C/4) and Programme and Budget (33 C/5) discuss
upstream and downstream functions in connection with the key functions. Unfortunately, the two
papers offer different definitions. In this evaluation downstream and upstream are understood
through the immediate beneficiaries of the provided services.

There is no definition of how the five key functions are to be operationalised. The difficulty of
this task is evidenced e.g. in the reviews and in the on-going work to define UNESCO’s policy
for capacity development.

UNESCO’s comparative advantage and added value have been addressed in progress
reporting, mid-term evaluations and in this evaluation to a varying degree. However, in the
absence of definition for these key concepts, it has proved very difficult to locate them in the
activities. For example, UNESCO strategy and policy papers point out that literacy and teacher
training are often neglected priorities in the Member States, and UNESCO can provide added
value and comparative advantage to promote them. This claim would benefit from an
evidenced-based justification and more detailed definition of what kind of added value and
comparative advantage UNESCO can bring, how and by whom. There is a need for programme
(if not organisation) level definition with clear justification and understanding how the added-
value and comparative advantage will be delivered. As a specialised UN agency for education,
it would be expected that the required expertise in UNESCO’s core areas would be readily
available in-house to Member States and field offices.

In planning for the country level interventions, the Field Offices have faced difficulties in
identifying the available expertise and resources in UNESCO.

Recommendations

The evaluation recommends Capacity Building for EFA-programme to be continued and
believes that with some modifications and strategic reorientation, it can play even bigger role in


                                                                                                ix
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

enhancing UNESCO’s mandated service delivery to the Member States. The Memorandum of
Understanding between UNESCO and the contributing donors should cover the six-years period
of the Medium-Term Strategy 2008-2013 (34 C/4) and the operational programme guidelines
and objectives should be revised and defined on a biennium basis, following the C/5
documents.

The evaluation believes that the revised Memorandum of Understanding would benefit from the
existing best practices in donor harmonisation as outlined for example in Joint Financing
Arrangements and should include more detailed definition of e.g.:
           a. Objectives of the programme and scope of the MOU with reference to the
              relevant strategic documents and the programme guidelines, which include
              identification of challenges and hierarchy of objectives with indicators to respond
              to the challenges (Cap-EFA programme document to be revised biannually).
           b. Contributions, donors’ pledges;
           c. Consultations and decision making process;
           d. Disbursements;
           e. Procurement;
           f. Reporting;
           g. Review and evaluation;
           h. Auditing; and
           i. Modifications, donor accession and withdrawal

The Cap-EFA support should invoke country ownership and be based firstly upon needs in the
Member States and on national education plans and be part of the joint efforts at country level.
A local education group should verify the project/programme request before it is sent to
UNESCO headquarters. While ensuring full coherence between the Cap-EFA funding and
UNESCO regular programme, following from the identified challenges in UNESCO for effective
service delivery, the evaluation recommends that the objective of the revised Cap-EFA
programme is geared towards strengthening UNESCO capacity to deliver on its mandate
and to better serve the Member States in Education for All. The programmatic nature of the
Cap-EFA programme document should be strengthened with clear and explicit identification of
challenges and problems to be addressed by Cap-EFA, followed by detailed definition of the
programme intervention logic and definition of indicators against which the progress can be
measured. There must be clear goals and objectives that permit easily reportable results.
There is a need for further discussions on the kinds of results that are appropriate for these
kinds of interventions/programmes.

Regarding prioritisation, the evaluation recommends that the support from Cap-EFA should
be based on two basic parameters:
   1. Needs in the member states where explicit priority should be given to a) the LDCs9 in
      the selection of countries and b) the needs for policy advice in these countries in
      selection of themes. Recent UNESS exercises can provide useful information on this.
   2. UNESCO’s capacity to deliver. Preference should be given to the LDCs with UNESCO
      FO in the country. Thematic preference should be given to the areas where UNESCO
      has proven expertise and identified in-house delivery capacity.

Aligned with the draft 34 C/5, Cap-EFA should further accentuate upstream policy advice.
Downstream project implementation should be allowed only in exceptional circumstances with
clear added value and feed-back mechanism to support UNESCO’s policy advisory role.

The Cap-EFA programme document should define UNESCO’s comparative advantage and
added-value with clear identification of actual service providers in UNESCO. Cap-EFA
should build on the already existing expertise in UNESCO and strengthen expertise in areas

9
    Or countries with the lowest Education Development Index in the Global EFA Monitoring Reports.


                                                                                                     x
                   Evaluation of Capacity Building for Education for All (Cap-EFA)
                                            Final Report

that are deemed important but where there is a need for more in-house expertise. There should
be a mechanism to ensure that utilisation of external experts strengthens also UNESCO’s
capacity, without compromising the quality or quantity of support to the Member
States/beneficiaries. Cap-EFA would benefit from cooperation with the on-going preparations of
UNESCO’s policy for capacity development.

Once the available expertise in UNESCO is mapped, the identified in-house experts should
participate closely in the preparation and in monitoring of the intervention funded by
Cap-EFA to ensure smooth delivery of expertise from within UNESCO.

Relevance, efficiency, effectiveness and sustainability of the activities could be further improved
by applying more rigorous appraisal (or ex-ante evaluation) procedures.

The consolidated reports should include more detailed financial information on what has been
financed, and make sure that the narrative reporting follows the finances. More detailed
financial information is available through the regular financial monitoring system.




                                                                                                 xi
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



1          Background to the Evaluation
In 2003 UNESCO and five Nordic Donors signed a Memorandum of Understanding (MOU) in
which the Nordic countries promised to provide funds to UNESCO for capacity building activities
in Member States through a special multi-donor account. The MOU stipulates that in the
programme Capacity Building for Education for All (Cap-EFA) -Extra-budgetary programme for
Technical Support to Member States, the support is to be based on UNESCO’s Medium-term
Strategy 2002-2007, Programme and Budget and International Strategy to Operationalise the
Dakar Framework for Action on Education for All, and funds are provided for capacity building
as described in UNESCO Policy Statement on Cooperation with Donors for the Dakar Follow-up
Action and Capacity Building for EFA –document.

Operational guidelines for the Cap-EFA programme are outlined in Capacity Building for EFA –
document, which is revised according to UNESCO’s biannual planning and implementation
cycles. The current Capacity Building for EFA -document is dated November 2005.

In February 2007, UNESCO and the Nordic donors to the Cap-EFA programme concluded that
year 2007 should be used as a transition period to critically review the programme and the
MOU. On the initiative of the donors, with Norway as the lead, it was agreed that an external
evaluation of the programme should be prepared and undertaken by UNESCO's Internal
Oversight Service (IOS). Recommendations of this evaluation will feed into the revision of the
draft Cap-EFA Programme Document 2008/2009 and the MOU.


           1.1         Objective of the Evaluation

The overall objective of the evaluation, as stated in the Terms of Reference (TOR), is:

“To improve UNESCO performance in education by assessing the utility of the current MOU for
 extra budgetary support to the realisation of Education for All and help donors and UNESCO
                            decide on future forms of cooperation.”

The Terms of Reference (TOR) (Annex 1) require an assessment of how, and to what degree
the Cap-EFA programme and its funding modality (the MOU) have contributed to relevant parts
of UNESCO education programme. The evaluation is to follow two main lines of inquiry:
    1. The effect of the Cap-EFA programme at the education sector programme level, and
    2. The usefulness of the MOU as a modality for joint donor financing.

These two broader evaluation questions are operationalised into a number of evaluation
questions in the TOR.


           1.2         Methodology

The evaluation was conducted in a phased manner as presented in the Figure 1 below.




                                                                                             1
                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                     Final Report

Figure 1: Overview of methodology

                Literature review and interviews to obtain overview
                          and understanding of Cap-EFA
                                                                                               First Phase



                     Inception report with elaborated evaluation
                                      questions                                               Second Phase


     Further reading of    Interviews with UNESCO HQ,        Field visits, interviews
        background         Nordics, UNESCO institutes,       with FO’s, national
        documents          regional bureaux                  stakeholders,                    Third Phase
                                                             development partners



                                    Assessment                                                Fourth Phase


                          Conclusions and Recommendations
                                                                                                 Final




To contextualise the Cap-EFA programme, the evaluation started with an analysis of secondary
data, comprising 1) programme management documents, 2) UNESCO strategy and policy
documents and 3) existing evaluations and other studies. Full list of consulted documents is
included in Annex 2.

During the second phase, the preliminary analysis of the Cap-EFA programme in its relevant
operational framework was included in the Inception Report. The Inception Report, after
discussion with the evaluation Reference Group, also included the elaborated evaluation
questions guiding the subsequent phases of the evaluation.

The third phase of the evaluation concerned with primary data collection through interviews and
field visits to ensure sufficient depth and breadth to the assessment, and subsequent
conclusions and recommendations. This was complemented with further reading of background
documentation, including mid-term evaluations that were carried out in all Cap-EFA programme
countries, and the latest progress reports.

Relevance, value added and usefulness of the Cap-EFA and the MOU for UNESCO were the
key issues addressed in the discussions and interviews with reference group members and
other relevant officers in UNESCO and Nordic donors’ representatives.

Field visits were carried out in Angola (including visit to Windhoek Cluster Office, Namibia),
Tanzania and Vietnam10. The field visits concentrated on two main evaluation issues: relevance
of the Cap-EFA activities to the national needs and priorities and cooperation with the main
stakeholders, and the possible added-value of the services through UNESCO’s comparative


10
  Tanzania and Vietnam are both One-UN pilots and were believed to provide useful lessons for added
value of UNESCO in countries with heavy donor participation. Angola was believed to give insights on
CapEFA operations in a country with no UNESCO FO. Finally in all countries the credibility of the findings
could be ensured, which was deemed important due to the complexity of the evaluation issues



                                                                                                             2
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

advantages, the five key functions (described in section 2.1 below), and the UNESCO “family
approach” (described in section 3.2.2 below).

In addition to the UNESCO Field Office (FO) officers, the key stakeholders interviewed during
the field visits included the relevant Ministry officials and other development partners, especially
within the UN system.

Due to the importance of the “family approach” for UNESCO’s provision of services, the
evaluators visited Bangkok Regional Bureau to get insight into UNESCO’s internal cooperation
and utilisation of UNESCO’s internal expertise. The list of people interviewed is included in
Annex 3.

            1.3         Structure of the report
This report presents the assessment of the programme leading to conclusions and
recommendations. It begins in chapter 2 with a contextualisation of the Cap-EFA programme by
discussing briefly UNESCO mandate in EFA and the identified challenges in delivering on it.
The report presents the operational environment as framed in the UNESCO Medium-Term
Strategy 2002-2007 (31 C/4) and in the previous biannual Programme and Budget (33 C/5)
followed by discussion on extra-budgetary funding and conclusions and recommendations of
previous evaluations and studies that were found to be relevant for this evaluation.

This is followed in by presentation in chapter 3 and assessment in chapter 4 of the Cap-EFA
Programme and its implementation in 2006-2008, including a discussion on the MOU, and the
objectives and management as described in the programme document (2005).

The report ends with conclusions and recommendations that are hoped to help UNESCO and
the programme donors to decide on the future co-operations and provide input to the revision of
the Capacity Development for EFA –programme document and Memorandum of
Understanding.

            1.4         Limitations to the evaluation
Evaluations, as other organisational development activities, are carried out with limited
resources. In the case of this evaluation, the number of countries visited does not necessarily
allow for generalised statements or conclusions at the programme level. For example, due to
the limited number of countries visited in this evaluation and as verification of all MTE findings
was not possible at country level, the findings from the MTE evaluations are referred to mainly
at the programme level only and complemented with findings from the field visits. The
evaluation attempts to present the field visit findings to illuminate and strengthen the findings
and key issues identified in other sources of information. In some cases the field level findings
are presented to highlight potentially critical issues to which attention should be paid in the
future.

While evaluating the effect of the Cap-EFA at the education sector programme level, it is
important to bear in mind that the programme is still on-going in its setting of two-year
interventions following the biennial programming. Therefore the assessment and conclusions
refer to potential for effectiveness, efficiency, impact and sustainability of the programme.




                                                                                                  3
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report


2           Contextualising Cap-EFA
            2.1         UNESCO Mandate and Core Functions
UNESCO is the main actor behind the Education for All –movement (EFA). The World
Education Forum that convened in Dakar in 2000 adopted the Dakar Framework for Action
(DFA) to Achieve Education for All which entrusts UNESCO with the task to “continue its
mandated role in coordinating EFA partners and maintaining their collaborative momentum”. In
addition to the mandate to coordinate the initiative, the DFA invited UNESCO to “refocus its
education programme in order to place the outcomes and priorities of Dakar at the heart of its
work”. 11

Accordingly, UNESCO’s Medium-Term Strategy 2002/2007 (31 C/4) establishes the six goals of
the DFA as the overriding priority for UNESCO’s education strategy and the follow-up to Dakar
as the guidance for shaping future activities of the organisation.12

The Strategy ensures that “UNESCO will perform and assume a range of functions
corresponding to its role as the international lead agency for education”. These key functions
include13:
 • a laboratory of ideas, which entails anticipating and defining the most important emerging
    problems, and identifying appropriate strategies and policies to deal with them.
 • a standard-setter, serving as a “central forum for articulating the ethical, normative and
    intellectual issues of our time” and working towards universal agreements, benchmarking
    targets and mobilising international opinion.
 • a clearing house, consisting of gathering, transferring, disseminating and sharing available
    information, knowledge and best practices, and identifying innovative solutions and testing
    them through pilot projects;
 • a capacity-builder in Member States, in which UNESCO will organize international
    cooperation for servicing its stakeholders, especially its Member States, in building human
    and institutional capacities;
 • a catalyst for international cooperation, in which UNESCO will assume a catalytic role for
    development cooperation and it will seek to ensure that the objectives, principles and
    priorities it promotes are followed suit by other multi- and bilateral programmes and that
    projects are implemented, in particular at regional and national levels, through innovation,
    effective interventions and wise practices.

The Strategy (31 C/4) envisages both “upstream” and “downstream” functions for UNESCO.
According to the Strategy, the “upstream” work consists of advisory services to the Member
States for policy analysis and formulation, monitoring and reporting on the basis of UNESCO’s
role as a laboratory of educational ideas and setter of educational standards. UNESCO’s
downstream role will focus on capacity building, information exchange, and other modalities of
international cooperation. Particular attention will be paid to the generation of feedback loops so
that field experience and knowledge from capacity-building and information networks can enrich
and illuminate the upstream work. This feedback mechanism should further improve the
upstream policy advice UNESCO provides to the Member States.14




11
   Dakar Framework for Action, Education for All: Meeting our Collective Commitments, Text adopted by
the World Education Forum in Dakar, Senegal, 26-28 April, 2000.
12
   Approved Medium Term Strategy 2002-2007, 31 C/4, Contributing to peace and human development in
an era of globalisation through education the sciences, culture and communication, UNESCO 2002.
13
   ibid. para 30.
14
   ibid. paras 55 and 56.


                                                                                                   4
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

The Director General’s report on the strategic review           The key service providers in UNESCO
of     UNESCO’s      post-Dakar   role    in    EFA15           for the defined activities and functions
acknowledges that UNESCO’s dual role to                         are the Field Offices, which include 23
coordinate and to refocus in EFA has been a                     cluster offices (covering 143 Member
continuing challenge to the organisation. The report,           States) and 21 National Offices in highly
however, reassures that “…(I)n the period ahead, a              populated countries or in countries that
more confident and proactive UNESCO will assert its             are emerging from conflicts. Field offices
role as leader in education and as coordinator-in-              are supported by four Regional Bureaux
                                                                (Dakar, Beirut, Bangkok and Santiago de
chief of EFA at the international, regional, sub-
                                                                Chile) and eight UNESCO Institutes and
regional and country level. This will be pursued                Centres, which are to serve as “centres
through greater coherence, focus and concentration              of research and training excellence”, and
in its programmes; more efficient management of its             providers of technical support and
human       and   financial  resources;      improved           expertise in their area of specialisation to
coordination between Headquarters, the institutes               the Member States, the Secretariat and
and field offices; and a more proactive role at the             the Field Offices.
country level.”16

The strategy to improve UNESCO’s own activities is explained and presented in the Report by
the Director-General on the follow-up to the EFA Strategic Review and UNESCO’s Strategy for
2005-201517, which can be seen as UNESCO’s response to reverse the prevailing trend of EFA
not being on track with regards to achievements of its goals. The outlined strategy identifies four
strategic priority areas for UNESCO:
    1. Assist countries in education planning for EFA in particular through policy advice and
        capacity building;
    2. Coordinate the EFA coalition and maintain EFA momentum;
    3. Lead international initiatives in three areas of comparative advantage for UNESCO:
        literacy, teacher training in Sub-Saharan Africa, and HIV/AIDS and education; and
    4. Promote and assist in the development of quality education.

The Strategy 2005-2015 identifies the key functions for UNESCO to serve both as the
specialised United Nations agency for education and as lead coordinator of EFA partners.
These functions include, in addition to the above mentioned five key functions, progress
monitoring, resource mobilisation and promotion of quality education.18


           2.2         EFA in UNESCO Programme and Budgets

           2.2.1       Programme Priorities
The above objectives are operationalised in the UNESCO programme and budget (33 C/5) for
2006/2007. The principal priority in the programme is defined as: “Basic education for all, with
special attention being given to literacy, HIV/AIDS prevention education and teacher training in
sub-Saharan Africa”. The Programme (33 C/5) argues that EFA is a key development issue and
the 2000 Dakar Framework for Action on EFA and its six goals will continue to guide UNESCO’s
action in the field of education.19

Despite the stated conviction that Education for All can become a reality through concentration
of efforts and resources behind clearly articulated goals, huge challenges remain and progress


15
   170 EX/8, para 6, August 2004.
16
   ibid. Para 141.
17
   171 EX/8, March 2005.
18
   ibid. Para 26.
19
   Approved Programme and Budget 2006-2007, 33 C/5, UNESCO 2006.


                                                                                                               5
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

has been slow20. The UNESCO programme provides leadership in accelerating progress by
enhancing its role as lead coordinator of all EFA partners and by concentrating its own
programme activities and resources in three above stated priority areas, namely: 1) literacy, 2)
teacher training, and 3) HIV/AIDS and education. In the 33 C/5 document, these areas are
considered to be neglected in global EFA efforts and require sustained and coordinated action if
EFA goals are to be achieved. They are also considered to be areas where UNESCO has
comparative advantage and expertise to offer. Consequently the programme foresees
reinforced country-level actions through three new core initiatives:
     1. Literacy Initiative for Empowerment (LIFE);
     2. Teacher Training Initiative for sub-Saharan Africa (TTISSA); and
     3. Global Initiative on HIV/AIDS and education (EDUCAIDS).

These initiatives form “the programmatic core of UNESCO’s response to EFA at the country
level”. They are designed to respond to national needs and priorities in countries that are most
in need of support21 and where UNESCO’s intervention will have the greatest impact. To ensure
that the activities are complementary and mutually reinforcing, the 33 C/5 document informs
that UNESCO will adopt a coordinated approach to planning, implementation and evaluation,
involving Headquarters, field offices and institutes. The 33 C/5 further assures that UNESCO’s
action will be conceived in partnership with the countries themselves and with other EFA
stakeholders.

As the LIFE and TTISSA initiatives have formed the core of the Cap-EFA programme, these are
further described below.

            2.2.2       Literacy Initiative for Empowerment (LIFE)
LIFE, coordinated by UNESCO Institute for Lifelong Learning (UIL), is a ten-year (2006-2015)
global strategic framework for the implementation of United Nations Literacy Decade (UNLD)
(2003-2012). It aims at accelerating literacy efforts through collaborative action in the countries
that have a literacy rate of less than 50% or an adult population of more than 10 million without
literacy competencies. Within these parameters, LIFE targets the 35 countries with 85 % of the
world’s illiterate people.22

As specified in 33 C/5, LIFE provides a framework of support for literacy training based on
proven methods. The work includes both upstream functions, such as political motivation and
policy development, and downstream activities such as technical assistance, capacity building
and establishment of monitoring mechanisms.23

The overall objective of LIFE is to empower people, especially women and girls, who have
inadequate literacy skills and competencies. To achieve this, the revised Strategy and Vision for
LIFE (2007)24 identifies the following four immediate objectives:
    1. To reinforce the national and international commitment to literacy through advocacy and
       communication
    2. To support the articulation of policies for sustainable literacy within sector-wide and
       national development frameworks
    3. To strengthen national capacities for programme design, management and
       implementation


20
   See for example Global EFA Monitoring Reports, http://portal.unesco.org/education/en/ev.php-
URL_ID=49591&URL_DO=DO_TOPIC&URL_SECTION=201.html
21
   LDCs, sub-Saharan Africa, E-9 countries, countries in crisis or post-conflict countries
22
   LIFE presentation at UIL http://www.unesco.org/uil/en/focus/litinforemp.htm and UNESCO Education
Sector’s Communication Papers, 20 March 2008.
23
   33/C/5, page 36.
24                                             rd
   LIFE 2006-2017 Vision and Strategy Paper (3 edition), UIL, September 2007,


                                                                                                      6
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

     4. To enhance countries’ innovative initiatives and practices in providing literacy learning
        opportunities

LIFE logframe including expected results for the immediate objectives is included in Annex 4.

            2.2.3       Teacher Training Initiative in Sub-Saharan Africa (TTISSA)
TTISSA is a ten-year (2006-2015) UNESCO initiative that aims at increasing the quantity and
quality of the teaching force in Sub-Saharan Africa. According to TTISSA Concept Note (2006),
the identified challenges linked to enrolment, recruitment, salaries, attrition, HIV and AIDS,
gender, retirement, policies, training, professional development, and working conditions are
addressed through a holistic approach and at various levels. Activities, such as advocacy, policy
development, studies, production of toolkits and teaching and learning materials, regional
workshops, and capacity building will be carried out by UNESCO HQ, BREDA, IICBA and the
FOs in the region. Special attention will be given to the advocacy and adherence to the joint
UNESCO/ILO Recommendation Concerning the Status of Teachers (1966) and the UNESCO
Recommendation Concerning the Status of Higher Education Teaching Personnel (1997).25

The TTISSA logframe26 includes the following four outputs:
   1. Status and working conditions of teachers improved
   2. Teacher management and administration structures improved
   3. Appropriate teacher policies developed
   4. Quality and coherence of teacher professional development enhanced

The logframe was developed in 2006 in collaboration with the field offices, and the logframe
outputs and activities are included in Annex 5.

            2.2.4       Budgetary Priorities
UNESCO education budget for 2006/2007 as presented in 33 C/5 is divided into four
programmes, which are further divided into nine sub-programmes and 25 Main Lines of Action
(MLA).

Table 1: UNESCO Education Budget 2006/2007
                                                          Regular budget
                                                              Indirect
                                                              prog.     Total       Extra-
                                                              cost (to regular    budgetary   Total
                                      in MUSD Activities      HQ)       budget    resources resources
Strengthening EFA coordination and planning         8.91           0.15      9.07     10.57     19.64
Attaining basic education for all                 17.26            0.30    17.56      20.23     37.79
Enhancing quality education                         6.47           0.11      6.58     46.92     53.50
Supporting post-primary education                   4.41           0.08      4.48     41.41     45.89
           Sub-total Education Programmes         37.05            0.64    37.68     119.14    156.82
Education Institutes                              16.89                    16.89                16.89
Projects related to cross-cutting themes            1.05                     1.05                1.05
Personnel                                                                  52.18        3.82    55.99
                                        TOTAL     54.99            0.64   107.80     122.95    230.76
(Source: UNESCO Programme and Budget 2006/2007)




25
 UNESCO’s Teacher Training Initiative for Sub-Saharan Africa (TTISSA) 2006-2015, Concept Note
26
 UNESCO’s Teacher Training Initiative for sub-Saharan Africa (TTISSA) 2006-2015 logframe,
ED/HED/TED/2007.


                                                                                                        7
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

The above summary table of UNESCO Education budget for 2006/2007 shows that the regular
budget for programme activities (excluding personnel, UNESCO institutes and extra-budgetary
funding) is very small, only MUSD 37 for two years27. Consequently, it is also apparent that the
financing, and hence implementation, of the programme activities depends heavily on extra-
budgetary funds, which at the time the 33 C/5 was approved (October 2005, General
Conference) covered 76 % of the programme resources. It needs to be emphasised that the
figures for extra-budgetary funding in 33 C/5 are either already received or firmly committed but
they do not represent the total resources available for UNESCO during the biennium as
additional extra-budgetary funding may come in at any point of time.

The impact of the EXBF to the financed priorities in UNESCO programme and budget for
2006/2007 at the time the 33 C/5 was approved is explained below.

Table 2: Budgetary allocations to Literacy, Teacher Training and HIV/AIDS and Education in
UNESCO budget 2006/2007 in MUSD and in %.
                                                       In MUSD                            Percentages
                                      Regular
Sub-programmes                         Budget                                                           % of total
               and                    program                        Total      % of                     activity
                 Main lines of Action    me                        resource    regular         % of     resource
                                      activities         EXB           s       budget          EXB          s
LIFE and UNLD (sub-programme)        6.17     6.3      12.46    16.7     5.3                            8
             Implementing LIFE (MLA)     4.12     5.75     9.87     11.1                          4.8         6.3
  Promoting learning opportunities for all
    through NFE within the framework of
                           UNLD (MLA)          2.05     0.55      2.6               5.5           0.5         1.7
Teacher Education (sub-programme)          3.36     10.17    13.53    9.1                  8.5          8.7
           Implementing TTISSA (MLA)           2.35     0.11     2.46               6.3           0.1         1.6
   Professional development of teachers
         and education personnel (MLA)      1.01    10.06    11.06                  2.7           8.4         7.1
HIV/AIDS and education (sub-
programme)                              1.25     8.3      9.55     3.4                     7            6.1
         Leading the EDUCAIDS (MLA)         0.84     0.41     1.25                  2.3           0.3         0.8


 Supporting comprehensive responses to
     HIV/AIDS through education (MLA)    0.42     7.89     8.31      1.1    6.6    5.3
                               TOTAL 10.78    24.77    35.54    29.2 % 20.8 % 22.8 %
(Source: UNESCO Programme and Budget 2006/2007)

In 33 C/5, the three priority sub-programmes: literacy, teacher training, and HIV/AIDS and
education received only 22.8 % of the total financial resources available for education. From
UNESCO’s regular budget their share is a bit higher (29.2 %) but from the extra-budgetary
resources only 20.8 % is allocated to these priorities. Main Lines of Action (MLA) “Implementing
LIFE”, “Implementing TTISSA” and “Leading the EDUCAIDS”, which according to the
programme are the core of UNESCO’s response to EFA at the country level, receive mere 8.5
percent of the total education resources (both regular and EXBF). The budget as presented in
the 33 C/5 suggests that the actual financed priorities of UNESCO’s education programme are
somewhere else than in the “programmatic core”.



27
   For comparison, FTI Education Programme Development Fund received MUSD 52 for years
2005/2006. UNICEF’s expenditure to basic education and gender equality was approximately MUSD 450
in 2006 alone, Save the Children US expenditure on education programmes in 2006 was MUSD 49.


                                                                                                                     8
                                 Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                          Final Report

Graph 1 below shows, that while UNESCO regular budget priorities were in Universal Primary
Education, LIFE and UNLD and Policy Planning and evaluation for achieving EFA, two sub-
programmes - Quality education for learning to live together and Secondary and TVET – were
clearly the budgeted priorities. It is also obvious that this prioritisation is due to extra-budgetary
funding.

Graph 1: Sub-programme funding by sources in 33 C/5
                                                    Sub-programme funding by sources

         50

         45

         40

         35

         30
                                                                                                                                                    RB
         25                                                                                                                                         EXB
  MUSD




                                                                                                                                                    Total
         20

         15

         10

          5

          0
                Enhancing Policy Planning     Universal   LIFE and UNLD    Teacher        Quality      HIV/AIDS and   Secondary       Higher
               international and evaluation    Basic                      Education    education for    education     and TVET     Education for
               coordination for achieving     Education                               learning to live                            the knowledge
              and monitoring      EFA                                                    together                                     society
                  for EFA

                                                                   Sub-programmes

A closer analysis at the level of Main Lines of Action shows that “Promoting human rights,
peace, democratic citizenship and intercultural understanding through education” is allocated
25.9 % of the total education resources, followed by “Improving technical and vocational
education and training”, which receives 20 % of the education budget as included in the 33 C/5.
The high budgetary shares stem from hefty extra-budgetary allocations. Extra-budgetary
funding to Promoting human rights etc. alone exceeds the whole regular budget for programme
activities in UNESCO’s regular budget! The above two MLAs together with “Expanding and
renewing general secondary education” are allocated over half of all education resources and
almost 2/3 of extra-budgetary resources in the 33 C/5. Though very important, none of these
three MLAs are at the heart of EFA, as described in the Dakar Framework of Action.

Table 3: The biggest Main Lines of Action in UNESCO’s budget for 2006/2007
                                                                             In MUSD                                      Percentages

                                                               Regular
                                                                Budget                                                                        of total
                                                               program                         Total            % of                          activity
                                                                  me                         resource          regular                       resource
MLA                                                            activities        EXB             s             budget         % EXB               s
  Promoting human rights, peace, democratic
   citizenship and intercultural understanding
                            through education                       2.06          38.45           40.51                5.6         32.3            25.9
                                      Improving TVET                1.21          29.28           30.49                3.3         24.6            19.5
     Professional development of teachers and
                         education personnel                        1.01          10.06           11.06                2.7           8.4            7.1
                                  Implementing LIFE                 4.12              5.75          9.87              11.1           4.8            6.3




                                                                                                                                                            9
                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                     Final Report


     Expanding and renewing general secondary
                                     education        0.81       8.55       9.36        2.2   7.2     6
      Supporting comprehensive responses to
                 HIV/AIDS through education  0.42                7.89       8.31        1.1   6.6   5.3
(Source: UNESCO Programme and Budget 2006/2007)

               2.3           Importance and challenges of extra-budgetary funding
                             (EXBF)28
The above chapter highlights the role of extra-budgetary funding (EXBF) for UNESCO
Education Programme. The importance of EXBF was recorded already in the Medium-term
Strategy for 2002/2007 (31 C/4), which noted that the extra-budgetary contributions exceed
available programme funds by the rate of 5:1.29 With no substantial increments in the regular
budget, extra-budgetary funding is, and has been, the main source of financing of UNESCO’s
education programme activities.

There are three types of extra-budgetary funding arrangements:

           Appropriations to the Regular Budget, intended to reinforce an existing budget line,
           and to be expended before the end of the biennium. This modality is mainly used for
           relatively minor contributions directly related to Regular Programme activities, and for
           which the donor requires no particular narrative or financial reporting;

           Special accounts, created to support to UNESCO institute or large-scale programme.
           The special accounts are normally of a multi-year and multi-donor nature where one
           donor has no direct influence on the utilization of the contribution within the institute or
           programme in question; and no individual narrative or financial reporting is provided to
           the donor;

           Funds-in-trust agreements (FIT) directed towards a specific project or programme
           identified by the funding source in cooperation with UNESCO. A separate account is
           created for each activity; and detailed narrative and financial reporting is provided to the
           donor. Most funds-in-trust agreements are limited to a single donor and tied to a single
           project; but multi-donor trust funds in favour of multi-year programmes may also be
           envisaged. Funds-in-trust may be donated (intended for utilization beyond the donor’s
           territory) or self-benefiting.

In the FIT arrangement, a 13 % programme support cost (PSC) is levied whereas programme
support cost of 10 % for the special accounts was introduced only recently30. In FIT, budget
breakdowns are detailed and specific to identifiable components and requests for individual
changes in budget line allocations are administered by the Sector for External Relations and
Co-operation (ERC) and notified to donors. Administration of special accounts requires less
detailed reporting, and sectors are responsible for the handling of special account activities.
Since the introduction of the Table of Delegated Authority and Accountability (TOAA)31, ERC
has a strengthened supervisory role which involves monitoring all extra budgetary accounts –
both Funds in Trust and Special Accounts.




28
   Representatives of ERC expressed concerns that this section of the evaluation does not accurately or
relevantly present the strategies and policies for EXB funding and expressed reservations about the utility
of the part of the evaluation.
29
   UNESCO Medium-Term Strategy 2002-2007 (2002), 31 C/4, page 12.
30
   175 EX/35, Decision 36, August 2006 and Administrative Circular No. 2280
31
   Administrative Circular No.2285


                                                                                                          10
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

The importance of coherence and alignment between the regular programme and extra-
budgetary funding is obvious and it has been highlighted in several strategy documents
including clearly mentioned preference for multi-year and multi-donor arrangements. The
Medium Term Strategy 2002/07 stated that UNESCO is actively working towards consistency
between EXBF and the regular programme, and that EXBF would become an integral part of
UNESCO’s programming. The strategy also ensured that the objectives and expectations of
extra-budgetary donors will be respected. With regard to the various modalities, the strategy
discussed the possibility for defining longer-term funding strategies with the main external
funding partners and to codify them in formal cooperation agreements. In that context, multi-
donor and multi-year funding arrangements were to be explored´.32

Establishment of like-minded donor groups in (position to provide extra-budgetary) support to
core initiatives was also mentioned in UNESCO Strategy for 2005-201533. The report by the DG
on the Management of EXB resources and activities34 points out that most of UNESCO’s
funding sources continue to favour their own modalities and procedures, which increases the
administrative burden of UNESCO. The report envisages extensive consultations with the main
funding sources, which would preferably lead to more thematic and flexible funding modalities,
based on results oriented activity plans of the MLAs. This trend is reflected in the draft 34 C/5,
which reads that “innovative funding modalities, such as multi-donor, multi-year funding
arrangements with little or no earmarking in favour of large UNESCO programmes will be further
explored, as will the potential for enhanced UNESCO involvement in debt-for-development
swaps. Extensive consultations with the funding sources, both by groups and individually, will
continue to form a major component in these efforts.”35

This attention to the strategic focus and mode of operation of EXBF reflects a deeper process of
review and continuous improvement which has been underway over the last number of years.
Following two performance audit reports covering the period 2004 and 2005, carried out by the
External Auditor, a number of strengthened measures for EXBF
have been set in train to address identified weaknesses in          The auditors report on
programming/planning, resource mobilisation, implementation         biennium 2004-2005
                                            36
and monitoring of extra-budgetary activities .                      commends Cap-EFA on its
                                                                               strategic work: “Rather than
                                                                               waiting for donors to suggest
In response to these audits, the Director General announced an                 projects, this programme
Action Plan for improved management of the EXB funds37. In the                 seizes the initiative, and
plan, the main thrust is to ensure that additional financing                   develops a list of high priority
supports regular budget and UNESCO’s programmatic priorities.                  projects that are then
                                                                                                       1
The plan sets out a systematic overall strategy and guidance for               suggested to donors.”
planning,     resource      mobilisation,     project/programme
implementation, substantive and financial monitoring and reporting, evaluation, and tools for
senior management and governing bodies for monitoring EXB activities, including an early
warning system of under or overspending.38

The plan is a ‘living document’ that is updated and elaborated to ensure continuous
improvement in the management of extra budgetary activities (see for example 175/Ex 35). One
of the measures initiated as part of the Action Plan is the Table of Delegated Authority and


32
   31 C/4 (2002), paras 39 and 40.
33
   171 EX/8, Para 90, March 2005.
34
   175 EX/35, para 19.
35
   Draft 34 C/5, page 197.
36
   Report by the External Auditor on the performance audits undertaken in the 2004/2005, Draft action
plan proposed by the DG on management of extra budgetary funds. 174 EX/INF.4 Rev., Paris 30 March
2006.
37
   ibid.
38
   ibid


                                                                                                             11
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report

Accountability (TODAA)39, which was approved by the Director-General in January 2007. The
Table includes four sections with overall guiding principles and associated process steps. The
sections are:
   1. Programming of extra-budgetary activities aiming at enhancing alignment between RP
       and EXB activities, presenting, how in conjunction with draft C/5, sectoral strategic
       frameworks (SSF) are elaborated to cover both regular and EXBF activities with a
       seamless linkage between the RB and EXBF activities.
   2. Development and implementation of Strategic Plan for EXB resource mobilisation at
       various levels, while the overall responsibility rests with the ERC/CFS, which should also
       preferably be responsible for all formal presentations of programme/project documents
       to potential donors.
   3. Enhanced implementation of EXB activities through further simplification and
       harmonisation of existing procedures and a clearer distribution of labour. A major review
       is being carried out of all basic instruments relating to EXB cooperation to agree on a set
       of standard models to be applied. Echoing the External Auditors report, the Table points
       out that the current rules and procedures relating to implementation are by and large
       adequate but sometimes cumbersome and insufficiently adhered to by staff.
   4. More effective monitoring of the implementation by senior management and UNESCO’s
       governing bodies with anticipation that as integration between EXB and RB activities is
       pursued, reporting to the Governing Bodies will also be unified, preferably starting with
       the reporting on the 2008/09 biennium.

Additional measures to the TODAA are underway to further strengthen the alignment of extra
budgetary and regular programme activities. The regular programme includes diverse identified
needs appropriate to UNESCO’s remit and coherent with regular programme MLAs, but which
are not funded under the regular programme. These unfunded needs are now being cohered
together by sectors into a strategic proposal which sets out an “Additional Programme of
Targeted/Projected Extra Budgetary Activities”. In the Education Sector, a draft strategy was
prepared in respect of the unfunded education needs in 2007. The Strategy states that the
Education Sector, together with the Division of Cooperation with Extra-budgetary Funding
Sources will approach existing and potential donors to encourage them to fund activities in line
with the sector’s priority areas. This alignment to UNESCO’s priorities is considered to be a shift
from project to programme approach.40 The first consultative donor Forum was held on 28th
March 2008 at UNESCO Headquarters to present and discuss the ‘Additional Programme of
Targeted/Projected Extra-budgetary Activities’. This initial meeting of UNESCO and donor
community is expected to be followed by other joint and individual meetings with interested
funding partners.

While the TODAA does not indicate any preference or prioritisation of one funding modality over
another, the draft EXBF strategy for education sector expresses hope that the donors will
support the programme/thematic approach through pooled financing and leave the choice of
priorities to UNESCO. The MOU with the Nordic Donors and Switzerland is mentioned as an
example. According to the draft Education Sector’s Strategy (para 5.2), the advantages of the
pooled funding modality include lower support costs, simplified administrative procedures, joint
donor meetings and consolidated reporting. With the increasing emphasis over recent years
upon synergies and more focused programming, all extra budgetary activities are now expected
to be in sound alignment with the regular programme.

The draft Education sector strategy also identifies key challenges for implementation of the
strategy. These challenges include for example lack of human resource capacity in some of the
Field Offices; separation between regular programme and EXBF activities in the Field Offices,


39
     Administrative Circular No.2285
40
     UNESCO Education Sector’s Extra-budgetary Strategy for 2008-2013 (draft version November 2007)


                                                                                                  12
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

with specific staff allocated to each type of activity; and potential difficulties in combining country
needs and UNESCO’s strategic priorities (bottom-up vs. top-down planning).

            2.4         Evaluation of UNESCO’s Support to National Planning
                        for EFA

A major evaluation of UNESCO Support to National Planning for EFA was published in early
200641. The evaluation sought to obtain Member States perspectives as beneficiaries of
UNESCO support to planning for EFA. The findings and recommendations emphasise
UNESCO’s role among the donors and international agencies, its representativeness and
access to expertise covering the whole education sector. This unique role is perhaps best
captured in a quote from Burkina Faso:

           o   It’s the role of partners to accompany the government in its choice. It’s the
               role of UNESCO to help the government make a good choice. Otherwise the
               partners will push their own agenda”.42

The evaluation, however, identified major problems in fully realising that role. The evaluation
finds that the way UNESCO delivers its technical assistance does not fully support its role as a
lead technical agency and resource for EFA. The main elements of UNESCO’s assistance to
support national planning, capacity building and technical assistance (TA), have suffered from
inadequate preparations and targeting and insufficient capacity to provide long-term support,
including much-needed follow-up in supporting various aspects of management of education
programmes.

The most successful capacity building activities are associated in the evaluation with larger
extra-budgetary financed projects that have had the time and the resources to assess and
address capacity-building needs more holistically. However, UNESCO’s general approach
towards extra-budgetary funding is deemed another feature undermining UNESCO’s role as a
lead agency. The evaluation compared UNESCO to a typical implementing NGO eager for
resources, to be active in the field, piecing together and stretching resources to maximum
effect.

Another problem associated with the extra-budgetary projects is the system of allocating
consultancies through internal networks of individuals across the organisational structure.
The evaluation found that in UNESCO the EXBF projects are considered to be ways to achieve
more significant goals and also opportunities for career development, professional
development, additional staffing and sub-contracting options.

The evaluation also concluded that in many countries UNESCO has been left outside the
mainstream of the cooperation framework and policy and planning dialogue between the
Member State and the education donors. This was considered to be a consequence of slow
adaptation to new aid modalities, and weaknesses in UNESCO country level planning, which
was deemed to be too reactive with too short a time span (biannual) and limited capacity to
participate in the main UN planning exercises the Common Country Assessment (CCA) and
United National Development Assistance Framework (UNDAF).

The recommendations of the evaluation included for example alignment to country
programming cycle that is consistent with other international agencies, high-level commitment

41
   Evaluation of UNESCO Support to National Planning for EFA, Education for Change, 2006,
IOS/EVS/PI/53, UNESCO. The evaluation looked at UNESCO’s general support to national planning, not
any specific programme or budget, in 2004-2005.
42
   ibid. page 31.


                                                                                                    13
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report

towards UNDAF planning with sufficient resources and participation in formal education groups
and a more structured and disciplined approach to be adopted towards EXBF that prioritises the
fit with global, regional or national programming or strategies.


            2.5          Consolidated findings of the Evaluations of UNESCO’s
                         Major Programme I Institutes and Centres

Consolidated Findings of the Evaluations of UNESCO’s Major Programme I Institutes and
Centres (2006)43 comprises findings and recommendations of evaluations of UNESCO’s
education sector institutes and centres44.

The institutes and centres are to serve as “centres of excellence”, and providers of technical
support and expertise in their area of specialisation to the Member States, the Secretariat and
the Field Offices (FO) through various capacity building activities. Other core functions include
acting as laboratories of ideas, information clearing houses, standard setters and catalysts for
international cooperation in line with UNESCO’s overarching strategies and programmes, and
with the key functions identified in the Medium-Term Strategy 2002/2007. In order for the
institutes and centres to serve as world-class centres of research and training excellence, they
require a high degree of specialised competence and knowledge. The evaluation found only the
International Institute for Educational Planning (IIEP) to match the above criteria.

The evaluations found the programmes and activities of the institutes and centres to be in full
alignment with UNESCO’s medium term objectives, education sector priorities and EFA goals.
Especially institutes and centres that specialise in central areas, have a global focus and are
well established were found relevant. For improved relevance the evaluation recommended that
the institutes and centres need 1) a clear mandate, 2) sufficient resources, 3) global orientation.

The effectiveness was assessed against the key functions set for UNESCO and for the
institutes (laboratory of ideas, clearing house of information, standard setter, capacity builder in
MS, and catalyst for international cooperation). Overall, the evaluation found the institutes and
centres disappointing or uneven in serving as laboratories of ideas but consistent and effective
as clearing houses helped by active roles in collecting and disseminating information,
experiences and best practices. There were limited results in standard setting due to limited
mandates, but with regards to capacity building especially IIEP was assessed as having made
very positive contributions towards capacity building in the Member States. Some institutes
were criticised for unclear focus and lack of follow-up undermining the sustainability of the
provided assistance. All institutes and centres were deemed active in promotion and facilitation
of cooperation.

The level and quality of interaction and coordination with other UNESCO bodies were overall
found to be poor. This stems both from low levels of contact and awareness, and from a lack of
clear definition and division of roles and responsibilities. The evaluation is of the opinion that the
lack of proactive and integrated planning processes for UNESCO as a whole impedes effective
coordination. In addition, the evaluation concludes that due to increasing extra-budgetary

43
   Consolidated Findings of the Evaluations of UNESCO’s Major Programme I Institutes and Centres,
Nick Davis, February 2006. IOS/EVS/PI/58 UNESCO
44
   The evaluations covered the following institutes and centres: International Bureau of Education (IBE);
UNESCO International Institute for Educational Planning (IIEP); UNESCO Institute for Education (UIE);
UNESCO Institute for Information Technologies in Education (IITE); International Institute for Capacity
Building in Africa (IICBA); UNESCO Institute for Higher Education in Latin America and the Caribbean
(IESALC); UNESCO European Centre for Higher Education (CEPES); and UNESCO International Centre
for Technical and Vocational Education (UNEVOC).


                                                                                                      14
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

funding, and potential tensions between donors’ demands and priorities of the institutes and
centres, there is a risk of diversion of resources from the core activities.

The report offers seven key issues for UNESCO to address. The most prominent issue is
clarification of the roles and responsibilities of the institutes and centres, which will promote
interaction and coordination with the Secretariat and assist in striking an appropriate balance
between operational activities and other core functions. The evaluation recommends research
capacity to be strengthened as well as introduction of contract-based funding to strengthen
governance and accountability. The other key issues include strengthening of resource-based
management and taking a strategic decision on the continuation of the smaller institutes.


            2.6         A Review of UNESCO’s Capacity-Building Function
Against the background of UNESCO’s Executive Board’s request to the Director General “to
render assistance for institutional capacity building of member states in UNESCO’s domain” a
review of UNESCO’s Capacity Building Function45 was published in February 2007. The
purpose of the review was to:
    1. Inform senior management of good practices to be considered in designing capacity
       building initiatives; and
    2. Enable sectors to improve their capacity building interventions.

The review identifies UNESCO’s strengths in its ideas, innovations, international standard
setting and power to bring national and international groups together. According to the Review,
UNESCO can bring added value to capacity development programmes through its core
functions, by applying its global experience, lessons learnt, ensure international norms and
standards are respected, convene disparate stakeholders together, promote international good
practices and facilitate networking.

Referring to recent research, mainly studies by the European Centre for Development Policy
Management (ECDPM), the review points out that institutional changes are typically complex
and of long duration, and difficult to grasp with linear models such as results-based
management. The review concludes that much of UNESCO’s capacity building is characterised
by conventional training and technical assistance despite the understanding that sustainable
change in institutions requires much more than this. The Review finds that the activities are too
often narrowly focused short term technocratic responses to the needs and “run the risk of
achieving little”.

The review identified ten lessons and good practises for successful institutional capacity
building46 but found relatively few UNESCO projects to abide by these lessons and good
practices.

The review considered both normative and operational aspects in capacity building as important
but stressed that UNESCO should prioritise much more stringently its scarce resources. The
review recorded concerns of UNESCO’s own capacity to provide capacity building and
weaknesses in technical expertise and lack of experience in change management and
organisational development. Added with challenges in inter-sectoral programming, internal
communication and two-year planning cycle, the review concludes that UNESCO should
concentrate its capacity building activities in areas where it is most needed and where UNESCO
can provide added value. This includes possibility for continuing small capacity development
projects, some of an experimental nature.

45
   A Review of UNESCO’s Capacity-Building Function, Stiles Associates Inc., IOS/EVS/PI/71, UNESCO.
2007.
46
   ibid. see pages 2 and 17-22.


                                                                                                15
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



The evaluation also recommends that senior management in UNESCO should undertake a
further analysis and discussion to identify UNESCO’s options for strengthened capacity
development. The outcomes of these analyses are discussed in the next chapter.


           2.7         Internal Review of UNESCO’s Capacity Development
                       Programmes
As a continuation to the above review of UNESCO’s Capacity-building function, UNESCO
Education Sector, in cooperation with IIEP and UIS and with financial support from Norway, has
conducted    an internal review of
UNESCO’s        capacity     development     Factors contributing to good practices as identified
programmes to take stock of and draw         in the internal review:
lessons from current thinking on effective     • Country ownership and leadership;
capacity development approaches in             • High governmental commitment and strong
UNESCO and within the international                political will to implement the programme;
development community. The review is           • Appointment of a qualified national
                                                   coordinator to ensure day-to-day operations;
part of a larger exercise “Capacity
                                               • Participatory assessment and decision-
Development for Achieving the EFA
                                                   making mechanism with beneficiary groups;
Goals: UNESCO Strategy”, which aims to         • Sound understanding of, and respect for, local
produce UNESCO policy framework for                customs and constraints, along with the
capacity development.47                            physical presence of stakeholders from the
                                                        start of the process;
The review concluded that majority of           • Appropriate technology transfer; and
UNESCO’s         capacity      development      • Good teamwork and teambuilding, and
activities focuses on policy and technical        drawing on different partners’ experience in
issues (e.g. planning, data gathering and         the design and implementation of
analysis, curriculum development, teacher         programmes.
training policy) rather than operational or
process issues (such as strengthening national governments’ capacities for identifying gaps and
generate modalities to fill the gaps). The programmes are usually initiated by UNESCO, often
on the basis of needs identified through various means. Among several modalities for capacity
development and UNESCO’s role in capacity development, training has been predominant.

The internal review, echoing the independent review, concludes that the term “capacity
development” lacks consistency in UNESCO and there is no standard definition for it. UNESCO
as a whole lacks coherence in how it interprets and uses the term and the overall impression is
that the concept is used on and operation-by-operation basis and can be used to justify almost
any type of programming.

The review notes unevenness in UNESCO’s own human resource capacities to offer necessary
technical expertise for capacity development in Member States. The specific areas mentioned
include change management and organisational development and challenges relating to
planning, financing and monitoring of interventions and communication of UNESCO’s role.

The review points to the limited resources especially in the Field Offices to meet the capacity
development needs and commitments. The review indicates that better results could be
obtained if UNESCO acted as a facilitator of change management processes rather than as an
implementer of services.

As the next steps, the review recommends that UNESCO needs to:

47
  Capacity Development for Achieving EFA Goals: UNESCO Strategy, Review of UNESCO’s Capacity
Development Programmes, final review report, working document for internal use, February 2008.


                                                                                                 16
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report

     1. Clarify what it means by capacity development;
     2. Prioritise and consolidate the capacity development focus in its thematic strategies;
     3. Develop a long-term action plan for institutionalising UNESCO’s focus and approach,
        including agreement on concepts, objectives, processes, strategic partners and more
        efficient monitoring mechanism; and
     4. Communicate this approach widely throughout its offices and institutes so that UNESCO
        is perceived as one.

The review and its recommendations coincide well with the Cap-EFA review and elaboration of
the Cap-EFA programme.


            2.8          Challenges in the Field Offices
In an evaluation of a training course for UNESCO staff members in education policy analysis48,
the trainees concluded that the main bottle-necks for UNESCO to support governments in
sector policy work relate to insufficient professional capacities and the UNESCO’s internal
institutional framework which is not sufficiently geared towards country level concerns.

According to the course evaluation, the participants expressed that for UNESCO to deliver more
effectively on its mandate, there is a need to improve internal operations of the organisation.
The FO staff members pointed out that the institutional structure and ways of working between
the FOs and HQ/Institutes do not provide the kind of institutional support that is needed at
country level to function effectively. Problems are identified in internal communication, sharing
of information and staff resources. For example, most FO staff members attending the training
were unaware of existing UNESCO internal capacities, experiences, materials and instruments
related to policy and planning work. The FOs staff members believed that UNESCO as a whole
has a range of material and experiences to offer, but there is a need to pull this together, make
it known and better accessible. According to the course evaluation, this is linked also to the
perceived lack of external communication and visibility, which makes it difficult for UNESCO FO
to “market” successful experiences and products.

According to the FO staff, the greatest challenge relates to the “de-professionalisation” of the
education sector in the past 10 to 15 years.49 Technical competency in policy, planning and
financing issues related to UNESCO’s sector-wide mandate is seen as a pre-condition for FO
staff members’ technical advisory and capacity building role with government institutions, and
the entry point into dialogue and cooperation with donors at country level. According to the
course evaluation, the FO staff members feel ill-prepared for these roles.

In addition, the trainees raised concerns linked to UNESCO’s role in the changing aid
environment and how UNESCO can position itself more strategically in working with Member
States and international partners at country level.


            2.9          UNESS to ensure relevance and effectiveness of
                         UNESCO’s activities
As a response to the criticism and follow-up to the recommendations of the above mentioned
evaluation on UNESCO’s support to national planning for EFA (see 2.4), UNESCO launched
pilots for the preparation of UNESCO National Education Support Strategies (UNESS) in 2006.
48
   Training Course Evaluation based on written and oral comments by the participants as recorded by the
organisers of the training course, Professional Upgrading of UNESCO Education Sector Personnel:
Training Course on Education Policy Analysis – A joint undertaking by ED/EO, ED/SFS and IIEP, 21 Jan
to 1 Feb 2008. Participants included 28 staff members, of whom 21 were from FOs.
49
   This concern, referred to as erosion of intellectual capital, was raised also in interviews in the HQ


                                                                                                     17
                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                     Final Report

UNESS is a strategic programming exercise, which aims to ensure relevance and effectiveness
of UNESCO’s actions to Member States’ needs and demands. In 2006, first UNESS exercises
were carried out and documents prepared in the 11 Cap-EFA programme countries. The
preparation of UNESS documents has been largely financed by Cap-EFA. Now (March 2008)
there are 33 draft UNESS documents available in the UNESS web-site and the plan is to extend
UNESS to all Member States requiring cooperation with and/or support from UNESCO in the
field of education. Currently the documents are being prepared in some 60 countries.50

To support preparation of the documents, the Section for Education Support Strategies has
prepared Guidance Note “Building a UNESCO National Education Support Strategy (UNESS)
Document: 2008-201351. The Guidance Note provides advice in preparing the document and
includes a proposed structure and content for a UNESS document.

UNESS has been vested with multiple objectives and expectations. A model UNESS document
is intended to be a well-defined, evidence-based analytical description of needs in education
sector followed by a strategy that 1) responds to country’s needs, 2) takes into consideration the
mandate and comparative advantage of UNESCO, and 3) improves cooperation and synergies
with other development partners. At the same time, it is envisaged to be a demonstration of
UNESCO’s more active and constructive participation in joint-UN activities at country level and
input to the educational components of Common Country Assessments (CCA) and United
Nations Development Assistance Framework (UNDAF). Hence, UNESS is set to promote one-
UN and donor harmonisation agendas as well. UNESS is also intended to be useful in furthering
the Global Action Plan discussions at country level with the EFA convening agencies.

To improve the effectiveness and impact of UNESCO’s cooperation, UNESS is planned to help
UNESCO to select only a few critical areas for support and ensure commitment to these areas
over several years, hopefully attracting extra-budgetary resources.

Finally, UNESS is also hoped to serve as a tool for evidence-based UNESCO programmes to
inform strategy, programming and budgeting work.

               2.10          Summary of identified challenges and responses
The table below presents a summary of the main challenges identified above with the identified
UNESCO management responses (shaded).

Table 4: Summary of identified challenges and responses by sources
                                                                          Eva. of support to




                                                                                                                                                                                                         Capacity Building
                                                  External Auditors




                                                                                               Eval. of Institutes


                                                                                                                     External Review




                                                                                                                                                                           Action Plan for
                                                                                                                                                         Training course
                                                                                                                                       Internal Review




                                                                                                                                                                                                         Policy work for
                                                                          nat. planning


                                                                                               and Centres




                                                                                                                                                                                EXBF
                                                                                                                                                         for FO Staff




                                                                                                                                                                                             UNESS
                                     33 C/5




                                                                                                                     of CB


                                                                                                                                       of CB




     Limited financial resources              X                                    X                     X                   X                X                                    X                 X
     Regular Programme
     alignment with extra-                    X                       X            X                     X                                                                         X                 X
     budgetary funding
     Prioritisation/ unclear focus            X                       X            X                     X                   X                X                 X                  X                 X            X
     Professional capacities/
                                                                                   X                     X                   X                X                 X                                                 X
     technical expertise
     Planning to match needs,
     normative role and available                                     X            X                     X                                                                         X                 X
     resources

50
   http://portal.unesco.org/education/en/ev.php-
URL_ID=10200&URL_DO=DO_TOPIC&URL_SECTION=201.html
51
   http://unesdoc.unesco.org/images/0014/001485/148566E.pdf


                                                                                                                                                                                                                             18
                             Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                      Final Report

      Internal cooperation and
                                                      X       X      X       X      X          X
      coordination
      Lack of standard definition
                                                                     X       X                 X
      of key concepts




3               Capacity Building for Education for All – programme
                description
                3.1           Memorandum of Understanding (MOU)
In October 2003, a Memorandum of Understanding was signed between UNESCO and the five
Nordic Countries (NC) (Denmark, Finland, Iceland, Norway and Sweden), in which the NCs
promised to increase emphasis on education in their development assistance and provide funds
for UNESCO for capacity building in Member States through a special multi-donor account. The
support is based on UNESCO’s Medium-term Strategy 2002-2007, Programme and Budget and
International Strategy to Operationalise the Dakar Framework for Action on Education for All
and funds are provided for capacity building as described in UNESCO Policy Statement on
Cooperation with Donors for the Dakar Follow-up Action and Capacity Building for EFA –
document.

The MOU is open for all donors to join, but an individual agreement is signed with all requesting
donors. Furthermore, the MOU allows each donor to request their own priorities to be reflected
in individual agreements, with an understanding that these priorities will be respected.

With regard to the management of the programme, the MOU stipulates that UNESCO will issue
a tender notice for demand-driven activities in Member States, screen proposals, ensure that
programmes and activities are in line with national strategies for poverty reduction and
coordinated with other donors active in the country. Priority will be given for Least Developed
Countries (LDCs).

To monitor the use of funds, there will be annual joint consultations and UNESCO will submit
annual statements of accounts and consolidated annual report on the progress in relation to the
objectives. The MOU provides for independent evaluations of the programme by UNESCO
and/or by the NC.

In addition, the MOU allows separate funding mechanism to be used for EFA Global Monitoring
Report and UNESCO Education Institutes.

The MOU is a flexible modality, with references to objectives of the programme and to the
relevant strategic documents and programme guidelines. The donor contributions are, however,
only included in the bilateral agreements leaving the scope of the programme unclear. The
consultation mechanism and required reporting are described at a fairly general level.

                3.2           Capacity Building for EFA (2005)
Operational programme guidelines are outlined in Capacity Building for EFA –document, which
is revised according to UNESCO’s biannual planning and implementation cycles. The current
Capacity Building for EFA-document is dated November 2005.52 It is an operational plan that
seeks to strengthen and complement technical support and capacity building efforts that need
additional resources in a restricted number of countries and to focus more on the core of
UNESCO’s programme as defined in the 33 C/5 - literacy (under LIFE) and teacher education in

52
     Cap-EFA Programme Document for 2008/2009 is in draft form


                                                                                                   19
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

Sub-Saharan Africa (under TTISSA). The aim was to provide larger funding over two years for a
limited number of countries.

The objective of the programme is:
  To enhance member states institutional capacity in reviewing and readjusting their national
 education plans as well as managing and monitoring large scale literacy and teacher training
                                       programmes.


           3.2.1       Programme Components, Objectives, Implementation Strategies
                       and Expected Results
Deriving from the objective, the programme is divided into three components:
   a) Technical assistance and institutional capacity building in sector-wide educational
       planning;
   b) Literacy; and
   c) Teacher training in Sub-Saharan Africa.

The objective of technical assistance and institutional capacity building in sector-wide
educational planning component is to enhance institutional capacities to review and reform
national education plans. The programme document (2005) envisages 1) technical assistance
to national authorities to support planning and 2) establishment/support to interagency
cooperation as the two main operational strategies.

In the literacy component, Cap-EFA aims to enhance national capacities to develop,
implement, monitor and evaluate literacy policies, and to mobilise and coordinate partners to
ensure plans are brought to scale. The literacy policies should pay particular attention to
improved accessibility, quality, relevance and sustainability of literacy learning and skills
training, improvement of literacy environments and empowerment of learners, especially girls
and women. The activities should fall along the following three strategic lines:

   1. Integration of literacy into sector-wide education policy frameworks and other major
      development programmes;
   2. Strengthening national capacity for programme delivery, including training of managers
      and literacy instructors, gender-sensitive and learner-centred programme design, and
      enabling literacy environments:
   3. Assisting countries in designing monitoring and evaluation tools to collect reference data
      and assess the effectiveness of literacy programmes.

The expected results include:
   1. Literacy policies developed and/or strengthened;
   2. Increased access to literacy learning especially for girls;
   3. Improved quality and relevance of programmes;
   4. Increased sustainability of achievements; and
   5. Improved monitoring and evaluation.

In the teacher training component the objective is to assist member states in Sub-Saharan
Africa in improving their teacher training policies and practices to educate more and better-
qualified teachers. This is to be implemented through three strategic lines:
    1. Assisting Member States in developing and reforming holistic teacher policies and
        innovating teacher education curricula
    2. Paying special attention to effective use of information and communication technology
        (ICT) both as a teacher training tool to increase number and quality of trained teachers,
        and as a pedagogical tool to enhance student learning;




                                                                                              20
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report

     3. Promoting strategic social and political dialogues and partnership–building among
        stakeholders, including ministries, universities, teacher training institutions and teacher’s
        organisations.

The expected results are:
   1. National policies developed or reformed through social dialogues to improve the
      performance, working conditions and status of teachers, considering the overall national
      education policies and poverty reduction strategies.
   2. Development or consolidation of open and distance-learning teacher education
      programmes with increased and improved use of ICTs.
   3. Increased, effective use of ICTs in classroom among teachers to facilitate students’
      active learning and knowledge-building.
   4. Strengthened capacity of teacher-training institutions in using research results and co-
      operation networks to educate more and better-qualified teachers and educational
      leaders.

The defined objectives and expected results do not lend easily for meaningful monitoring or
evaluation. For each expected result one could say yes, is has been addressed and probably
reached to some extent, but in the absence of measurable targets and identified or sources of
verifiable information, the programme objectives and expected outputs do not allow for proper
programme level monitoring and assessment of effectiveness.

Despite clear similarities, the Cap-EFA programme document (2005) leaves the obvious
linkages between 1) LIFE and TTISSA and 2) UNESCO Medium-term strategy (31 C/4) and
Programme and Budget (33 C/5) unspecified. Concerning LIFE and TTISSA this is probably
due to unavailability of the respective strategies and objectives during the time the programme
document was prepared. The programme document (2005) describes challenges in literacy and
teacher training, but at such a global level that the response to the challenges with the limited
resources available from the Cap-EFA programme becomes rather redundant. This leads to
unclear programme intervention logic with missing hierarchy of objectives and quantified
indicators to measure programme success.

Another ambiguity is identified in the operationalisation of UNESCO’s normative role and key
functions and added value and comparative advantage. In 33 C/5, literacy and teacher training
are identified as neglected priorities in global EFA efforts and deemed as requiring sustained
and coordinated action if EFA goals are to be achieved. According to 33 C/5, literacy and
teacher training are also areas where UNESCO has 1) comparative advantage and 2) expertise
to offer53. This comparative advantage and the availability of expertise and resources are,
however, not clearly specified or substantiated neither in the Cap-EFA programme document
(2005) or in LIFE Vision and Strategy Paper or in TTISSA Concept Note54.

             3.2.2       Management of the Programme
The programme management is trusted with the executive office at UNESCO HQ. An invitation
to bid for extra-budgetary funds under the Cap-EFA programme, with relevant documents, was
sent to 26 LIFE and TTISSA first phase countries in November 200555. The invitation letter and
the attached guidance note stipulated that the programme is exclusively dedicated to up-stream
capacity building activities in approximately 10 countries.


53
   33 C/5 pages 20-22.
54
   In draft 34 C/5, the areas of UNESCO’s comparative advantage are technical and vocational education
and training and higher education.
55
   Angola, Burkina Faso, Burundi, Cap Verde, Central African Republic, Congo (DR), Congo, Ethiopia,
Ghana, Guinea, Madagascar, Mali, Niger, Nigeria, Senegal, Sierra Leon, Tanzania, Zambia, Haiti,
Bangladesh, Indonesia, Pakistan, Vietnam, Egypt, Morocco and Yemen.


                                                                                                   21
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

Stressing that only one project proposal will be accepted for each target country, the guidance
note advised the responsible field office to designate a Lead Project Officer (LPO) to lead and
coordinate project elaboration and implementation. The LPOs were encouraged to prepare the
proposals in close cooperation with relevant HQ divisions, institutes, centres and regional
bureaux, in line with their expertise and capacities. The importance of this UNESCO’s internal
cooperation, the family-approach, was highlighted in the invitation letter, which states that:
“Proposals prepared without involvements of relevant regional bureaux, institutes and centres
and HQs divisions, will not be considered. “

In accordance with the implementation strategy outlined in the programme document (2005),
the guidance note and project proposal format emphasised alignment with UNESCO’s regular
programme, national education priorities (PRSP) and development frameworks (FTI,
CCA/UNDAF). The proposals were to mainstream gender and HIV/AIDS issues and have clear
exit strategies for ensuring sustainability of results. Co-funding and collaborative actions with
other EFA partners were encouraged

The project proposals were assessed         UNESCO family approach is an internal
by an Assessment Committee against          implementation mechanism, which aims to streamline
12 key criteria (Annex 6) reflecting the    and coordinate interventions involving various
key issues highlighted in the               UNESCO entities. Field offices are primarily
                                            responsible for the implementation of all operations and
guidelines. The Committee included
                                            request services from Institutes, regional bureaux and
representatives of four regional            head quarters according to the needs of the Member
bureaux, the IIEP deputy director and       State. This approach aims to put a large pool of
three directors from HQ divisions.          UNESCO competencies at the disposal of Member
Though these same instances were            States and it is hoped to make the organization more
required to be included in the              effective at country level, and to ensure quality delivery.
preparation        and         possible
implementation of the projects, it was ensured that people who were proposed to have a role in
the implementation did not participate in the evaluation.

For planning purposes, a standardised project proposal was prepared, which following the
logical framework approach required the field offices to define a hierarchy of objectives with
goals, immediate objectives, expected results and activities with budgets and timelines. In
addition, verifiable indicators, sources of verification and risks and assumptions were to be
included in the logframe matrix. In addition to budgeting by activities, the budget is also
presented by expenditure components divided into personnel, sub-contracts, training,
equipment, and miscellaneous.

The proposals, once approved, are further operationalised in a detailed work plan, which
includes plans for personnel and for procurement of equipment, budget breakdown by activity
and by expenditure component, overview of budget revisions and timeline for implementing
activities.

On reporting and monitoring, three progress reports and a final report are required for the
project implementation period. The progress reporting formats guide the FOs to take account of
the activities against expected results utilising the logframe matrix used for planning. In addition,
there is a narrative part to the report requesting analytical and concise answers to:

   1) Achievement of objectives;
   2) Challenges and lessons with regard to the development of the sub-sector;
   3) UNESCO’s comparative advantage, impact on national development processes, role of
      UNESCO-family approach and operationalisation of UNESCO’s normative role;
   4) Cooperation with other UN agencies and other development partners, sustainability,
      scaling-up, mainstreaming pilot experiences and the main enabling and impeding factors
      for the same; and


                                                                                                    22
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

   5) Major conclusions and consequences of the mid-term evaluation (3rd progress report
      only).

While most of the above issues follow logically from the project proposal, the third set of
questions related to comparative advantage, family approach and operationalisation of
UNESCO’s normative role was not included in the project proposal format or in the assessment
criteria. The inclusion of these issues into reporting was requested by donors, and it aims at
encouraging the FOs to focus and take more analytical approach to their assessment of the
activities.

For the annual consultations with the Nordic donors, the Executive Office summarises the
country reports into a consolidated report.

The reports from the FOs do not, and are not requested to include financial information.
Financial monitoring is carried out by utilising UNESCO regular financial monitoring procedures.
The field offices and the approved UNESCO implementing partners are allocated with
resources on the basis of the approved work plan, which details expenditure by the authorised
unit (FO, RB, institutes, centres), and by expenditure components (personnel, international
consultants, administrative support, training, equipment and miscellaneous). The authorised
units can change the allocations between the expenditure components in close
consultation/coordination with the involved UNESCO entities, and with the approval of the HQ
executive office.

The system enables monitoring by both expenditure unit and expenditure component
dimensions and the Executive Office is able to monitor the overall financial performance of the
programme and proactively intervene in the programmes that are in a risk of not being able to
utilise all the allocated funds. The financial monitoring is conducted on a monthly basis with
comparisons of total allocated funds, accrued expenditure, commitments and the balance, on
the basis of which the progress is rated very unsatisfactory, unsatisfactory, satisfactory and very
satisfactory. In case of persisting very unsatisfactory progress, a project is subject to potential
reallocation of funds, in close consultation with the FO concerned.

In addition to the above management tools, the Executive Office sent a Guidance Note to all
field offices concerning mid-term evaluations (Annex 7), which were carried out in all Cap-EFA
country programmes in November 2007. The Guidance Note gave the Lead Project Officers in
the FOs responsibility for coordinating the mid-term evaluations and ensuring that the common
issues are sufficiently addressed in all evaluations.

The guidance note requested the LPO’s to ensure that the mid-term evaluations focus on the
actual outcomes and reached benchmarks and effect on national policy and programme
development (effectiveness). The evaluations were to judge if the results justify costs and to
assess the usefulness of the project document and work plans as management tools
(efficiency) and their relevance to actual needs (relevance). Furthermore the evaluations were
to assess cooperation with development partners, especially with other UN agencies,
collaboration with national stakeholders, scaling up and sustainability beyond project
completion, Finally, the evaluations were also requested to assess the use of UNESCO’s
comparative advantage with specific reference to UNESCO’s normative role and five key
functions, the inter-sectoral approach, the family approach and UNESS. The above issues were
included in the guidance note also as a checklist to summarise the evaluation findings.




                                                                                                23
                                Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                         Final Report


4               Assessment of Cap-EFA Implementation 2006/2007
Following the 2005 review and revision of the Capacity Building for EFA–document, activities to
meet the above objectives are now carried out in eleven LIFE or TTISSA countries56, which
were selected through an internal bidding process, as described above, from amongst 23
applicants.

Table 5: Eligible and selected countries
     Eligible countries                         LIFE (L)        LDC          UNESCO          Selected
                                                TTISSA                        Office
                                                   (T)
     Angola                                         T             X                               X
     Burkina Faso                                   T             X
     Burundi                                        T             X
     Cap Verde                                      T             X
     Central African Republic                       T             X
     Congo DR                                       T             X              X
     Congo                                          T
     Ethiopia                                       T             X              X
     Ghana                                          T                            X
     Guinea                                         T             X                               X
     Madagascar                                     T             X
     Mali                                           L             X              X
     Niger                                          L             X                               X
     Nigeria                                        L                            X
     Senegal                                        L             X              X                X
     Sierra Leone                                   T             X                               X
     Tanzania                                       T             X              X                X
     Zambia                                         T             X              X
     Haiti                                          L             X              X
     Bangladesh                                     L             X              X                X
     Indonesia                                                                   X
     Pakistan                                      L                             X                X
     Vietnam                                                                     X                X
     Egypt                                         L                             X                X
     Morocco                                       L                             X                X
     Yemen                                         L              X

The MOU stated prioritisation of the least developed countries (LDCs) seems somewhat
ambiguous in the selection of the Cap-EFA programme countries. Out of 18 eligible LDCs
seven (39%) were selected whereas half (four) of the eligible non-LDCs were selected. The
selected countries include four LDCs (Angola, Guinea, Niger and Sierra Leone) where there is
no UNESCO field office, while five LDCs with FO were not selected. In addition to LIFE and
TTISSA countries, invitations to bid were sent to Indonesia and Vietnam, and Vietnam was also
selected as Cap-EFA country. In the interviews it was emphasised that the aim of the selection
process was to ensure quality of the programme activities and to get best value for money.

According to the latest revised budget breakdown (February 1, 2008), the total Cap-EFA budget
amounts to MUSD 11.8, which is divided between literacy activities in seven countries, with total
allocation of MUSD 6.5 (55 % of total funds), teacher training activities in four countries with the
budget of MUSD 3.6 (30 %) and regional and institutional support, UNESS and carry over
projects with MUSD 1.7 (15 %). The country budgets vary between USD 700 000 for Sierra
Leone and USD 1 000 000 for Vietnam, but in light of UNESCO’s contexts and typical regular
programme project volume, the amounts are relatively big, as emphasised in several interviews
57
   . However, compared to e.g. sector support or budget support modality that are being
56
  The same countries are selected to pilot the above mentioned UNESCO National Education Support
Strategy (UNESS) that was launched also in 2006.
57
  For comparison in the draft 34 C/5, average regular budget per activity in Education Sector is USD 82
000.


                                                                                                        24
                                           Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                                    Final Report

increasingly prioritised by multi- and bilateral donors, the donors commit them-selves to support
education in the Member States with considerably larger annual contributions for longer period
of time.

Table 6: Selected Countries, names of the projects and budgets
                                                                                                             BUDGET
COUNTRY                                  NAME OF THE PROJECT                                                 (USD)        %
    Bangladesh                           Strengthening    capacity   of     GO/NGO     managers     and
    (L)                                  policymakers for planning, monitoring and evaluation                953 575
                                         Capacity Building to enhance policy develop, institutional
                        Egypt (L)        change, and programme delivery in literacy and NFE                  869 400
                                         Literacy for empowerment of women, building capacity for
                        Morocco (L)      sustainable development                                             995 351
                                         Appui à l’élaboration d’une politique nationale de l’ENF et
     Literacy




                                         renforcement des capacités institutionnelles de formulation et
                                         mise en œuvre de la politique nationale de l’alphabétisation et
                        Niger            de l’ENF au Niger.                                                  739 463
                                         Capacity-building for improving quality and scope of literacy
                        Pakistan (L)     programmes                                                          980,000
                                         Renforcer les capacités des personnels d’alphabétisation et
                        Senegal (L)      d’éducation non formelle qualifiante                                950,400
                        Vietnam (L)      Strategic development of LIFE resource support system               1 012 150
                        Total literacy                                                                       6 500 399    55
                                         Reconstruction of the education system: Improving the quality of
                        Angola (T)       teacher education                                                950 504
                                         Projet de Renforcement des Compétences des Enseignants du
                        Guinea (T)       Primaire                                                         895 000
     Teacher Training




    Sierra Leone Support to strengthening the capacity of primary and basic
    (T)             education teacher training institutions                                                  703,800
                    Capacity-building of teachers and NFE educators in the
    Tanzania (T) development of Literacy policy and practice                                                 944,050
    Total teacher training                                                                                   3 584 544    30
    Regional and Institutional support                                                                       364 879
    UNESS                                                                                                    750 000
     Other




    Carry-over projects                                                                                      600 876
    Total other initiatives                                                                                  1 715 755    15
GRAND TOTAL                                                                                                  11 800 648
Source: Revised Cap-EFA budget breakdown, 1 February 2008.

According to the situation as of December 2007, the accrued programme expenditure in the
eleven programme countries was MUSD 4.3 or 42.2 % of the total budget available for
countries. Added with committed funds of MUSD 1.9 (18.4%), the total assigned funds amount
to MUSD 6.2 representing 60.5 % of the budgeted funds for the eleven countries. Given the
remaining time available for programme implementation it is likely that the programme will
exceed the average UNESCO expenditure rate for extra-budgetary funds which stood at 67 %
in 200758. There are, however, considerable differences between the countries. Whilst Tanzania
has assigned 76 % of the available funds, only 26 % have been assigned in Niger.
Consequently, the budget for Niger has been reduced from December 2007.




58
     Details on operational Extrabudgetary projects as at 25/02/08


                                                                                                                          25
                          Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                   Final Report

Table 7: Expenditure, committed and total assigned funds in CapEFA as of 3 December 2007
                                                                                 Total
                  Planned    Expenditure         %      Committed       %      assigned      %
Tanzania             944,050     607,605        64.4       109,814     11.6      717,419    76.0
Bangladesh           953,575     347,227        36.4       341,815     35.8      689,042    72.3
Guinea               895,000     469,372        52.4       166,792     18.6      636,164    71.1
Pakistan             980,000     463,834        47.3       217,832     22.2      681,666    69.6
Vietnam            1,012,150     506,225        50.0       180,744     17.9      686,969    67.9
Sierra Leone         795,000     425,777        53.6        95,131     12.0      520,908    65.5
Senegal              950,400     432,522        45.5       135,523     14.3      568,045    59.8
Egypt                869,400     229,103        26.4       256,627     29.5      485,730    55.9
Angola               950,504     353,441        37.2       163,303     17.2      516,744    54.4
Morocco              995,351     347,766        34.9       131,282     13.2      479,048    48.1
Niger                943,026     156,718        16.6        89,447      9.5      246,165    26.1
TOTAL             10,288,456   4,339,590        42.2     1,888,310     18.4    6,227,900    60.5
Source: Monitoring Cap-EFA Implementation, status as of 3 December 2007.

The changes in budgets have occurred in all countries (see Annex 8 for more detailed
calculations). As compared to the first programme budget breakdown in 2006, the total funds
have increased by USD 555 000 in 2008. The biggest increments were allocated for the three
Asian countries and Angola, which all received USD 90 000 to 129 500 of additional budget.
Morocco, Senegal and Tanzania received 70 000 to 80 000 addition, Guinea 55 000 and Sierra
Leone 35 000. Compared to the funds allocated in 2006, Egypt’s budget has been cut by USD
10 000 and Niger’s by USD 120 00059.

There have been some changes in how the budget is allocated between various expenditure
components. Fee contracts represent over half of the total budget and the budget for these
expenditures has further increased during the programme implementation period. Another
significant increase is found in the budget allocated for missions and national consultants, which
has increased in all countries. The major decrease is in budget for training, most notably in
Senegal and Sierra Leone, where almost the whole budget for training was transferred to fee
contracts.

Table 8: Expenditure by components in 2006 and 2008
             Missions
                and
              national International    Admin.     Fee
            consultants consultants     support contracts Training Equipment Misc.     TOTAL
     2006      616,877    1,169,385     201,286 5,105,510 1,747,400  425,290 203,482 9,469,230
       %              7          12           2        54        18        4       2
     2008    1,085,008    1,084,085     319,624 5,921,867 1,108,795  381,611 183,904 10,084,893
       %             11          11           3        59        11        4       2
Source: Cap-EFA budget breakdown in 2006 and 2008

An estimation of the programme management costs accrued by hiring staff to the HQ and FOs
(contract duration at least six months) amounts to MUSD 1.77 representing 15 percent of the
programme budget (as in 2008). This has provided the total of 683 person-months of human
resources for the programme management and implementation in the HQ and especially in the
FOs. The biggest management costs are in the countries where there are no UNESCO FO -
Guinea, Sierra Leone, Niger and Angola. In terms of person months, the FOs in Bangladesh,


59
  If additional financing from 2007 is included, the cuts amount to USD 105 600 for Egypt and USD 203
563 for Niger.


                                                                                                    26
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report

Vietnam, Senegal, Sierra Leone and Tanzania have received the most substantial surge in their
human resources through the Cap-EFA programme.

Table 9: Management cost and additional human resource in Cap-EFA
                                                 Total months60     Costs

                            HQ                          24               240 000
                            Pakistan                    35                72 200
                            Bangladesh                  82                99 800
                            Vietnam                     78                67 000
                            Egypt                       48                92 200
                            Morocco                     24               123 120
                            Senegal                     64               115 800
                            Niger                       80               164 600
                            TOTAL LIFE                 411               734 720
                            Guinea                      62               216 000
                            Sierra Leone                72               278 000
                            Tanzania                    72               102 768
                            Angola                      42               202 171
                            TOTAL TTISSA               248               798 939
                            GRAND TOTAL                683             1 773 659
                                                   Source: Calculations by ED/EO

There appears to be very little correlation between management cost and expenditure rates, but
rough comparisons between expenditure rates and prevalent expenditure components would
suggest that in the countries with low progress in expenditure the share of budget for training is
relatively high (16 % in Angola, 26 % in Egypt and 45 % in Niger). Whereas in countries with
highest rate of assigned budget, the share of fee contracts is higher than the average share of
59 % (Tanzania 73 %, Guinea 63 %, Vietnam 79 %, Bangladesh 79 % and Pakistan 62 %). The
robustness of this analysis is, however, questionable.

As the 33 C/5 document does not include all extra-budgetary financing and its implementation
period does not coincide fully with that of Cap-EFA (Cap-EFA expands to 34 C/5 by a couple of
months), direct comparisons are not possible. However, if compared to the 33 C/5 allocations to
literacy and teacher training, the financial contribution of Cap-EFA is quite substantial as shown
in the table 10 below.

Table 10: Total 33 C/5 and Cap-EFA allocations to Literacy and Teacher Training
                           33 C/5 (MUSD)        Cap-EFA (MUSD)           Cap-EFA in %
  Literacy                      12.5                    6.5                   52 %
  Teacher training              13.5                    3.6                   27 %
Source: 33 C/5 and Cap-EFA budget breakdown 2008.

The support from Cap-EFA to literacy equals to over half of total funds for UNESCO’s
operations in literacy in the C5 document and about ¼ of funds for activities under teacher
training. On the basis of interviews in the HQ and in the field, the importance of Cap-EFA to
LIFE, TTISSA and the available resources in FO’s has been remarkable.




60
   This is an estimation based on information from ED/EO. Some months are part-time, some co-
financed.


                                                                                                27
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

                             • Cap-EFA has provided LIFE with a lifeline and demonstrated
                               that LIFE is more than a discourse.
                                                                                       -UIL

In Vietnam Cap-EFA is the largest EXB funded project and out of seven staff members with
project contract, four are funded directly from Cap-EFA. In Tanzania, extra-budgetary funds,
Cap-EFA included, double the regular budget.

            4.1         Cap-EFA in Literacy
Cap-EFA financed literacy projects are implemented in six LIFE countries and in Vietnam.
According to the management documentation (proposals, progress reports and mid-term
evaluations), all projects comprise several components, some of which are common in all
countries. An attempt to categorise the various components is presented in the Table 11 below.

Table 11: Components of Literacy programmes in Cap-EFA
                                Bangla-       Egy       Morocc       Nig      Pakista    Sene   Vietn
                                 desh          pt         o          er         n         gal    am
     Govt/NGO cooperation         X            X          X                     X
     Training of managers         X            X          X           X         X         X      X
     Curriculum
                                                                                 X        X
     development
     Materials development                     X                                 X        X      X
     Innovative approaches                                 X                              X
     Emphasis on women             X           X           X                     X        X
     CLCs                          X                       X                     X               X
     ICT                           X                                             X               X
     M&E                           X           X           X          X          X        X      X

All literacy projects train literacy managers and support monitoring and evaluation for literacy
and non-formal education (NFE). Government – NGO cooperation is emphasised in
Bangladesh, Egypt, Morocco and Pakistan, whereas curriculum development is explicitly aimed
at projects in Pakistan and Senegal. Development of materials (usually on the basis of a review
of existing material) is part of the programme in Egypt, Pakistan, Senegal and Vietnam.
Innovative approaches can be found more clearly in Morocco (IGA) and Senegal (agro-
industry). Women and girls are especially targeted in all countries except Niger and Vietnam.
Community Learning Centres are established and supported in Bangladesh, Morocco, Pakistan
and Vietnam. Use of ICT is most clearly emphasised in Bangladesh, Pakistan and Vietnam.

In assessing coherence of the activities to the programme document (2005), the activities are
clearly aligned with two of the three Cap-EFA literacy implementation strategies: 1)
strengthening national capacity for programme delivery including training of managers and
literacy instructors, gender-sensitive and learner-centred programme design, and 2) enabling
literacy environments and assisting countries in designing monitoring and evaluation tools.
Support to the third strategy - Integration of literacy to different development programmes,
SWAPs and PRSPs - is less evident in the implementation, though there are examples such as
utilisation of project experiences in the National Human Development Fund for Poverty
Reduction in Morocco and in preparation of National Education Strategy 2008-2020 in Vietnam.

The activities are also in line with the literacy objectives and activities mentioned in the 33 C/5,
especially concerning down-stream literacy provisions at local level, such as Community
Learning Centres (CLC) in e.g. Vietnam, support to local actions on education for rural people
and the use of ICTs for reaching the unreached in literacy and basic education programmes. In
addition, 33 C/5 mentions research on and promotion of literacy assessment programmes,
including LAMP (Literacy Assessment and Monitoring Programme) and NFE-MIS (Non-formal
education management information system), which have been prominent in Cap-EFA financed
activities.



                                                                                                        28
                          Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                   Final Report



The LIFE programme objectives and activities were revised in 2007. The Cap-EFA activities
match well with the immediate objective to strengthen national capacity for programme design,
management and implementation as well as support to innovative approaches.

               4.2         Cap-EFA in Teacher Training
Teachers and teachers’ training have been addressed in four TTISSA countries. On the basis of
available background documentation, the various components in the projects are summarised in
the Table 12 below.

Table 12: Components of Cap-EFA projects to support teacher training
                                         Angola            Guinea             Sierra        Tanzania
                                                                              Leone
     Teacher training policy
                                            X                 X                 X              Z
     development
     Capacity of TTIs                       X                 X                 X              X
     Institutional structures               X                 X                 X
     Materials development                  X                                   X              X
     Curriculum development                 X                 X                 X              Z
     Use of ICT                             X                                                  Z
     Cross-cutting issues                   X                 Z                 X              Z
     Training teachers                      X                 X                 X              X
X for major components, Z for smaller components

In the Cap-EFA programme document (2005), the objective of improved teacher training
policies is to be achieved through three strategic lines. The first strategic line is to assist
Member States in developing and reforming holistic teacher policies and innovating teacher
education curricula. Teacher policies are addressed in all countries with an emphasis on
teacher training and all projects have worked with relevant Teacher Training Institutes (TTI) and
assisted in developing or revising curricula.

The second strategy in Cap-EFA calls for special attention to effective use of Information and
Communication Technology (ICT) both as a teacher training tool to increase number and quality
of trained teachers, and as a pedagogical tool to enhance student learning. This strategy is
being implemented mainly in Angola and it is in the process of starting now in Tanzania.

The third strategy, promotion of strategic social and political dialogues and partnership–
building among stakeholders, is less evident in the programme activities.

Assessment of coherence and relevance of Cap-EFA to TTISSA objectives is restrained by
revisions in TTISSA strategic framework and objectives after the Cap-EFA programme and the
teacher training activities were planned and prepared61. However, if compared to the TTISSA
logframe from December 2006 (see Annex 5) the Cap-EFA activities have concentrated on
promoting the planned output regarding development of appropriate teacher policies and to
some extent quality and coherence of teachers’ professional development. The revised TTISSA
framework offers a number of activities and instruments that are closely linked to UNESCO’s
key functions (such as advocacy of the ILO/UNESCO recommendations on the status of
teachers, development of teacher indicators, standardisation framework for TTIs, guidelines for
transparent staff appraisal) that have received less attention in Cap-EFA.

The certain inconsistency or ambiguity between Cap-EFA and TTISSA objectives is
compounded by the objective setting in 33 C/5, which presents objectives, strategies and
activities that are largely the same as in the Cap-EFA programme document (2005) and

61
     Interviews confirmed Cap-EFA alignment to the original TTISSA framework.


                                                                                                       29
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report

TTISSA strategic framework, but there is no clear logic or hierarchy in the objectives and
subsequent implementation strategies and activities.

              4.3         UNESS
There are currently 33 draft UNESS documents available in the UNESS web-site, which have
been largely financed by the Cap-EFA programme. Support to the preparation of UNESS
documents is not directly a provision of technical support or capacity building for Member States
but rather UNESCO’s internal work. The Guidance Note for UNESS explains that the resources
required for its design and adjustment will come from within the regular programme funds
allocated to FOs for programme support as of 2008. However, since UNESS was launched
during the implementation of the previous programme and budget (33 C/5) for feeding the
current programme and budget (34 C/5) planning work, an important initial investment was
required to catalyse the kick-off period62.

In Vietnam the finalisation of UNESS for 2006-2010 is still pending due to the one-UN
approach. The work on UNESS came to a halt as the other UN agencies expressed in
connection with the preparation of the One-Plan-Two reluctance to adopt UNESS as a UN
Education Strategy but were supportive to the development of UNESS as an internal UNESCO
tool to join the One-Plan. UNESCO has prepared a UNESCO Country Programming Document
(UCPD), presently governing UNESCO operations in Vietnam.

In Tanzania, the UNESS is still very much a draft document and it is intention of the UNESCO
field office to review, update and finalize it. At this stage the draft UNESS has identified five
main components for support and partnership. It pays attention to the government –
development partner agreement (Joint Assistance Strategy for Tanzania – JAST) for
coordination and partnership. One of the five UNESS components is focusing on ‘Improving
quality of education’, with specific interventions on teaching and learning in primary and
secondary and adult/non-formal education, and establishes linkages between TTISSA, LIFE
and UNLD. It has the objective of developing capacity of tutors, teachers and NFE educators in
the development of literacy competencies to promote quality learning.

              4.4         Relevance to national priorities
In Vietnam, the Cap-EFA programme was formulated in consultation with the MoET and it was
agreed that the focus should be on NFE and post literacy support aiming at enhancing the
capacity of the newly established CLCs. While there is a strong donor presence at all levels in
the formal sector, UNESCO is the only donor providing systematic technical and financial
assistance to the development of non-formal education
(NFE) in Vietnam.                                        The check list for the mid-term
                                                                     evaluations does not explicitly address
In Tanzania, the overall framework of the Cap-EFA                    relevance of the project objectives and
programme was based upon a report of a national                      activities to the identified needs. The
study team. The project design was based upon                        relevance to the national needs was,
UNESCO supported pilot programmes implemented in                     however, first of the assessment criteria in
collaboration with the UNESCO Cluster Offices in                     the proposal assessment sheet, followed
Ghana, Kenya, and Tanzania. The design of the                        by documented national consultations.
                                                                     Literacy and teacher training are included
programme took into account and built on the
                                                                     as proposed areas of UNESCO
outcomes of the reviews of the pilot programme and                   intervention in the UNESS documents as
was prepared in cooperation between International                    well, suggesting continued relevance of the
Reading Association (IRA), UNESCO and government                     supported areas.
of Tanzania.



62
     http://unesdoc.unesco.org/images/0014/001485/148566E.pdf, see page 17.


                                                                                                           30
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

The justification for the project in Tanzania is that UNESCO has been working in a largely
neglected sub-sector through support to adult education and non-formal sub-sectors. In doing
so it supports the government’s goal of addressing poverty through the National Strategy for
Economic Growth.

The Cap-EFA project is testing diagnostic teaching and the project is beginning to be embedded
in the two key departments (teacher education and non-formal adult education) of the Ministry
of Education and Vocational Training in Tanzania. It would have been expected, however, that
this was the situation already at the design stage.

The mid-term evaluation report of the Cap-EFA project in Tanzania report spends a lot of time in
justifying the intervention, but there seems to be an unquestioned acceptance in the report that
there was room for this sort of project. Whether it was an appropriate intervention and one that
fits the UNESCO mandate functions for implementing Cap-EFA and that it was pitched at an
appropriate level was not questioned. For example, UNESCO could engage more closely in the
development of a National Teacher Strategy, which still missing in Tanzania.

To ensure common understanding of the Cap-EFA objectives within national priorities in
Angola, all UNESCO implementation partners (Windhoek, Maputo, BREDA, UIS, IIEP, HQ) met
for a joint planning mission in early 2007. The Cap-EFA activities - development of National
Qualification Framework for Teachers and support to distance learning and establishment of
Community Learning Resource Centres – were in general found to be relevant and needed
during the field visit. There are, however, many initiatives to support teacher training in Angola
and the Ministry of Education is having difficulties in managing and coordinating them.


            4.5         Effectiveness
The development of a nationwide LIFE Resource Support System in 64 provinces in Vietnam to
institutionalise sustainable literacy and non-formal education capacities in more than 8000
CLCs has been a major accomplishment.
However, as highlighted in the mid-term report     In the mid-term evaluations the
– and documented in several meetings during        effectiveness of the projects was assessed by
                                                   assessing the extent to which the expected
the mission – there is some uncertainty
                                                   results are being achieved, and by rating how
whether the rapid expansion of CLC has been        instrumental the programme has been in
matched by similar expansion of capacity to        developing policies and programmes (implying
deliver substance. In a number of provinces        this being the objective of the programme).
there is a risk that CLCs have been established    All mid-term evaluations, except for Niger, are
by name rather than by substance. It also          of the opinion that the progress towards
appears that the plans might have been too         achieving results is satisfactory. Great majority
optimistic     about    acceptance     rate   of   (seven out of 11) deemed the programme very
recommendations of new standards – as e.g.         satisfactory in developing programmes and
the acceptance of LAMP standards is likely not     policies. However, despite the positive
                                                   assessment, seven evaluations recommended
to be fully achieved during the life time of the
                                                   readjustment of programme activities of which
programme.                                         three evaluations recommended also major
                                                          revision in the work plan
In Tanzania, Cap-EFA aims at improving
quality of learning and teaching by expanding Diagnostic Teaching programme to the teachers
and Teacher Training Colleges, including preparation of teaching and learning materials and
development of teachers’ manuals. The field visit found that there is a high degree of
acceptance in the teacher education department and amongst key institutions that the
diagnostic teaching project is important in raising the quality of teaching in the classroom.
Importantly, it has proved popular with the teacher trainers in the teacher colleges and those
teachers who have been exposed to the approach. The field visit confirmed that teacher




                                                                                                 31
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

educators, curriculum developers, inspectors who participated in the programme had acquired a
shared understanding of literacy and a common vision for achieving that goal.

According to the mid-term evaluation, capacity development functions have ‘permeated the
whole programme, for developing the knowledge base and insights of individuals while
strengthening and expanding their abilities to contribute to the institutions in which they work
and the communities in which they reside’. This might be the case, but this has to be set in
context where fairly small numbers have been trained as trainers and raises questions about
effectiveness and efficiency of the intervention.

Lately the lessons learnt from the project have been shared “upstream” with the Ministry of
Education and Vocational Training and development partners. However, to verify acceptability
and effectiveness and appropriateness of diagnostic teaching to the teacher college training
programmes, an independent evaluation is recommended.

The Cap-EFA project in Angola is facing challenges with all four project components.
Preparations for National Qualification Framework for Teachers (NQFT) were halted due to
elaboration of Teacher Master Plan, in which UNESCO did not and was not invited to
participate. The development of the NQFT could only begin after the approval of the Teacher
Master Plan and a working group for detailed elaboration of the NQFT was established in
January 2008.

Technical Assistance for partial review of curriculum by mainstreaming cross-cutting issues and
production of materials is suffering from shipment problems, which has delayed arrival of
science kits. Planned production of 15 modules for Open and Distance Learning is assessed
too optimistic and of the three supported Community Learning Resource Centres one only
needs to be furnished (Cacuaco), one is under construction (Caála) and one has not started yet
(Cunene). The Centres are under the auspices of Ministry of Social Communication, with
connections to several ministries. The management of the Centres is still not decided. These
challenges were described by an Angolan counterpart in the following way:

                           • Consultants are to appear and give parameters, to mark and
                             indicate ways. I still don’t feel this from them. This project
                             has not yet legs to walk.
                                                -Interview with Cap-EFA counterpart in Angola

           4.6         Efficiency, project and programme management

In Vietnam, the Cap-EFA programme has been implemented in a flexible manner allowing for
adjustments of results and activities to reflect the changing needs agreed to between the
National Project Execution Team of MoET and
UNESCO and the activities have been                Efficiency and project and programme
implemented efficiently in the 64 provinces.       mabnagement issues were addressed in the
                                                         mid-term evaluations through assessment of
The Cap-EFA support is not included in                   project document and work plan as tools for
                                                         coordination and implementation and whether
national budget, but is being managed by the             the costs are justified by the results. Except for
UNESCO Hanoi office following UNESCO rules               Niger, in all countries project document and
and regulations. The implementation of the               work plan were deemed satisfactory (8) or very
CLC component of the Cap-EFA programme is                satisfactory (2) as coordination and
based on direct contracts between UNESCO                 implementation tools. All evaluations found the
and all 64 provinces of Vietnam, and as each of          costs justified by the results.
them include payments in three instalments the




                                                                                                          32
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

Cap-EFA programme has led to an additional management burden of UNESCO Hanoi office –
which, however, has to be compared against the programme budget allowing for recruitment of
four staff.

In addition to UNESCO Hanoi office, the provinces have faced severe capacity problems in
meeting the requirements of the UNESCO procedures, which have delayed release of funds.
Due to a rather short time frame of the programme, this has led to a situation where several
provinces have been forced to front expenses through funds from national sources. All parties in
Vietnam agree that UNESCO regulations are time consuming to follow and have led to micro
management as majority of contracts is well below USD 10.000.

Management cooperation between UNESCO Paris and Hanoi has functioned well. The Hanoi
office complimented Paris for facilitating development of a highly informative questions-answers
system as well as for providing very detailed feed-back on monthly expenditures.

The mid-term evaluation in Vietnam was carried out as a self-evaluation by MoET and
UNESCO Hanoi. Although not allowing for external perspectives, the evaluation enabled the
MOET officials to visit a number of provinces with slow implementation, and did therefore allow
for joint learning by MOET and UNESCO officials. With only four months left of the programme,
only minor changes can be implemented. However, proposals to study the qualitative aspects of
CLCs and to further promote up-stream functions of UNESCO appear relevant. It should be
noted that the National Programme Execution Team was not fully aware of the criteria applied
for assessing UNESCO aspects such as the family approach.

In Tanzania, as in Vietnam, the Cap-EFA support is outside the national budget and is
managed by the UNESCO Tanzania office following the UNESCO rules and regulations. The
number of office staff to manage the technical programmes is inadequate and has meant that
the members spend an inordinate amount of time in the office dealing with administration and
attending meetings.

Despite decentralisation, the mid-term evaluation reported, and it was confirmed in
discussions that regulations for disbursing project funds are time consuming, have not
allowed for sufficient flexibility and in some cases led to inefficiencies and disruptions in the
scheduling of activities. This has caused problems in scheduling activities and disruptions in
organising some of the activities. The MTE reports that, in some cases, the disruptions in
disbursements have led to higher expenses in managing project activities (e.g. higher costs
for facilitators’ travel rebooking, etc.).

The MTE report of the Tanzania Cap-EFA programme is very analytical and detailed. It is
very informative with its comments and suggestions for the way forward that are appropriate
and appear relevant. One of its main contributors and author is also the Director of
International Relations in the institution that is the main service provider in the project (IRA).

The activities in Angola have suffered from a lack of proper office with necessary equipment
and a vehicle, without which working in Luanda is very difficult. The Ministry of Education also
complained about UNESCO’s rules being very strict and requiring heavy administrative
procedures. Cultural issues, especially observation of rigid protocol, has made access to people
more difficult and contributed to challenges in information flow and sharing. An additional
problem was raised concerning the use of interpreters, which has led to loss of information and
knowledge according to the Angolan stakeholders.




                                                                                                 33
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report


           4.7         Sustainability and chances for scaling-up
In Vietnam UNESCO has been able to leverage the ongoing national CLC development
processes. The project experience and its
outputs have already been used for MoET’s             All mid-term evaluations believe that the
                                                      benefits of the programme will last beyond
preparation for the National Education Strategy
                                                      completion of extra-budgetary support. All
2008-2020. The CLC regulations, which have            MTEs also are of the opinion that
been awaited since 2005 and now reportedly due        UNESCO should continue to focus on the
in March 2008, could provide the basis for MoET       area of intervention.
to scale up activities based on national funding
with strategic UNESCO inputs. As discussed
above under effectiveness, there is some uncertainty whether the capacity of the CLSs to
provide quality services is sufficient, and there appear to be needs for further capacity building
of CLCs. It is, however, doubtful whether UNESCO should engage itself further in in-depth,
large scale capacity building.

Testing diagnostic teaching in Tanzania is an example of an innovative project, which should
now fit into the regular programme of the MoEVT and be managed and financed through normal
channels. The reinforcement of the technique has embedded a good level of understanding to a
small group of participants and seems to have run its life. In accordance with the project work
plan, UNESCO office is increasingly engaging central administration level actors, and it is time
now for the programme to be absorbed and incorporated into the teacher education sector
planning. It is questionable, whether additional years of piloting would increase sustainability of
the project.

           4.7.1       Cooperating Partners
As UNESCO’s resources are limited, the sustainability of the activities initiated by UNESCO will
depend on financing from 1) Member State government through the national budget, and 2)
development partners. At the project proposal phase, the identified partners in all countries
include the central administration and I/NGOs, universities and regional administrations. Sister
UN organisations are identified partners in most countries but of the multi- or bilateral partners
only five partners were identified all together. Potential co-funders, as identified at the project
proposal phase, are even fewer.

During the project implementation, the FO’s have been able to link up with additional
cooperating partners. On the basis of updated information from the FO’s (March 2008), the
number of new cooperating partners in Cap-EFA activities has increased considerably. The
partners are typically UN organisations (especially UNICEF and UNDP) and bilateral partners
that do not usually participate in sector wide pooled financing or budget support modalities. See
Annex 8 for the list of identified cooperating and co-funding partners at the project proposal
phase and in March 2008.


4.7.2    UNESCO in national cooperation                    The mid-term evaluations found Cap-EFA
                                                           very satisfactory in generating effective
         mechanisms                                        collaboration with UN partners in Guinea
Despite the critical mid-term self-evaluation, in          (only UNICEF) and Niger (only UNDP) but in
Vietnam UNESCO was found to be well                        Tanzania, Vietnam and Sierra Leone it was
                                                           deemed unsatisfactory. This is an important
perceived by other donors. However, the other
                                                           finding especially for Tanzania and Vietnam,
donors expressed during the field visit that they          which are one-UN pilot countries.
would like to see UNESCO more actively
involved in coordination processes. Donors                 MTEs found generation of effective
suggested that UNESCO could initiate e.g. an               cooperation with bilateral partners
up-date of the EFA plan to reflect new realities           satisfactory in eight countries. Both in
and use that as a platform for further                     Vietnam and Tanzania cooperation with
                                                           bilateral donors was considered
                                                           unsatisfactory.

                                                                                                      34
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report

involvement. Also, the Ministry in Vietnam perceives UNESCO as a very important actor in the
field of NFE and welcomed further policy advice to be provided.

In Vietnam UNESCO was co-chairing the Education Sector Group at the time of its
establishment but has since stepped down. UNESCO was offered to co-chair Education Sector
Group in 2008 but had decline due to limited human resource capacity in the Hanoi office. As a
member of the One-UN coordination group, UNESCO has co-convened the education group
and been involved in monitoring of and defining allocation priorities for funds under the One-UN
plan.

Also in Tanzania UNESCO is perceived as an important player and the right partner to support
teacher education. UNESCO has been an active member of the UN-coordination mechanism
and has been actively involved in the development of the UNDAF, particularly in the design of
the education component. UNESCO’s expected results of the Cap-EFA programme (as well as
the ones from the Regular Programme) all contribute to the relevant UNDAF country
programme outputs and outcomes.

Since end of 2006, the UN agencies working in education in Tanzania, chaired by UNESCO,
have met regularly to improve coordination and integrate interventions through joint planning. In
addition to being a member of the develop partner group, UNESCO has added additional
responsibilities through its role as secretariat for UN cooperation in education.

            4.8          Comparative advantage and added value through family
                         approach
As with the capacity development (see Chapters 2.6 and 2.7 above), there is no standard,
thematic or geographical definition of UNESCO’s added value or comparative advantage.63 In
the field visits, UNESCO’s comparative advantage was found in technical expertise, standard
setting, ability to bring partners together and linking results of pilot projects to policy
development.

Though not clearly stated, UNESCO’s comparative advantage and added value can (and
perhaps should) be sought from UNESCO’s own
                                                     In the mid-term evaluations use of
resources and often referred expertise in            UNESCO’s comparative advantage was
education as compared to expertise procured          assessed very satisfactory (4 countries) or
from outside. According to the consolidated          satisfactory (7 countries). However, many
progress reports from February and July 2007,        evaluations do not discuss UNESCO’s
the Cap-EFA Programme has strongly promoted          comparative advantage in more detail. In the
and     strengthened      the    "UNESCO-family-     four evaluations (Angola, Senegal, Vietnam,
approach" (see Chapter 3.2.2 above). The             Egypt) where comparative advantage was
consolidated reports state that the approach has     discussed, it was usually related to either
received positive reactions from the Field Offices,  UNESCO’s five key functions or to its
                                                     normative role, cross-cutting issues and
who report that the approach has helped to
                                                     family approach. These are also mentioned
increase significantly the technical resource base   in the progress reports.
of UNESCO for its country-level actions. It has
been instrumental in adding value of UNESCO
vis-à-vis national counterparts, especially in the countries where UNESCO is not physically
present or has FOs with few staff members.



63
  This issue has recently been addressed by UNESCO’s Education Sector’s Communication Papers, of
which the first set has been prepared. The first set of communication papers consist of ED sector’s core
messages and three thematic communication papers on Literacy, Teacher Training and Secondary
Education.


                                                                                                       35
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

The UNESCO family approach activities represent 19 % of the programme budget and all
projects benefit from the services of several UNESCO units. All projects have requested
services from the HQ, but it seems that field offices demand specialised services rather from
institutes than from regional bureaux, except for Bangkok. Seven countries are supported by the
UNESCO Institute of Statistics (UIS) and its competence in LAMP. Six countries are benefiting
from services from UNESCO Institute of Lifelong Learning (UIL) and the Bangkok Regional
Bureau.

Consolidated report (July 2007) also points out that Cap-EFA countries implementing literacy
and non-formal education projects apply more the family approach than Cap-EFA countries
implementing teacher education projects (in average 16% and 10% respectively of total country
programme funds). The report suggests that this may be an indication that UNESCO has more
specialized services to offer across its structure in the area of literacy, whereas field offices
make more use of local competencies or other organisations to provide technical services in
teacher education contexts. Another explanation may in the insufficient capacity of UNESCO
International Institute for Capacity Building in Africa (IICBA) to provide relevant services to the
member states64. IICBA relies heavily on external experts in its service provision, but for
example in Sierra Leone IICBA’s input was highly appreciated.

Consolidated reports and country progress reports point out that there have been problems in
coordination and in availability of services from within the UNESCO family. Other reported and
recorded concerns relate to short duration of visits. These challenges were confirmed in the
interviews with FO representatives, and raised also in the Training Course on Education Policy
Analysis65 in which trainees from FOs requested explicit information about resources (technical
capacities and instruments) available in UNESCO, where these resources can be accessed and
how they could be used to “sell” UNESCO in the framework of joint activities with partners at
country level. According to the training course evaluation, most FO staff members were
unaware of existing UNESCO internal capacities, experiences, materials and instruments
related to policy and planning work.

In Bangkok regional bureau, experts mentioned that that their roles were not always clear. The
reason for this was found in the nationally driven nature of Cap-EFA and at times, the FOs do
not have full understanding of the areas of technical expertise available in Bangkok or of the
technical issues they are requesting support to. This sometimes leads to problems and lack of
clarity in scoping of inputs from Bangkok based experts.

UNESCO Institute of Lifelong Learning (UIL) that is
                                                                     In the mid-term evaluations
responsible for coordinating LIFE, finds UNESCO’s                    added value to the programme
comparative advantage in the combination of different                through UNESCO family
operational modes – policy driven research, evidence-based           approach was rated very
advocacy, action-oriented capacity building and networking.          satisfactory in Angola, Vietnam
UIL believes that the “family approach” is a good                    and Pakistan, satisfactory in
implementation mechanism, but finds that it would benefit            seven countries and
from more clear rules and coordination mechanism. UIL                unsatisfactory in Sierra Leone.
pointed out that the availability and preparedness of experts
could be better ensured, if the institutes and centres providing services to the FOs were
informed of the progress of the project on an on-going basis. This could be ensured if the
centres and institutes took part in the monitoring of the activities. UIL also pointed out the limited
resources in the institute as a critical factor.

64
   See Davis: Consolidated Findings of the Evaluation of UNESCO’s Major Programme 1 Institutes and
Centres, 2006.
65
   Professional Upgrading of UNESCO Education Sector Personnel: Training Course on Education Policy
Analysis – A joint undertaking by ED/EO, ED/SFS and IIEP, 21 Jan to 1 Feb 2008. Participants included
28 staff members, of whom 21 were from FOs.


                                                                                                   36
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report



In Vietnam the inputs from Bangkok RB have been of short term nature, and it appears that
while being cost efficient from the perspective of the Cap-EFA programme, the effectiveness of
the inputs depends on the UNESCO FO staff capacity to follow-up on the inputs provided by
experts from Bangkok. The pre-occupation of the staff with time consuming small contracts’
management and constraints of the UNESCO Hanoi office to provide technical specific follow-
up may constitute a barrier for MoET acceptance of up-stream advocacy and standard setting.

In Angola the UNESCO collaborative moment showcased in the joint meeting early 2007
appear to have withered since according to the management documents, mid-term evaluation
and field visit.

            4.9          UNESCO’s key functions
There appears to be no standard, single definition on how the five key functions for UNESCO –
standard setting, idea laboratory, clearing house for information, capacity building and catalyst
for international cooperation (see also Chapter 2.1 above) - are to be addressed or
operationalised in the context of EFA or in general66. The mid-term evaluations and progress
reports offer a range of explanations and activities in fulfilling the functions. The following
activities and roles have been identified for UNESCO in Vietnam and Tanzania by this
evaluation.

            4.9.1        Capacity Building
Capacity building is the main focus of the programme in Vietnam and substantial achievements
have been made on building capacity of Community Learning Centres (CLCs) at provincial level
as well as of targeted departments within the MoET. According to the mid-term evaluation only
14 out of 64 provinces were found to be lacking behind expectations. In-depth knowledge of the
general functioning of CLCs at the commune levels is, however, scarce as limited feed-back
from communes is flowing up-stream and as pointed out in the mid-term evaluation report a
further study is needed as to establish information on the actual functioning of CLCs.

Also in Tanzania capacity building is a major focus of the project design. Much of the design
was built on the strengths and success of the previous pilot programme on building individual
competencies in groups of educators in diagnostic teaching techniques.

Capacity building has targeted professional development of a small group of teachers’, tutors’
and others in understanding new literacy concepts, its development and delivery. Capacity
building efforts have been more successful in enhancing knowledge and skills of individuals
rather than institutions and organisations although this is now beginning to change as the
project is supporting key institutions. In this regard, the project has more lately been successful
in broadening the involvement of inspectors and curriculum developers as part of targeting
institutional support.

            4.9.2        Laboratory for new ideas or addressing emerging challenges
In Vietnam, the project serves as a platform for generating political support and public
awareness of the broader, ethical issues of NFE such as education as public good, life-long
learning. With the help of this project, UNESCO therefore played its role as laboratory of ideas
within NFE e.g. by introducing cross-cutting issues and life skills, and propelled the government
to consider appropriate NFE initiatives.



66
  The earlier idea to cross-tabulate projects or project components with key functions was abandoned as
impossible and futile for evaluators to fill in.


                                                                                                    37
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

In Tanzania, the project has been successful in demonstrating appropriate approaches to
teaching and learning that address identified needs of the sector. The approach is that improved
literacy skills lead to improved overall learning outcomes and complementary approaches of
harmonising fragmented MIS and establish a sustainable ESMIS. The idea here is to provide a
systemic view of the education sector for evidence-based policy and decision-making for the
sector and testing them through pilot projects.

The project is now beginning to act as a clearing house and platform for encouraging
engagement of MoEVT departments and key institutions through disseminating knowledge and
best practices mainly in the links between literacy and teaching. The role as a laboratory of
ideas has been implemented late in the project with an understanding that the capacity building
efforts focused on training of trainers should move upstream.

            4.9.3       Clearing house for sharing or disseminating information, testing an
                        innovative approach
In Vietnam, the main achievements are web-based and practical information sharing on
innovative approaches, the incorporation of cross-cutting issues and life skills into NFE training
material as well as providing seed-money for operationalisation of CLC activities at a time
where provincial DoETs were awaiting issuance of regulations from MOET on how to support
the functioning of the vast number of newly established CLCs.

In Tanzania, UNESCO has been responsible for raising renewed attention to how a teaching
technique can contribute to quality education. This is not surprising since there is widespread
evidence that it is teaching that is a key element of quality improvement. The approach or
technique has been linked to the need of promoting a reading culture in the country, and the
dissemination of the diagnostic teaching programme and approaches to literacy development.
According to the mid-term evaluation the project can be considered ‘to have contributed, to
some extent, to direct the sector’s attention to the critical role played by higher level of writing
and reading skills in the overall improvement of student’s learning outcomes’.

Lately, there have been up-stream activities aiming at sharing the lessons learnt from the
project with development partners, MoEVT and the more catalytic approach to the key
institutions. This is proving to be effective in embedding the ideas and approach at these levels.

            4.9.4       Setting standards
In Vietnam, the CapEFA Programme introduced LAMP, a household survey on literacy, in order
to reply to the challenges and opportunities identified from the national exercise of the EFA Mid
Decade Assessment for improvement of the quality of data and information. The progress on
acceptance of LAMP has been slow and the related objectives are not likely to be achieved fully
in the life time of the programme. This is due to (among others) MoET sensitivity on whether the
proposed standards for criteria setting and definition of target group could possibly lead to
results inferior to the existing information on learning acquirement, and worries associated with
the costs of the LAMP.

In Tanzania, the Cap-EFA supported development of Lifelong Learning Management
Information System (LL-MIS), which is based on the NFE-MIS methodology for mapping and
monitoring adult and non-formal education. It will contribute to improving the quality of data and
information since its outputs will feed into the Education Sector Management Information
System (ESMIS). The Cap-EFA component of LL-MIS will be integrated into the education
sector policy guideline, which is grounded in critical linkages between sub-sectors’ policies,
strategies, legal frameworks and promotes sector-wide planning, budgeting and outcomes-
oriented monitoring.




                                                                                                 38
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

           4.9.5       Catalyst for international cooperation
The catalytic role of UNESCO for international cooperation is not very visible in the Cap-EFA
programme. There are, however, useful experiences on bringing national players together67.

In Vietnam, cooperation between authorities at commune, provincial and central levels has
been promoted. Cooperation with a number of INGOs such as ActionAid has also been
promoted and there are several examples on how the Cap-EFA programme and NGOs have
worked together at provincial level.

Coordination with other donors working primarily in the formal system has been limited to
logistical cooperation such as compilation of materials from different agencies and in terms of
information sharing particularly with UNICEF.

In the short time of Cap-EFA implementation in Tanzania, UNESCO has been successful in
raising levels of cooperation between the key institutions and the major beneficiaries through
involving them in the training functions.

UNESCO is also playing an advocacy role, informed by national and international research and
evidence from the project. Cooperation has been secured between key institutions, MoEVT and
development partners on the importance of higher level of literacy development as key to the
overall improvement of the quality of national education.

Indirectly, Cap-EFA interventions provide and have provided useful experiences to be shared in
e.g. regional and international conferences on literacy and teacher training.

           4.10        Upstream or downstream?
The invitation letter to the FOs to seek financing from Cap-EFA and the attached guidance note
(see also chapter 3.2.2 above) emphasised that the programme is exclusively dedicated to up-
stream capacity building activities. This was included also in the evaluation grid for project
proposals, which required specific justification for proposed downstream activities. The
difference between upstream and downstream is not explained in documentation but here it is
understood in relation to the direct (not final) beneficiaries of the Cap-EFA activities68.

On the basis of the project proposals and progress reports, the direct beneficiaries of the
activities in the programme can be divided into three categories: ministry officials, policy makers
(upstream), regional administration officials, NGOs, trainers of trainers, universities and other
intermediate organisation (mid-stream) and teachers, illiterates, students (downstream).

Table 13: Direct beneficiaries of Cap-EFA activities
                     Upstream; Ministry,      Mid-stream; trainers of                      Downstream;
                        policy makers              trainers, NGOs,                      teachers, illiterates,
                                               universities, regional                        students
                                                    administration
  Angola           x                          X                                     x
  Bangladesh       x                          X                                     x
  Egypt            X                          x                                     x
  Guinea           x                          x                                     x
  Morocco          x                          x                                     x
  Niger            x                          x                                     ?
  Pakistan         X                          x                                     x

67
  Referred to as “honest broker” in background documentation.
68
  In 31 C/4 and 33 C/5 documents upstream and downstream are linked to UNESCO’s key functions but
the definitions are different. See Chapters 2.2 and 2.2.1 above.


                                                                                                            39
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report

    Senegal          X                                x                                x
    Sierra           x                                x                                x
    Leone
    Tanzania         x                                x                                x
    Vietnam          x                                x                                x

Typical upstream beneficiaries of activities, ministry personnel and policy makers are found in
all projects but not exclusively, as mid-stream and downstream actors are identified in all
proposals as well.

In Vietnam, the approach emphasises the capacity of the local CLCs to reflect and meet the
local needs. Consequently the project focus and the budget have been on downstream, in
activities aiming at capacitating the CLCs to perform their role. The upstream monitoring,
however, has due to insufficient feed-back loops been inadequate, leading to uncertainty at
central level on how well the CLCs are operating in practice. Although some upstream activities
aiming at building capacity at central level have been included in the programme, they have not
been as prominent as the downstream activities. A more sufficient flow of information from lower
to central levels would have enabled a closer link between the experiences gained at
decentralised level to enhanced learning at central level.


5       Conclusions
               5.1        Relevance, effectiveness, efficiency and sustainability of
                          Cap-EFA financed activities

In the absence of clear programme level outcomes, measurable targets and identification of
sources of information, the assessment against the “traditional” OECD/DAC defined parameters
for development cooperation and evaluations, the assessment of relevance, effectivness,
efficiency and sustainability is based on the mid-term evaluations that were carried out in all 11
programme countries. On the basis of the mid-term evaluations, Cap-EFA implementation has
been successful. The activities in literacy and teacher training are largely coherent and aligned
with the relevant UNESCO policy and strategy documents.

Under literacy, Cap-EFA activities in seven countries are aligned mainly with two of the three
Cap-EFA literacy implementation strategies. In all projects literacy managers have been trained
and monitoring and evaluation tools and functions have been developed. Integration of literacy
into different development programmes has not been that prominent. In teacher training, Cap-
EFA has supported development of teacher policies and through cooperation with relevant
teacher training institutes, improved teacher training, especially by developing the curricula.
Two other implementation strategies, effective use of ICT and promotion of strategic social and
political dialogues and partnership building have received less attention.

In Vietnam, Tanzania and Angola, the Cap-EFA activities were deemed relevant and justifiable
to the needs in the country. However, in all countries field visits identified needs at a more
upstream policy level that would perhaps have fitted better UNESCO’s mandate.

The mid-term evaluations (MTE) that were conducted in all programme countries, found Cap-
EFA effective in delivering for results and in developing policies and programmes. This was
largely confirmed in the field visits, with reservations to quality (CLC capacity in Vietnam) and
small number of beneficiaries (Tanzania). In Angola there are problems in all project
components to achieve the planned results.




                                                                                               40
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report

The MTEs deemed Cap-EFA also efficient by justifying the costs against achieved results and
by appreciating the project document and the work plan as useful tools for coordination and
implementation. The main finding in the field visits was the heavy administration load caused by
UNESCO rules and procedures in project implementation, which has put substantial burden on
UNESCO staff and also on the cooperating partners. The programme has commendably
introduced management tools, formats for country reporting, and proactive financial monitoring,
which function well and are appreciated by the Field Offices. However, the use of standardised
reporting formats is not extended to the consolidated reports, which suffer from the
programme’s unclear objective setting and lack of clear programme logic, which the reporting
should follow. The consolidated reports should include more detailed financial information on
what has been financed, and make sure that the narrative reporting follows the finances. More
detailed financial information is available through the regular financial monitoring system.

On sustainability, the MTEs unanimously believed that the benefits from the programme will
last after the completion of the extra-budgetary support. During the field visits, the sustainability
and scaling up was not deemed as foregone conclusion. Though the number of cooperating
partners has increased during the programme implementation, the number of co-funding
partners was rather limited. It is expected that the chances for scaling up and for improved
sustainability would increase if the cooperation with the financing partners was confirmed
already at the planning phase.

The field visits found UNESCO to be well perceived by the development partners and Member
State governments. UNESCO has played an active role in coordination in education sector,
especially within the UN system

The identified challenges especially in relation to relevance and sustainability could be
addressed during the planning phase with the help of more rigorous appraisal (or ex-ante
evaluation) process.


            5.2              Cap-EFA response to identified challenges
UNESCO is a custodian for EFA with a mandate to coordinate the EFA partners. It is also, as a
specialised UN agency, expected to serve as the lead technical agency and resource on EFA
for the Member States. This role is to be fulfilled mainly through UNESCO’s five key functions:
1) idea laboratory, 2) standard setter, 3) clearing house, 4) capacity builder and 5) catalyst for
international cooperation. These functions are enforced by the Field Offices with the help from
Head Quarters and UNESCO Institutes and Centres, which all are expected to be centres of
excellence in their respective field of expertise. It is presumed, or hoped, that this mandate and
vast resource base of excellence would bring added value in the services provided to the MS
and give UNESCO comparative advantage over the other development partners.

A number of challenges, as described in Chapter 2 above, have been identified that limit
UNESCO to effectively deliver on its mandate. Though not explicitly targeted in the Cap-EFA
programme document (2005), Cap-EFA is also addressing these issues. The summary of Cap-
EFA responses to the identified challenges is presented in the table 14 below.

Table 14: Cap-EFA responses to identified challenges
           Challenge                                   Cap-EFA Response
   Limited financial
                                   Added USD 11 million.
   resources
   RB alignment with EXBF          Aligned with the priorities stated in the 33 C/5 though
                                   MOU that enables pooled and harmonised support.
   Prioritisation/ unclear         Thematic focus on prioritised literacy and teacher
   focus                           training, emphasis on LCDs less evident as well as



                                                                                                  41
                          Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                   Final Report

                                    exclusiveness on upstream policy advice.
     Professional capacities/       Added human resources (683 person months), not
     technical expertise            certain if increased professional capacities for policy
                                    advice.
     Planning to match needs,       UNESS
     normative role and
     available resources
     Internal cooperation and       Family approach.
     coordination
     Lack of standard               Not identified.
     definition of key concepts


              5.2.1        Additional resources, harmonisation and alignment
Cap-EFA has had a major role in the implementation of UNESCO’s education programme. The
available USD 11 million over two years is a substantial amount of additional financing in
UNESCO’s education budget. As the two programme areas, literacy and teacher training, are
relatively prioritised also in UNESCO’s regular budget (i.e. limited fungibility), it is safe to
assume that Cap-EFA has to a considerable extent enabled and supported implementation of
“the programmatic core to of UNESCO’s response to EFA at country level” as described in 33
C/5.

The programme has also enabled a substantial, 659 person-months increase in human
resources in the field offices in addition to the mobilisation of a vast number of international and
national experts. This contributes greatly to the challenge of limited resources in the Field
Offices. Cap-EFA has also enabled project implementation in countries where UNESCO does
not have a permanent field office. This expansion is not without problems, as witnessed in
Angola and highlighted e.g. in the evaluation on UNESCO support to national planning69. The
expansion has led to increased management costs, and from quality assurance and
management back-stopping point of view it is a challenging constellation. For activities in a
country with no UNESCO Field Office, close cooperation, including management support should
be ensured from UN organisations with permanent office in the country.

The Memorandum of Understanding between UNESCO and the Nordic Countries (and
Switzerland) has allowed integrated planning and programming and full coherence of regular
budget and extra-budgetary funds, which has been identified as a major concern with the extra-
budgetary funding, and objective in the Action Plan to strengthen management of the EXBF. In
UNESCO’s Education Sector, the MOU is referred to as a model for thematic and flexible
funding modalities within the upcoming Education Sector strategy for extra-budgetary funding.

The MOU is also referred to as the most harmonised and aligned way of EXBF support to the
regular programme and budget in terms of implementation and management modalities. The
identified benefits include for example simplified administrative procedures, joint donor meetings
and consolidated reporting. The MOU has allowed integrated planning and programming and
supported coherence of regular budget and extra-budgetary funds. The alignment goes,
however, only half way as Cap-EFA planning and reporting has not fully followed UNESCO
standard procedures.

Cap-EFA has introduced management tools, formats for country reporting, and the proactive
financial monitoring, which function well. Despite initial problems and complaints about the work
required to fill in the formats, also the FO’s report to be satisfied with the more demanding
formats. The interviewed FO’s appreciated and saw the value of questions and answers and
detailed feedback from the HQ and supporting UNESCO entities for improving performance.
69
  Evaluation of UNESCO Support to National Planning for EFA, Education for Change (2006). See esp.
page 21.


                                                                                                 42
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report



Use of the standardised reporting formats is not extended to the consolidated reports, which
suffer from the programme’s unclear objective setting and lack of clear programme logic, which
the reporting should follow. The consolidated reports should include more detailed financial
information on what has been financed, and make sure that the narrative reporting follows the
finances. More detailed financial information is available through the regular financial
monitoring system.

Another dimension on the harmonisation and alignment agenda is the extent to which
UNESCO’s own implementation modalities and procedures support harmonisation and
alignment at the country level as guided in the Paris Declaration for Effectiveness70. Through
UNESS, there is a move towards more shared analysis especially within the UN system, but as
discussed above, UNESCO typically implements projects following its own arrangements and
procedures. These procedures are deemed very laborious and time consuming and they put
additional management burden not only to UNESCO FOs but also to the implementing partners,
including the Member State administration. Though the activities are aligned to the national
priorities, UNESCO refrains from using the country systems and implementation structures and
procurement systems. UNESCO is not among the SWAP or general budget support partners,
which are considered most advanced in harmonisation and alignment. It seems also that the
cooperating partners are more easily found amongst development partners that do not
subscribe to budget support or pooled funding arrangements.

            5.2.2        Prioritisation
Cap-EFA has three prioritisations: needs-based orientation to favour the Least Developed
Countries as per the MOU, thematic prioritisation to literacy and teacher training in accordance
with the Cap-EFA programme guidelines (2005) and upstream capacity building as per the
invitation letter to the FOs to seek financing from Cap-EFA.

Though LDCs constitute the majority of the countries benefiting from Cap-EFA, the prioritisation
was not unequivocal and in relative terms more non-LDCs were selected from the eligible
countries. This is probably due to the selection criteria, which emphasised technical quality of
the proposals higher than the needs in the country. There was little added value seen in the
“competitive” selection process for the pilot countries. Selection should be based on needs, not
on technical merit of the proposals, which easily leads to support to countries and offices that
need the support least.

The Cap-EFA funds are allocated directly to UNESCO’s key priorities, literacy and teacher
training, greatly promoting thematic prioritisation of the education programme. There has been a
discussion of whether the thematic prioritisation of literacy and teacher training has left other
relevant and potentially more needed projects without support and implementation. UNESCO’s
mandate covers the whole EFA and the needs in the Member States for EFA are vast and go
beyond literacy and teacher training. In addition to the identified needs in the Member States,
the most defining factor in selecting substance areas for UNESCO support should be the
availability of the required expertise in UNESCO (capacity to deliver), and suitability to
UNESCO’s mandate.

The programme emphasis on upstream policy advice is considered ambiguous71. In addition to
ministry level cooperating partners, all projects engage in down-stream activities of e.g. training
of teachers or illiterates together with mid-stream actors such as regional administrations, NGOs
and teacher training institutes. While this is important, it does not necessarily strengthen the


70
  http://www.oecd.org/document/18/0,2340,en_2649_3236398_35401554_1_1_1_1,00.html
71
  This conclusion needs to be seen against the background of no clear definition for upstream and
downstream functions and activities.


                                                                                                    43
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

image of UNESCO as an upstream policy adviser to the ministries and policy makers in the
Member States.

The evaluation was not able to conclude on the reasons for relative “downstreamism”. It would
seem, however, that there is a demand in the ministries and among the development partners
for more upstream advice from UNESCO. This would suggest that the problem is more in the
supply side. This is supported by the problems recorded in the training course evaluation of
Professional Upgrading of UNESCO Education Sector Personnel (see Chapter 2.8 above),
where the FO staff expressed being ill-prepared to fulfil their technical advisory and capacity
building role at country level.

Therefore, while the overall implementation of the programme is largely deemed successful and
the projects to a high extent in accordance with the identified needs in the selected countries,
and UNESCO strategies and policies, the strategic orientation of the programme could be
sharpened towards more upstream policy advices in countries with biggest need of support
covering the whole UNESCO EFA mandate.

           5.2.3       Technical expertise, internal cooperation and coordination
The UNESCO family approach in Cap-EFA has tried to promote the provision of services from
within UNESCO on the basis of needs identified by the field offices. The family approach has
been generally well received in the FOs but they also report on challenges in identification,
coordination and availability of experts from within the “family”. Another identified problem is
linked to the short duration of services the “family members” can provide. To ensure
effectiveness and impact of the services, the field office staff should be able to follow up on the
services, for which there is not always sufficient resources.

On the service provider side, the institutes and regional bureaux are struggling with limited
resources to meet the requests, which may come with short notice with unclear definition of
roles and expectations. There seems to be room for improvement in the enforcement of the
family approach through improved communication and closer engagement of the “family
members” in the institutes, centres and HQ in planning and monitoring of the projects. More
active participation throughout the project cycle would help ensure that the required services are
available and the experts are better aware of the expectations.

The evaluation was not able to accomplish the extent to which Cap-EFA has strengthened the
upstream technical advisory and capacity building role of the Field Offices.


           5.2.4       Planning
Cap-EFA has been instrumental for UNESCO to pilot and roll-out UNESS, which is hoped to
greatly improve the relevance, effectiveness and sustainability of UNESCO’s activities at the
country level. Planned implementation of UNESS will provide UNESCO with a strategic
planning document that 1) ensures relevance of UNESCO’s activities to countries’ needs and
priorities, 2) increases cooperation and harmonisation with development partners, especially
within UN-system, and 3) improves effectiveness of UNESCO’s activities. The fulfilment of these
objectives remains to be seen as overwhelming majority of the documents are still only found in
draft form.


           5.2.5       Definition of key concepts
Cap-EFA implementation has suffered from a number of unclarities, some of which are beyond
Cap-EFA and should be addressed by UNESCO as a whole, while some are programme
specific.



                                                                                                44
                           Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                    Final Report



The objectives in the programme document (2005) are coherent with 33 C/5 and the Strategy
and Vision for LIFE and to large extent with TTISA objectives as well. The discrepancy with
TTISSA objectives is explained by revision of TTISSA objectives and expected results after the
Cap-EFA programme and the activities were planned. The main problem, however, is identified
in the definition of problems to be addressed that is not at a level that can be realistically
addressed by the Cap-EFA programme. This has led to unclear programme intervention logic
with a mismatch between the challenges and the available resources, missing hierarchy of
objectives and lack of quantified indicators. The evaluation was not able to clearly identify which
objectives are promoted with how big budgetary allocations. While contribution of Cap-EFA to
LIFE and TTISSA is acknowledged by all, it was not possible to quantify that added value in
more detail.

Consolidated Cap-EFA programme reporting has been affected by both unclear objective
setting and lack of definition for consolidated reporting format.

UNESCO Medium-term strategy (31 C/4) and Programme and Budget (33 C/5) discuss
upstream and downstream functions in connection with the key functions. Unfortunately, the two
papers offer different definitions. In this evaluation downstream and upstream are understood
through the immediate beneficiaries of the provided services.

There is no definition on how the five key functions are to be operationalised, and the difficulty
of this task is evidenced e.g. in the reviews and in the on-going work to define UNESCO’s policy
for capacity development.

UNESCO’s comparative advantage and added value have been addressed in progress
reporting72, mid-term evaluations and in this evaluation to a varying degree. However, in the
absence of definition for these key concepts, it has proved very difficult to locate them in the
activities. For example, UNESCO strategy and policy papers point out that literacy and teacher
training are often neglected priorities in the Member States, and UNESCO can provide added
value and comparative advantage to promote them. This claim would benefit from an
evidenced-based justification and more detailed definition of what kind of added value and
comparative advantage UNESCO can bring, how and by whom. The reports from the Field
Offices offer several options for both, but there is a need for programme (if not Organisation)
level definition with clear justification and understanding how the added-value and comparative
advantage will be delivered. As a specialised UN agency for education, it would be expected
that the in-house expertise in the core areas would be readily available to Member States and
field offices in top quality.

In planning for the country level interventions, the Field Offices have faced difficulties in
identifying the available expertise and resources in UNESCO.


6              Recommendations
The evaluation recommends Capacity Building for EFA-programme to be continued and
believes that with some modifications and strategic reorientation, it can play even bigger role in
enhancing UNESCO’s mandated service delivery to the Member States. The Memorandum of
Understanding between UNESCO and the contributing donors should cover the six-years period
of the Medium-Term Strategy 2008-2013 (34 C/4) and the operational programme guidelines
and objectives should be revised and defined on a biennium basis, following the C/5
documents.


72
     Not included in the project proposal format of guidelines but introduced later to the progress reports.


                                                                                                               45
                          Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                   Final Report

The evaluation believes that the revised Memorandum of Understanding would benefit from the
existing best practices in donor harmonisation as outlined for example in Joint Financing
Arrangements and should include more detailed definition of e.g.:
    a) Objectives of the programme and scope of the MOU with reference to the relevant
        strategic documents and the programme guidelines, which include identification of
        challenges and hierarchy of objectives with indicators to respond to the challenges (Cap-
        EFA programme document to be revised biannually).
    b) Contributions, donors’ pledges;
    c) Consultations and decision making process;
    d) Disbursements;
    e) Procurement;
    f) Reporting;
    g) Review and evaluation;
    h) Auditing; and
    i) Modifications, donor accession and withdrawal

While ensuring full coherence between the Cap-EFA funding and UNESCO Regular
Programme, following from the identified challenges in UNESCO for effective service delivery,
the evaluation recommends that the objective of the revised Cap-EFA programme is geared
towards strengthening UNESCO capacity to deliver on its mandate and to better serve the
Member States in Education for All. The programmatic nature of the Cap-EFA programme
document should be strengthened with clear and explicit identification of challenges and
problems to be addressed by Cap-EFA, followed by detailed definition of the programme
intervention logic and definition of indicators against which the progress can be measured.
There must be clear goals and objectives that permit easily reportable results. There is a need
for further discussions on the kinds of results that are appropriate for these kinds of
interventions/programmes.

Regarding prioritisation, the evaluation recommends that the support from Cap-EFA should
be based on two basic parameters:
   1. Needs in the member states where explicit priority should be given to a) the LDCs73 in
      the selection of countries and b) the needs for policy advice in these countries in
      selection of themes. Recent UNESS exercises can provide useful information on this.
   2. UNESCO’s capacity to deliver. Preference should be given to the LDCs with UNESCO
      FO in the country. Thematic preference should be given to the areas where UNESCO
      has proven expertise and identified in-house delivery capacity.

Aligned with the draft 34 C/5, Cap-EFA should further accentuate upstream policy advice.
Downstream project implementation should be allowed only in exceptional circumstanced with
clear added value and feed-back mechanism to support UNESCO’s policy advisory role.

The Cap-EFA programme document should define UNESCO’s comparative advantage and
added-value with clear identification of actual service providers in UNESCO. Cap-EFA should
build on the already existing expertise in UNESCO and strengthen expertise in areas that are
deemed important but where there is a need for more in-house expertise. There should be a
mechanism to ensure that utilisation of external experts strengthens also UNESCO’s capacity,
without compromising the quality or quantity of support to the Member States/beneficiaries.
Cap-EFA would benefit from cooperation with the on-going preparations of UNESCO’s policy for
capacity development. Cap/EFA should underline the importance in making full use of the one-
UN processes in the development of the UNESCO support to the education sector locally, and
of other existing cooperation mechanisms set up by the local education group.comparable data
in its fields of competence


73
     Or countries with the lowest Education Development Index in the Global EFA Monitoring Reports.


                                                                                                      46
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

Once the available expertise in UNESCO is mapped, the identified experts should participate
closely in the preparation and in monitoring of the support funded by Cap-EFA to ensure
smooth delivery of expertise from within UNESCO.

A more rigorous ex ante appraisal of the programme logic, options for design and
implementation and expected impacts should be conducted in future and would strengthen
sustainability and in addition will also strengthen relevance and effectiveness. Sustainability and
chances for scaling up piloted activities must be given careful consideration in the next program.
It seems logical to move toward a programme approach in order to ensure sustainability and
greater impact.


A summary of key findings, conclusions and recommendations is included in Annex 9.




                                                                                                47
Annex 1 Terms of Reference

                               Terms of Reference for an Evaluation of
       Capacity Building for EFA (Cap-EFA) – Extra-Budgetary Programme for Technical
                                  Support to Member States

BACKGROUND

"Capacity-building for Education for All" (Cap-EFA) is an extra-budgetary technical assistance
programme to support national capacity-building activities in high priority areas for educational
development. The programme is funded through a group of donor agencies which has pooled
voluntary contributions. This basket funding modality was intended to enable UNESCO to move
from a project to a programme approach in the implementation of extra-budgetary activities, and
through this to increase coherence and contribute to an improved achievement of its Regular
Programme objectives.

The programme74 1 was launched in 2003 through a series of funds-in-trust projects financed by
Norway, Denmark, Finland, Flanders and Italy. In October 2003 a Memorandum Of
Understanding (MOU) was signed between UNESCO and the five Nordic donors (Denmark,
Finland, Iceland, Norway, Sweden) in which they agreed to pool extra budgetary funds in a
special account. Through this, they aimed to jointly reinforce the programme75. The annual
contribution to the special account through the MOU varies but is approximately $5million.

The Cap-EFA programme applies a programme approach by addressing key priority areas of
the education sector's Regular Programme, hence contributing to the attainment of the
objectives set by UNESCO's governing bodies in the C/4 and the C/5. As of 2006, the
programme moved from one-year to biennial interventions in a restricted number of countries to
allow increased focus within the general frameworks of LIFE and TTISSA763. These are two of
three key initiatives launched by UNESCO in 2005/6, while the third theme (HIV AND AIDS) is
being mainstreamed as a cross-cutting issue.

Cap-EFA has been subject to regular monitoring and evaluation at the programme and project
level, including regular progress reports and two external evaluations. At present, mid term
evaluations of the Cap-EFA projects at country level are being undertaken as part of the
Programme’s evaluation and monitoring strategy. These are expected to be finalised by
December 2008. The evaluation proposed here should draw upon all of these evaluative
exercises, and indeed should build upon all relevant previous reports in order to enhance the
focus and analytical power of the evaluation.

OBJECTIVES

The overall objective of the evaluation is to improve UNESCO performance in education by
assessing the utility of the current MOU for extra budgetary support to the realisation of
Education for All and help donors and UNESCO decide on future forms of cooperation. The

74
   The programme’ being that as described in the UNESCO policy statement on co-operation with donors for the
Dakar follow-action’ and "capacity-building for education for all”. See also
http://unesdoc.unesco.org/images/0012/001254/125428e.pdf
75
     See also http://unesdoc.unesco.org/images/0013/001336/133687e.pdf
76
  life: literacy initiative for empowerment (2005-2015), TTISSA: teacher training initiative for sub-saharan africa
(2006-2015).
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

evaluation is to assess how and to what degree the Cap-EFA programme and its modality (the
MOU) have contributed to relevant parts of the UNESCO education programme.

The evaluation will follow two main lines of inquiry:
1) The capefa Programme -the effect at the education sector programme level and
2) The MOU -the usefulness of the MOU as a modality for joint donor financing.


1) The capefa Programme - the effect at the education sector programme level

The capefa programme should be evaluated against the following standard assessment criteria:

EFFECTIVENESS: Did the capefa programme have clear goals and indicators? Have the goals
and expected results specified in the capefa programme been reached?

RELEVANCE: Is the capefa programme consistent with unescos priorities and are the
interventions at the right level?

EFFICIENCY: Are the costs of the capefa programme justified by its results and are activities
implemented in an efficient manner ?

SUSTAINABILITY: Will the supposed benefits of the capefa programme last beyond completion
of the extra budgetary support?

IMPACT: To what extent, if at all possible, could major development effects at country level be
attributed to the capefa programme?

The following specific question might focus the evaluation: How effectively have the activities
financed under the MOU cohered with Regular Programme initiatives and with other related
activities in UNESCO and its institutes? In particular the evaluation should specify:

• What has been the added value of capefa to UNESCO’s EFA efforts: at Head Quarter level?
• What has been the added value for the contributory donors?
• To what extent have the UNESCO institutes been involved in Cap EFA financed activities?
• Has the programme had an impact upon the co-ordination between UNESCO’s different
offices and institutes (BREDA, IICBA, field offices, institutes, HQ)?
• Has capefa supported collaboration with other development partners? How effective has
UNESCO’s communication and collaboration with donors and relevant multilateral organisations
been in the context of capefa?
• To what extent has UNESCO created awareness of its capacity building programme at country
level and among relevant donor/multilateral organizations generally? What have been the
linkages with other major initiatives for example FTI (Fast Track Initiative)?
• To what extent have Cap-EFA supported pilot projects been based on the country’s own
priorities and what has been the role of non-UNESCO national staff?
• To what extent do the pilot projects at country level follow a uniform methodology or a
standard approach to analysis or capacity building?
• To what extent have Cap-EFA supported pilot projects been instrumental in scaling up and
developing national policy and programmes? To what extent has Cap-EFA informed national
educational planning?
• What are the opportunity costs of the cap-EFA? Has it in any way limited UNESCO from
allocating resources to other high priority areas?
• To what extent has the Nordic support to CAP-EFA weakened or strengthened UNESCO itself
in ensuring a more coherent and holistic approach to the implementation of its education
programme?



                                                                                                  49
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

• Has the UNESCO Medium-Term Strategy and Programme and Budget provided a functioning
frame for the capefa programme?
• How has the programme evolved over the period since its inception, and for what reasons?
Has
UNESCO attempted to extract specific lessons learnt and apply these at a more general level?

2) The MOU -the usefulness of the MOU as a modality for joint donor financing.

The MOU may be seen as a model for continued coordinated programme support to UNESCO.
Therefore, an assessment of it should aim at providing information for both participating and
non-participating donors on how effective the partnership among the donors and with UNESCO
has been in fulfilling the aims of the programme. In particular the evaluation should cover the
following questions:

• To what degree has the current MOU served to reduce transaction costs, improve
coordination, harmonization and information among stakeholders?
• Are there lessons to be learned from the MOU for other comparable strategic interventions
within UNESCO and elsewhere?
• Each of the 14 articles of the MOU sets out an agreed position, policy for action or aim. How
has the MOU performed in relation to each and what have been the outcomes and
consequences?
• The MOU recalls the C/4 Medium Term Strategy as the basis for support and refers to the
UNESCO policy statement on co-operation with donors for the Dakar follow-up action. How
have these policy directions been understood and implemented in the context of the MOU?
• Has the funding modality under the MOU and the associated programmatic approach
strengthened UNESCO’s institutional capacity to respond to education development needs of
participating countries?
• To what extent has UNESCO used the opportunity of the MOU to attract more co-ordinated
and coherent support from other donors?

METHODOLOGY AND SCOPE OF WORK

Tenderers should propose clear well, argued methodologies for data collection, analysis data
management and reporting. The methodology, methods and tools should respect the UN
Evaluation Group Norms and Standards.

The programme under evaluation has a wide geographical reach operating in 11 pilot countries,
namely: Senegal, Angola, Guinee, Niger, Sierra Leone, Tanzania, Bangladesh, Vietnam,
Pakistan, Morocco and Egypt. The funding modality, in addition to the MOU signatories, is of
relevance to a wide range of interested stakeholders. The evaluation will require data capture
methods which result in rigorous, wide reaching assessments and which are applied in cost
effective ways. It is likely to be based on three main activities: text analysis of key documents –
which must include a review of previous evaluative and progress reports related to the
programme, in depth interviews with key stakeholders and field trips to a representative sample
(minimum two) of the pilot countries. Tenderers should consider telephone interviews or where
field conditions permit, web casting and video conferencing to supplement face to face and field
based data collection. Other methods -both quantitative and qualitative such as survey work,
case study and observations -applied in the most effective way within the constraints of a limited
budget will be welcomed.
Stakeholders will be a major source of data in the evaluation and proposers should be clear on
how they will gather data from the range of donors, potential non donors, wider agency
community as well as programme staff and Headquarter personnel. Tenderers may also
propose innovative methodologies as long as these are clearly appropriate, valid and credible.




                                                                                                  50
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report

The Evaluation Function of the Internal Oversight Service will provide access to relevant
contacts and information sources not otherwise available publically.

Proposers should include a management plan for the work including full details of team
composition (together with cvs) and their approach to management, delivery and meeting tight
deadlines. It is estimated that at least two persons could be contracted for up to six weeks each,
but it is up to tenderers to consider how best to set up and organize the team, task organization
and delivery.

PROFILE OF EVALUATORS

The evaluators should have a relevant professional background in education or social and
political science, they should be familiar with education sector issues and UNESCO and have
proven experience with evaluation and/or systematic assessment of development programmes.
They should have proven expertise in tasks similar to this evaluation. Specifcally they should:

• At least 15 years of programme evaluation experience and or systematic assessment of
development programmes;
• Demonstrated experience and knowledge of programme evaluation methods and data
collection;
• Relevant in country or regional experience and linguistic competencies necessary for any
fieldwork (English competency is required and basic knowledge in Spanish and French are
beneficial);
• Appropriate gender balance and geographic (north – south) representation; and
• No prior direct engagement in the activities subject to this evaluation.
For in country field work the tenderer may propose to include team members based in country
or in the sub-region.

Experience of the team should be demonstrated through listing relevant projects undertaken in
the last 5 years and by inclusion of cvs of each team member. The core team should be
independent in relation to UNESCO and not be regular UNESCO staff. In addition to satisfying
these professional criteria, tenders should note the award criteria set out in the ‘Guidance’
document associated. These criteria will be applied in the evaluation of tenders to judge the
technical quality, methodological soundness and management capability demonstrated in the
tender: specifically: 1. Does the tender demonstrate understanding of the task in relation to the
requirements set out in the Tender Specifications? (Score 20 points). 2. Is the approach set out
in the tender appropriate to address all the questions and issues in the Tender Specifications?
(Max 20 points). Does the proposed methodology, methods and tools show overall quality,
comprehensiveness, appropriateness feasibility and rigour? (Max 40 points). 3. Is the planning
and management of the work sound? (maximum 20 points). Tenderers should also note that in
addition to the quality of the tender, price will also be taken into account in order to ensure value
for money. All bids which pass the award criteria threshold of 75 points will be eligible for
assessment on the basis of cost. Financial bids should be prepared separately.

TIMING AND REPORTING

IOS of UNESCO will be responsible for contracting and managing the external evaluation. The
evaluation will report to the reference group of key stakeholders and experts who will supervise
the quality assurance and conduct of the evaluation.

An evaluation reference group will be established by UNESCO and the signatory donors may
assign up to two representatives on this committee. Donor signatories to the MOU may assign
representatives to work jointly with the contracted evaluation team as deemed necessary. All
costs related to such participation, however, will be borne by the agencies themselves.



                                                                                                  51
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

Timescales are indicative but are as follows: finalization of TOR, 26 October 2007 and launch of
request for proposals. Contract established with the consultant team end of November 2007.
Work plan would be estimated at 3.5 months with delivery of the final report no later than mid
March 2008.

OUTPUT

The evaluation deliverables shall be: an inception report, a draft evaluation report; A final
evaluation report (not exceeding 60 pages, excluding annexes, to be delivered in Word and
PDF files). A debriefing note summarizing the preliminary findings of the work, conclusions of
analysis and recommendations for a future joint donor arrangements

1. Inception Report. Week three after contract signature.
The inception report will describe how the methodology proposed by the contractor will be
implemented in the light of an examination of the quality and appropriateness of existing data,
and in particular how the methodology will address each evaluation requirement. It should draw
upon the results of an initial review of the literature and documentation relevant to analysis of
the funding partnership and the programme. The inception report therefore is more than a
workplan as it should offer the first substantive input into the evaluation on the context,
problematics, issues to be addressed and the methodology that will be applied. A kick off
meeting in Paris will be scheduled for one week after the delivery of the inception report.

2. Draft evaluation report. March 1st 2008

The Consultant(s) will prepare a draft evaluation report (not exceeding 60 pages, with annexes
as per need, to be delivered in word and pdf files). The draft report will be reviewed by the
Reference Group and circulated among stakeholders for comments.

3. Final evaluation report. March 15th 2008
The final report should not exceed 60 pages, with annexes as per need, and be delivered in
word and pdf files. It should include but not necessarily be limited to, the following elements:
executive summary, programme description, evaluation purpose and methods, major findings
(presented in terms of achievements and challenges), conclusions and recommendations,
lessons learnt and or factors contributing to the achievement of results or lack thereof. The
Executive Summary must be presented in a format suitable for presentation decision makers. It
should correspond to the suggested format: background of the programme evaluated, major
findings (key achievements and key challenges) and recommendations.

4. A debriefing note March 15th 2008
A debriefing note summarizing the preliminary findings of the work, conclusions of analysis and
recommendations for a future joint donor arrangements.

DOCUMENTATION

The following key documents are referred to in the MOU in paragraph 4 and constitute the
major points of reference for this evaluation:

• MEMORANDUM OF UNDERSTANDING between the United Nations Educational, Scientific
and
Cultural Organization and the Governments of Denmark, Finland. Norway, Sweden and Iceland
on
Support to UNESCOS activities in the field of Education For All, Paris 20 October 2003.
(Attached)
• UNESCO Medium-Term Strategy 2002-2007.(C/4) (see unesco.org)
• UNESCO’s biennial Programme and Budget for 2002/2003, 2004/2005; 2006/2007 (C/5) (see


                                                                                                 52
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

unesco.org)
• The International Strategy to operationalize the Dakar Framework for Action on Education for
All.
Http://www.unesco.org/education/efa/global_co/global_initiative/strategy_2002.pdf
• UNESCO Policy Statement on Cooperation with Donors for the Dakar Follow-up Action.
Http://unesdoc.unesco.org/images/0012/001254/125428e.pdf
• Capacity Building for EFA (see below)
• In addition the evaluation will be expected to review all relevant evaluative documentation on
the
Programme including previous external evaluation, progress reports on the Capacity Building
for EFA together with individual project evaluations which will be reporting shortly after the
planned start of this evaluation. (see UNESCO Evaluation Function website:
http://portal.unesco.org/en/ev.php-URL_ID=24383&URL_DO=DO_TOPIC&URL_SECTION=-
465.html




                                                                                               53
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



Annex 2 List of documents consulted
Management Documents

Analysis of 34 C/5 Workplans, February 2008

Capacity-Building for Education for All 2008-2013 – Multi-donor programme for Technical
Services to Member States, May 2007 and December 2007

Cap-EFA Tanzania Proposal

Cap-EFA Tanzania Progress Report #1 -#3

Cap-EFA Tanzania Mid-term Report

Cap-EFA Tanzania Work Plan July 2007

Cap-EFA Tanzania Mid-Term Review report. February 2008.

CapEFA Viet Nam Proposal

CapEFA Viet Nam Progress Report #1 - #3

CapEFA Viet Nam Mid-term Report

CapEFA Viet Nam Work Plan July 2007

Guidance Note for Building a UNESCO National Education Support Strategy (UNESS)
Document: 2008-2013, 2006

Guidance Notes for Joint Project Proposal Form in Capacity Building for EFA 2006/2007

Guidance Note to Mid-term evaluations of Cap-EFA country programmes 2006/2007, October
2007

Information note on UNESCO collaboration with multi- and bi-lateral agencies at county level in
the context of the Cap-EFA programme, updated March 2008

Invitation note for Bidding for Extra-budgetary Funds under Capacity-Building for EFA
Programme, November 2005

Memo on Additional Programme of targeted/projected extra-budgetary activities, ADG/ED,
January 2008

Minutes of Nordic Donors’ meeting, Capacity Building for EFA, Extra-budgetary programme for
Technical Support to Member States, 27-28 February 2007

(Consolidated) Progress report Capacity Building for EFA, Extra-budgetary programme for
Technical Support to Member States, February 2007

(Consolidated) Progress report Capacity Building for EFA, Extra-budgetary programme for
Technical Support to Member States, July 2007

Progress Report on UNESS roll-out, July 2007



                                                                                             54
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



Project Progress Reports for Cap-EFA from Angola, Bangladesh, Egypt, guinea, Morocco,
Niger, Pakistan, Senegal, Sierra Leone, Tanzania, Vietnam, July 2007 and January 2008.

Project Proposals for Cap-EFA 2006-2008 from Angola, Bangladesh, Egypt, guinea, Morocco,
Niger, Pakistan, Senegal, Sierra Leone, Tanzania, Vietnam

Report on completion of LIFERSS in Hoa Binh Province, DoET Hoa Binh, November 2007


Strategic Implementation Plan for Capacity Building for EFA within the Framework Agreement
on Programme Cooperation in the Field of Development Cooperation between the Government
of Norway and UNESCO


Policy and Strategy Papers

Capacity Development for Achieving EFA Goals: UNESCO Strategy, Review of UNESCO
Capacity Development Programmes, Final Review Report, UNESCO Education Sector,
February 2008

Common Country Assessment and UNDAF Guidelines for UN Country Teams 2007

Education and Training Sector Development Programme (2008-2017). URT 2007.

Education Development Strategic Plan 2001-2010, Socialist Republic of Vietnam

Education for All Global Action Plan: improving support to countries in achieving the EFA goals

Education Law of 2005, Socialist Republic of Vietnam

(The) Dakar Framework for Action - Education for All: Meeting Our Collective Commitments
Text adopted by the World Education Forum Dakar, Senegal, 26-28 April 2000.

(An) International strategy to operationalise the Dakar Framework for Action on Education for All
(EFA), April 2002.

Literacy Initiative for Empowerment LIFE 2006-2015, Vision and Strategy Paper (3rd edition),
UIL, September 2007.

The National Strategies for Growth and Reduction of Poverty (MKUKUTA 2005, MKUZA 2006)
Tanzania and Zanzibar.

(Draft) One UN Joint education workplan 2007-2008

(Draft) One UN M&E Framework and Success Criteria. December 2007

The Prime Minister Decision 112on approval the Proposal “Building learning society in the
period 2005-2010” Vietnam

Report by the Director General on the Action Plan for Improved Management of Extra-
budgetary Funds, 34 C/41, October 2007

Report by the Director General on the implementation of and follow up to the Dakar Framework
for Action, 164 EX/7, April 2002


                                                                                               55
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



Report by the Director-General on the follow-up to the EFA Strategic Review and UNESCO’s
Strategy for 2005-2015 (171 EX/8), March 2005

Report by the Director General on the Management of Extrabudgetary Resources and Activities,
175 EX/35, August 2006.

Report by the Director-General on the strategic review of UNESCO’s post-Dakar role in EFA
(170 EX/8), 2004

Report by the External Auditor on the Performance Audits undertaken in 2004-2005 biennium.
Draft action plan proposed by the Director-General on management of extra-budgetary funds
(174 EX/INF4 Rev.), March 2006

Report by the Joint Inspection Unit (JIU) on achieving the universal primary education goal of
the millennium declaration, 170 EX/35, 2004

(Draft) Study for Literacy Policy in Zanzibar March 2006

UNESCO Approved Programme and Budget 2006-2007 (33 C/5)

UNESCO Education Sector’s Extra-budgetary Strategy for 2008-2013, draft November 2007

UNESCO Medium-Term Strategy 2002 – 2007 (31 C/4) ”Contributing to peace and human
development in the era of globalisation through education, the sciences, culture and
communication.

UNESCO Medium-Term Strategy 2008-2013, 34 C/4, draft, 2007

UNESCO COUNTRY PROGRAMMING DOCUMENT (UCPD) for the Socialist Republic of VIET
NAM, 2nd edition

UNESCO’s Teacher Training Initiative for Sub-Saharan Africa (TTISSA) 2006-2015, Concept
Note

UNESS 2006-10 for Vietnam, draft #3

UN in Education, (draft) Joint Work Plan for 2008/09 Tanzania

Table on Delegated Authority and Accountability, Administrative Circular No. 2285, 18 February
2007

The Tanzania Development Vision 2025 (Mainland) and 2020 (Zanzibar)

Viet Nam EFA Assessment 2000

Viet Nam EFA Action Plan 2003-15

(Draft) Zanzibar Education Sector Development Programme (ZEDP) 2007-2015


Evaluations, studies etc.

Achieving the universal primary education goal of the Millennium Declaration. New challenges
for development cooperation, by Doris Bertrand, Joint Inspection Unit, 2003


                                                                                                 56
                     Evaluation of Capacity Building for Education for All (Cap-EFA)
                                              Final Report



(The) Capacity Building for EFA Programme of UNESCO, Technical Assistance on Dakar 2000
EFA Goals under Extra-budgetary Activities, Evaluation Report by Decide – APS, August 2004

Capacity Building to Enhance Policy Development, Institutional Change and Program Delivery
in Literacy and Non-Formal Education in Egypt, Mid-term evaluation of Cap-EFA in Egypt, by
Salah Sabri Sebeh and Fatma M. Mossallem, December 2007

Consolidated Findings of the Evaluations of UNESCO’s Major Programme I Institutes and
Centres, Nick Davis, IOS/EVS/PI/58, February 2006

Evaluation of UNESCO’s Response to HIV/AIDS, Kim Forss and Stein-Erik Kruse,
IOS/EVS/PI/48, April 2004

Evaluation of UNESCO Support to National Planning for EFA, Synthesis Report by Education
for Change, 2006.

Evaluation of the UNESCO Japanese Funds-in-Trust EFA arrangement, May 2007

Follow-up Survey on UNESCO Publication “Embracing Diversity: Toolkit for Creating Inclusive,
Learning-Friendly Environments” (ILFE Toolkit), UNESCO

Notes from Professional Upgrading of UNESCO Education Sector Personnel: Training Course
on Education Policy Analysis – A joint undertaking by ED/EO, ED/SFS and IIEP, 21 Jan to 1
Feb 2008

Reconstruction of the Education System in Angola: Improving the Quality of Teacher Education,
Mid-term Evaluation Report for Cap-EFA Angola, Jose P Castiano.

Report by the External Auditor on the Performance Audits undertaken in the 2004-2005
Biennium, 174 EX/27, March 2006

A Review of UNESCO’s Capacity-Building Function, Stiles Associates Inc. IOS/EVS/PI/71,
February 2007

Snapshot of FTI in Viet Nam, UNESCO

Strengthening Capacity of Managers (GO/NGOs) and Policymakers for Literacy and Life Skills,
Mid-term evaluation of Cap-EFA in Bangladesh, by Mahmudul Alam, December 2007.

Support to implementation of Education for All Mid Decade Assessment in Vietnam, Final
Report, UNESCO

467 GLO 1003.7, Sub-National EFA Assessment of Disparities and Exclusion implemented
within the Capacity Building for EFA Programme within the context of the EFA Mid-Decade
Assessment, Final Report

467GLO1003.19.1.1.Pacific EFA Mid-Decade Review, Final Report




                                                                                           57
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report



Annex 3 List of people interviewed

Mr. Kaviraj Appadu, Adviser, SIDA
Mr. Akio Arata, Chief, Section for Bilateral Government Funding Sources, Division for
Cooperation with Extrabudgetary Funding Sources, UNESCO
Mr. Ole Briseid, Deputy Permanent Delagete to UNESCO, Norway
Mr Gwang-Chol Chang, Section for Education Support Strategies, UNESCO
Ms. Jaya Conhye-Soobrayen, EO/ED, UNESCO
Ms. Ulrike Hanemann, UNESCO Institute for Lifelong Learning
Mr. Jussi Karakoski, Education Adviser, Ministry for Foreign Affairs of Finland
Mr. Arshad Saeed Khan, Senior National Specialist for Development Education, UNESCO,
Pakistan FO
Ms. Janne Kjaersgaard Perrier, EO/ED, UNESCO
Ms. Min Jeong Kim, Programme Specialist, EO/ED, UNESCO
Mr. Joseph Ngu, Director, IICBA
Dr Caroline Pontefract, Section for Teacher Education, Division of Higher Education, UNESCO
Mr. Svein Osttveit, EO/ED, UNESCO
Mr Mohamed Radi, Section for Education Support Strategies
Mr. Poul Erik Rasmussen, Senior Education Adviser, Ministry of Foreing Affairs, Denmark
Ms. Patricia Safi, ECR, UNESCO
Mr. Magnus Seamundsson, Adviser, SIDA
Mr. Lucio Sia Jr. Programme Specialist, Section for Teacher Education, Division of Higher
Education, UNESCO
Mr. Qian Tang, Director, Office of DADG and Executive Office, UNESCO
Ms. Mami Umayahara, Education Sector, UNESCO
Ms Ramya Vivekanandan, Section for Teacher Education, Division of Higher Education,
UNESCO

Tanzania:

UNESCO Office
Ms. Cecilia Barbieri, Education Programme Specialist
Ms. Mary Eyakuze. DPG/ED UN Secretariat
Ms. Cathleen Sekwao, NPO-Technical Backstopping
Mr. Anthony Ntileme, Literacy Coordinator
Dr. Criana Connal (NFE-MIS consultant)
Dr. Marc Bernal UIS Regional Advisor


Telephone conference with Dr. Jim Wile (IRA Washington – Mid-term reviewer and Director of
External Relations, IRA)
MoEVT
Dr. Kamisi Dihenga, Permanent Secretary
Mr. Oliver Mhaiki, Deputy Permenant Secretary
Ms. Rose Maasenga, Director, Teacher Education Department
Mr. Andrew Binde, Deputy Director
Mr. Richard Chediel, Programme Focal point
Mr. Enea Mhando, TTISSA Coordinator
Mr. Gnage, Head of Management Sector Adult Education, Coordinating UNESCO focal point
Ms. Philomena Mualiganu, Adult Education Specialist
Ms. Basilina Levira, Head of Academic Section and Monitoring

Tanzania Institute of Education (TIE)



                                                                                         58
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

Dr. Paul Mushi, Director
National Examinations Council of Tanzania (NECTA)
Dr. Joyce Ndalichako, Chief Executive Secretary
Zanzibar – MoEVT Office
Dr. Abdullah mid Mzee, Permanent Secretary, Ministry of Education and Vocational Training
Ms. Maryam A. Yussuf, Commissioner for Education, MoEVT
Mr Abdullah M. Abdullah, Commissioner for Policy and Planning and deputy Permanent
Secretary, MoEVT
Ms Khadija A. Moh’d Mbarak, Director, Department of Policy and Planning, MoEVT
Ms. Mwanaate Shaaban, Cap-EFA and TTISSA Focal Point, State University of Zanzibar
(SUZA)
Mr. Mussa Hassan, Cap-EFA Literacy co facilitator

Visit to Kiwengwa Primary School – meet with staff and observe demonstration teaching
session on Diagnostic teaching.

Sida office
Mr. Anders Frankenberg, Development Partner Group/ED Chair
Ms. Mary Eyakuze. DPG/ED UN Secretariat

UNICEF office
Dr. Jim Ackers, Chief, Basic Education and Life Skills
Mr. Joe Vere (NFE/MIS)

Meeting with Consultative Team of Education Stakeholders
Mr. Marietha Uri, Morogoro TC
Mr. John Sichone, Klerruu TC
Ms. Andrea Nchango, Klerruu TC
Mr. Anders Mutembei, MoEVT Inspectorate
Ms. Eliwhasayako Makundi, TIE
Ms. Agnes Mwiki, MoEVT Inspectorate
Ms. Phaorosia Makhanda, TIE
Suleman Sumra, Education Consultant and member of ESDP Sector Review


Vietnam

UNESCO Office:
Ms Vibeke Jensen - Head of Unesco Office
Mr Eisuke Tajima - CapEFA LPO, Unesco
Dung Doan - Assistant Programme Officer (gender)
Tuan Nguyen Dang - Assistant Programme Officer (CLC)
Ngan Nguyen Thi Thanh - Assistant Programme Officer (LAMP)
Dung Le Thi My - Temp adm support

CED MOET
Mr. Hoang Minh Luat – Director of CED, MOET (Chairman of NPT)
Mr. Nguyen Cong Hinh – Deputy Director of CED, MOET (Vice Chairman of NPT)
Ms. Nguyen Thi Thu Huyen – Deputy Director of CED, MOET
Ms. Thai Thi Xuan Dao – Director Non-formal Education Centre, National Institute for Education
Strategy and Curriculum Development (NIESAC) (Member of NPT)
Mr. Nguyen Cong Hinh – Deputy Director of CED, MOET (Vice Chairman of NPT)
Ms. Nguyen Thi Thu Huyen – Deputy Director of CED, MOET
Ms. Pham Thi Ngoc Hai - Official of CED, MOET
Mr. Nguyen Van Nghia – Official of CED, MOET


                                                                                            59
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

Mr. Phi Duc Nam – Official of Planning and Finance Department (PFD), MOET
Ms. Nguyen Thi Kieu Oanh – Official of PFD, MOET

UNICEF
Mr. Jonathan B. Miller - Chief, Education Section, UNICEF
Ms. Le Kieu Lan – Primary Education Sepicalist, UNICEF
Mr. Nguyen Quoc Binh – Education Policy Specialist, UNICEF

Ms Anouk van Neck, Team Leader, Social Sector, European Commission
Ms. Mai Thi Thanh - Senior Education Officer, The World Bank
Mr. Truong Quoc Can - Coordinator, Education and Reflect ActionAid Vietnam

Mr. Bui Ngoc Dao – Deputy Director (Chairman of Provincial Project Team), DoET, Hoa Binh
province
Mr. Nguyen Duc Hai – Head of Continuing Education Unit (Member of Provincial Project Team),
DoET, Hoa Binh province
Ms. Bui Le Thuong – Deputy Director (Member of Provincial Project Team), Continuing
Education Centre (CEC), Hoa Binh province
Ms. Nguyen Mai Hien – Member of Management Board, CLC of Dan Chu commune, Continuing
Education Centre (CEC), Hoa Binh province
Ms. Duong Thu Huyen – Member of Management Board, CLC of Yen Mong commune,
Continuing Education Centre (CEC), Hoa Binh province
Ms. Vi Thi Man - Member of Management Board, CLC of Yen Mong commune, Continuing
Education Centre (CEC), Hoa Binh province


Angola

Samuel Mondlane (project coordinator - Angola)
Justino Jeronimo (director of INFQ –Ministry of Education)
Eduardo Mulende (INFQ-assessor – Ministry of Education)
Maria Imaculada Curado (Director of the International Gabinet – Ministry of Education)
Silke Felton (Chief Education – UNICEF)
Nicolau Bubuzi, National Programme Officer – UNESCO)
Agueda Gomes (Research Assistant – UNESCO)




                                                                                         60
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report



Annex 4 LIFE Logframe


       Immediate                                             Expected results
       objectives
 To reinforce the        •   Partnerships developed with UN agencies to engage in joint planning with
 national and                government through a shared agenda for basic education.
 international           •   Comprehensive advocacy campaign for LIFE designed, funded and implemented
 commitment to               through newspapers, radio and television.
 literacy through        •   New partnerships to engage in the programme at national and international, public
 advocacy and                and private, traditional and non-traditional levels.
 communication           •   Political lobbying at country and international level encouraging commitment to LIFE
                         •   Additional funding for literacy at national and international levels
 To support the          •   Education policies and systems reviewed and monitored in selected countries.
 articulation of         •   Review of PRSP in particular to assess government commitment to EFA and
 policies for                literacy.
 sustainable literacy    •   National policies and programmes to integrate literacy into all appropriate
 within sector-wide          development activities: joined-up thinking.
 and national            •   Good practices in literacy policy analysed and disseminated to all target countries,
 development                 with realistic implementation strategies.
 frameworks              •   Mechanisms to support the participation of socially excluded groups within the
                             education system.
                         •   Cooperation with Government, UN, bilateral and multilateral agencies in joint
                             planning and shared agendas for collaboration
 To strengthen           •   Country assessment of needs, including capacity to manage and technical support
 national capacities         requirements.
 for programme           •   Scale and quality of literacy provision enhancing focus on women and girls.
 design,                 •   Appropriate programmes are designed and undertaken within targeted countries.
 management and          •   Human resources are adequately developed and allocated to deliver programmes in
 implementation              a sustainable fashion.
                         •   Programmes for multi-tasked teachers developed.

 To enhance              •   New projects formulated focusing on women and girls and implemented taking into
 countries’ innovative       consideration needs assessments, review findings and lessons learned from past
 initiatives and             practice.
 practices in            •   Non-formal education promoted through broad community based development
 providing literacy          programmes.
 learning                •   Civil society literacy and EFA networks supported and established at country level
 opportunities               to promote LIFE promoting South to South cooperation with literacy projects.
                         •   Partnerships with the private sector developed to undertake interventions within the
                             workplace.
                         •   Good practice identified and disseminated.




                                                                                                                    61
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report


Annex 5 TTISSA Logframe


               Output                                                    Activities
 Status and working conditions of       •   Support to UNESCO/ILO CEART to improve outputs and
 teachers improved                          effectiveness
                                        •   Advocacy of the Recommendations
                                        •   Advocacy through high profile events like World Teachers’ Day
                                        •   Enhanced reflection of teachers issues in key international
                                            reports and dialogue (with GMR)
                                        •   Review of teachers salaries and conditions in relation to that of
                                            other professions and to GDP and PPP
                                        •   Establishment/strengthening of UNITWIN/UNESCO chairs
                                            programmes
 Teacher management and                 •   Development of framework to support gvt in administrative and
 administration structures improved         professional management of teachers with review and
                                            dissemination of relevant materials
                                        •   Support to the development of Teacher Indicators in relation to
                                            teacher management and administration
 Appropriate teacher policies           •   Support gvts to review, develop and implement appropriate
 developed                                  teacher policies, including for non-formal education
                                        •   Strengthen quality assurance mechanisms for teacher education
                                        •   Support to the development of educational workplace policies
                                            with regarding HIV/AIDS (using UNESCO/ILO policy frameworks)
                                        •   Development of gender policy briefs with ILO, UNICEF, FAWE
                                            and OXFAM
 Quality and coherence of teacher       •   Development of a standardisation framework for Teacher
 professional development enhanced          Training Institutes
                                        •   Evaluation of the impact of teacher training on classroom
                                            practice and its implications for pupils and their learning
                                            outcomes in collaboration with UNESCO UNITWIN/Chairs
                                            programme
                                        •   Support appropriate and effective deployment of open and
                                            distance learning and ICTs
                                        •   Support to MoES in coordinating, evaluating and accrediting in-
                                            service training
                                        •   Development of guidelines to support transparent staff appraisal
                                            and performance review, consistent with the Recommendations
                                        •   Sourcing/development and dissemination of briefs, modules,
                                            learning materials to support teaching in non-traditional settings
                                        •   Development of ODL School Empowerment Programmes via
                                            blended learning




                                                                                                                 62
                        Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                 Final Report



Annex 6 Project proposal assessment criteria

               Capacity Building for EFA Programme 2006/2007
                                      Screening Format


Title and Reference Number of proposal: (to be entered by ED/EO/CTP)



Assessment committee member:

Project beneficiary country:

Responsible Field Office:

Lead Project Officer:

LDC country (YES/NO)         (To be entered by ED/EO/CTP)


For each project rate the following criteria from 1 (lowest) to 5 (highest)


I. Project in line with the country priorities and other development plans and
frameworks at the national level

II. National consultations took place and are documented

III. Project in line with the up-stream Capacity-Building for EFA Programme

IV. The down-stream activities are justified and necessary to reach the expected
goals

V. Project has clear achievable results and performance indicators for each activity

VI. Project proposal shows logical linkages between goals, objectives, expected
results and activities against inputs (logical framework matrix)

VII. The budget is set realistically in relation to proposed activities

VIII. Project elaborated in co-operation with other Regional Bureaus, Institutes or
HQ Divisions

IX. The National Office has the required Monitoring and Implementation Capacity

X. The relevant and required capacities of other UNESCO units responsible for
implementing specific activities are documented in the Implementation strategy
of the project

XI. Measures are proposed for sustainability beyond current project cycle

XII. Issues related to HIV/AIDS and Gender are mainstreamed

TOTAL

Comments (please note any element that requires clarification):


                                                                                          63
                    Evaluation of Capacity Building for Education for All (Cap-EFA)
                                             Final Report


Annex 7 Guidance Note to Mid-term evaluations
                                                                                      10 October 2007
  Mid-term evaluations
  CapEFA country programmes 2006-2007
  GUIDANCE NOTE:
  - Some inputs to address common issues

  All CapEFA country programmes’ mid-term evaluations need to be carried out in
  November 2007 to be available by mid December 2007. Respecting this timing is
  important, because the mid-term evaluation will serve several purposes, namely to:

     1. Assess the results of activities undertaken at the mid-term point of implementation
        to improve effectiveness of the Programme and take corrective action as
        necessary for the remaining period of the current phase;
     2. Formulate recommendations in literacy/non-formal education/teacher training for
        further partnership-building and mobilisation at country level;
     3. Provide a basis for the preparation of the CapEFA Programme (2008-2009) in the
        country concerned, if relevant;
     4. Ensure that the findings feed into the next CapEFA progress report, due for
        submission to HQ-ED/EO towards the end of 2007 (19 Dec 2007 – to be
        confirmed);
     5. Inform the external evaluation of the CapEFA Programme, contracted by
        UNESCO’s Internal Oversight Service (IOS) and planned to start in November
        2007.

  The Lead Project Officers are responsible for coordinating the mid-term evaluation for
  their specific CapEFA country (preparation of TOR; contracting a consultant; respecting
  deadlines; etc). While this exercise needs to be designed as deemed most productive for
  the further execution of activities, LPOs are kindly requested to ensure that the common
  issues outlined below be included in the TOR and hence addressed by the evaluation.

  The LPO should share the TOR with HQ-ED/EO prior to the exercise and by 22 October
  2007. Some offices have already done so; others have not planned for it yet. These are
  kindly requested to give it priority and make a special effort so that the above common
  objectives can be fulfilled.

  In the few cases, where Work Plans do not specifically foresee funds for this exercise,
  LPOs should accommodate it under general monitoring and evaluation costs, or
  otherwise within the current programme budget through a revision. The costs for the mid-
  term evaluation vary from country to country and depend on the methodology adopted,
  but as an indication the budget should not exceed 5,000 USD.




                                                                                                  64
                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                     Final Report



     Common issues, approaches and questions
     The following issues and questions should guide, but not limit, the preparation of the TOR
     for the mid-term evaluation. The issues raised emerge from information, reporting and
     management needs and from fora and ongoing discussions with the supporting donors.
     Some are therefore similar to the last report format. Questions marked in italic are drawn
     directly from the TOR prepared for the IOS evaluation of the CapEFA Programme.

     Basic principles and management:
     1. The mid-term evaluation should set out to assess the benchmarks reached so far. The
        focus should be on actual outcomes; and NOT on inputs such as workshops, events, or
        other outputs, but rather on the effect that these inputs have had (=outcomes).
     2. The mid-term evaluation should focus on drawing lessons learnt in substance and
        management and on that basis formulate recommendations for the remaining period of
        implementation; and for UNESCO’s actions in the relevant area of intervention in a
        medium term perspective.
     3. The evaluation should contextualize the actions of the Programme within the broader
        development context in the country; such as national education sector plans and policies
        and assistance frameworks of development partners.
     4. The mid-term evaluation should refer to the project document and work plans and
        assess whether these are effective as working tools (clear goals and indicators,
        distribution of labour) and whether they are in alignment with the reality and needs.

     Assessing the character of the capacity-development processes:
     5. The assessment of what the Programme has achieved so far in capacity-building and
        up-stream activities should be examined in the following three dimensions:
        • Human resource development: the process of equipping individuals with the
           understanding, skills and access to information and training that enables them to
           performs effectively
        • Organizational development: the elaboration of management structures, processes
           and procedures, not only within organizations but also the management of
           relationships between the different organizations and sectors (public, private and
           community)
        •  Institutional and legal framework development: making legal regulatory changes to
           enable organizations, institutions and agencies at all levels, and in all sectors, to
           enhance capacities”
                                                                                               (IIEP, Ed June Seminar 2007)

     Assessing the use of UNESCO’s comparative advantages:
     The analysis should also focus on the following issues:
     6. How and to what extent is UNESCO’s normative role at the global level being translated
        into actions at country level through the CapEFA Programme?
     7. To what extent is the CapEFA Programme applying inter-sectorial approaches? - The
        assessment of what the Programme has achieved in cross-cutting issues, such as HIV
        and AIDS, gender and ICTs should be examined and highlighted if relevant.
     8. What has so far been the effect of the “UNESCO-family-approach”77 in the context of
        providing technical assistance to the country?



77
    Note: The ‘UNESCO-family approach’ refers to an internal implementation mechanism that seeks to streamline and coordinate
interventions involving various UNESCO entities. Field Offices are primarily responsible for the implementation of all operations
and request services from Institutes, Regional Bureaux and Headquarters according to the kind of technical support that a
specific country needs to advance its development objectives. This approach seeks to put a large pool of UNESCO competencies
at the disposal of Member States in order to make the Organization more effective at country level, and ensure quality delivery.




                                                                                                                             65
                  Evaluation of Capacity Building for Education for All (Cap-EFA)
                                           Final Report

9. To what extent is the Programme otherwise drawing on UNESCO’s comparative
    advantages in its implementation? (eg: its five functions: UNESCO as a coordinator;
    standard-setter; capacity builder; a clearing house; and a catalyst for cooperation).
10. How is the area, in which the Programme currently intervenes, reflected in the UNESCO
    National Education Sector Strategy - UNESS (chap 5)?

Partners and sustainability:
11. How is the Programme working with other UN agencies, bi-lateral agencies and other
    development partners at country level? To what extent has such collaboration been
    effective? Should it be better explored and how?
12. How is the Programme working with national stakeholders to achieve its objectives? To
    what extent has such collaboration been effective? Should it be better explored and
    how?
13. To what extent has CapEFA supported pilot projects been instrumental in scaling up and
    developing national policy and programmes?
14. Are the costs of the Programme justified by its results?
15. Will the supposed benefits of the CapEFA Programme last beyond completion of extra-
    budgetary support?
16. To what extent, if at all possible, could current major developments at country level be
    attributed to the Programme?


Methodology:
   • It is recommended that the mid-term evaluation be participatory (interviews; focus
      groups; questionnaire) and that it includes a desk-study of relevant programme
      documents (HQ/FO/Ministry level docs);
   • The scope of the report should not exceed 15-20 pages, but can have annexes. It
      should have a 1 page executive summary with the main conclusions of the
      evaluation exercise. Language: English or French.
   • The mid-term evaluation should be undertaken preferably by an independent
      educational analyst with good knowledge of UNESCO and the education system in
      the country concerned. It could also be lead by the Regional Bureaux/HQ/Institutes
      in case this has been foreseen in the WP, and only if the exercise can be done in the
      required timeframe under this modality.


Checklist:
Please ensure that the evaluator uses the following checklist to ensure that all common
issues have been addressed in the mid-term evaluation. The list should be included in the
evaluation report:




                                                                                            66
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report



 Checklist:
(to be filled in by the evaluator based on the          Very                                                        Very
conclusion of his/her report)                       unsatisfactory         Unsatisfactory   Satisfactory        satisfactory
1. Expected results reached so far
towards major benchmarks
2. Project document and work plan as
effective tools for coordination and
implementation
3. Use of UNESCO’s comparative
advantages

4. Added value of technical
assistance through the ‘UNESCO-
family-approach’
5. Programme being instrumental in
developing policies and programmes
6. Programme being participatory and
inclusive in its approaches
7. Cross-cutting issue addressed: HIV
and AIDS

8. Cross-cutting issue addressed:
Gender

9. Cross-cutting issue addressed:
ICTs

10. Programme generating effective
collaboration with UN partners
11. Programme generating effective
collaboration with bi-lateral partners
                                                                     YES                                   NO
12. Does the programme activities
need to readjust during the remaining
period of implementation?
13. Does the work plan need to
undergo any major revision for the
remaining period?
14. Are the costs of the Programme
justified by the results?
15. Will the supposed benefits of the
Programme last beyond completion
of extra-budgetary support?
16. Should UNESCO continue to
focus on this area of intervention?
Comments:




                                                                                                                     67
                                                Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                                         Final Report




Annex 8 Cap-EFA budget per country and expenditure component 2006 – 2008
                             missions
                             and
                             national    international admin.     fee                                                  change    change
                             consultants consultants support      contracts training    equipment misc.      TOTAL     in USD    in %
Pakistan              2006        55,020       51,600    15,000     593,000   131,000      20,000   14,380     880,000
                      2008        71,020       72,600    15,000     603,000   183,000      20,000   15,380     980,000 100,000        11.4
Bangladesh            2006        42,800       76,155    10,000     721,260                  1,000  12,360     863,575
                      2008      139,500        22,380    16,600     750,235     14,500       4,000   6,360     953,575    90,000      10.4
Vietnam               2006        57,950       31,700    10,000     740,000     15,000     10,000   18,000     882,650
                      2008        81,450       55,700    16,000     800,500     23,000     14,000   21,500 1,012,150 129,500          14.7
Egypt                 2006        87,840      164,160    55,000     276,000   225,000      65,000    7,000     880,000
                      2008      118,300       152,460    48,400     250,880   226,000      67,200    6,160     869,400 -10,600        -1.2
Morocco               2006        32,177      195,674               606,000                13,000    8,500     855,351
                      2008        75,267      265,550    10,624     572,393     50,017     13,000    8,500     934,710    79,359       9.3
Senegal               2006        50,504      141,496     8,000     186,000   431,900      42,500   19,600     880,000
                      2008      178,556        33,866     8,000     604,688     56,190     56,500   12,600     950,400    70,400       8.0
Guinea                2006        67,337      101,698    30,000     550,883               108,220   21,862     880,000
                      2008        76,201      110,684    33,500     580,210     27,000     34,668   32,737     895,000    15,000       1.7
Sierra Leone          2006        50,000       30,000     8,000     210,000   400,000      40,000   22,000     760,000
                      2008        55,000       30,000     8,000     590,000     50,000     40,000   22,000     795,000    35,000       4.6
Niger                 2006        37,968      119,632    20,000     203,150   431,000      22,350   24,000     858,100
                      2008        70,763      146,575    15,000     141,613   330,750      16,763   18,000     739,463 -118,637      -13.8
Tanzania              2006        42,720      109,228    17,000     675,102                10,000   15,000     869,050
                      2008        94,089       13,228 123,500       689,972        114     18,260    4,887     944,050    75,000       8.6
Angola                2006        92,561      148,042    28,286     344,115   113,500      93,220   40,780     860,504
                      2008      124,862       181,042    25,000     338,376   148,224      97,220   35,780     950,504    90,000      10.5
TOTAL LIFE and        2006      616,877     1,169,385 201,286     5,105,510 1,747,400     425,290 203,482 9,469,230
TTISSA                2008    1,085,008     1,084,085 319,624     5,921,867 1,108,795     381,611 183,904 10,024,252 555,022           5.9
Increase in USD                 468,131       -85,300 118,338       816,357 -638,605      -43,680 -19,578      555,022
Increase in %                       75.9          -7.3     58.8         16.0      -36.5      -10.3      -9.6       5.9
Carry-over projects   2006      105,137        59,388    34,754     265,753   173,402     100,317   14,955     725,606

                                                                                                                                             68
                                            Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                                     Final Report




                        2008    37,445    55,742     28,624     478,395       80,341        39,618            710 720,876
Regional and            2006                   0                             550,000                              550,000
institutional support   2008    51,865   109,764                     500      78,750                     4,000    244,879
                        2006                                                 750,000                              750,000
UNESS                   2008   133,049    98,534     27,032     197,480      227,120        43,000      23,785    750,000
                        2006                                                                                    2,025,606
TOTAL OTHERS            2008                                                                                    1,715,755
                        2006                                                                                   11,494,836
TOTAL CAPEFA            2008                                                                                   11,740,007




                                                                                                                            69
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report


Annex 9 Cap-EFA cooperating partners and co-funders

Cap-EFA cooperating partners in the project proposals
                Ministry,              I/NGOs,                UN                multilateral      bilaterals
                central                universities,          organisations     donors
                administration,        regional
                policy makers          administration
  Angola                X                     X
  Bangladesh            X                     X
  Egypt                 X                     X
  Guinea                X                                                       Wold Bank         USAID
  Morocco               X                     X               UNICEF                              USAID
  Niger                 X                     X               UNICEF,
                                                              UNDP
  Pakistan               X                    X               UNDP,                               JICA, USAID
                                                              UNICEF, WFP
  Senegal                X                    X               FAO
  Sierra                 X                    X               UNDP,
  Leone                                                       UNICEF
  Tanzania               X                    X               UNICEF            EU                SIDA
  Vietnam                X                    X               UNDP,                               not specified
                                                              UNICEF, ILO,
                                                              WHO, FAO,
                                                              UNIDO,
                                                              UNAIDS,
                                                              UNODC, IOM

(Potential) co-funders as identified in the project proposals
               Ministry,           I/NGOs,              UNESC        UN           multilater   bilaterals
               central             universities,        O            organisa     al donors
               administrati        regional                          tions
               on, policy          administrati
               makers              on
  Angola                                                UIS                                    Japan, US
                                                        Bangkok
  Banglades
  h
  Egypt                                                 project                                USAID
  Guinea
  Morocco
  Niger                            (ACDI, AIF)                       (UNICEF
                                                                     )
  Pakistan                         (ESRA)                                                      (JICA)
                                                                                               (USAID)
  Senegal      in kind                                                                         France
                                                                                               (though the
                                                                                               ministry)
  Sierra
  Leone
  Tanzania     in kind             IRA
  Vietnam                                               (UNICE                                 (JICA)
                                                        F)                                     (NZAID)

Cap-EFA (potential) cooperating partners as per March 2008
                Ministry,             I/NGOs,                 UN                multilateral      bilaterals
                central               universities,           organisations     donors
                administration,       regional
                policy makers         administration
  Angola                X                    X                UNICEF,           EC                Spain, Italy
                                                              UNHCR, Joint
                                                              UN
  Bangladesh             X                    X               UNDP,                               SIDA, CIDA
                                                              UNICEF,
                                                              WFP, Joint UN



                                                                                                                 70
           Evaluation of Capacity Building for Education for All (Cap-EFA)
                                    Final Report

Egypt      X                   X             UNDP                             (USAID)
                                                                              (JICA)
Guinea     X                                 UNICEF              Wold Bank    USAID (JICA),
                                                                              GTZ, France
Morocco    X                   X             UNICEF, Joint       EC           USAID, Spain
                                             UN
Niger      X                   X             UNDP
Pakistan   X                   X             (UNFPA)             (EC)         JICA, USAID
                                             (ILO), (WFP),                    (Spain)
                                             Joint UN
Senegal    X                   X             WFP, (ILO)          World Bank   (JICA) (CIDA)
                                             (UNICEF)                         (Luxembourg)
                                                                              (China)
Sierra     X                   X             UNDP,
Leone                                        UNICEF
                                             UNHCR
Tanzania   X                   X             UNICEF, Joint
                                             UN
Vietnam    X                   X             UNDP,               World Bank   (JICA) (Spain)
                                             UNICEF,
                                             UNFPA, ILO,
                                             WHO,
                                             UNAIDS, Joint
                                             UN




                                                                                         71
                       Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                Final Report


Annex 10: Summary of key findings, conclusions and recommendations
           Findings                             Conclusions                         Recommendations
 A number of challenges have          Cap-EFA has been a                      Capacity Building for EFA-
 been identified limiting             deliberate and successful               programme should be
 UNESCO to effectively deliver        response to most of the                 continued. With some
 on its mandate                       identified challenges.                  modifications and strategic
                                                                              reorientation, it can play even
                                                                              bigger role in enhancing
                                                                              UNESCO’s mandated service
                                                                              delivery to the Member States

 UNESCO has limited financial         Cap-EFA has had a major role            Cap-EFA support should be
 resources to deliver on its EFA      in the implementation of                aligned with UNESCO strategy
 mandate.                             UNESCO’s education                      (C/4) and programme and
                                      programme. The available                budget (C/5) cycles.
                                      USD 11 million over two years           Coherence to UNESCO
                                      is a substantial amount of              strategy and programme and
                                      additional financing in                 budget needs to be ensured,
                                      UNESCO’s education budget.              but the MOU would benefit
                                                                              from existing best practises in
 Extra-budgetary funding has          The Memorandum of                       donor harmonisation and
 not always been fully aligned        Understanding between                   include more definition of e.g.
 and coherent with UNESCO             UNESCO and the Nordic                   programme objectives and
 regular programme.                   Countries (and Switzerland)             scope and management
                                      has allowed integrated                  procedures.
                                      planning and programming
                                      and full coherence of regular
                                      budget and extra-budgetary
                                      funds.
 UNESCO activities have               Cap-EFA prioritised                     The Cap-EFA support should
 suffered from unclear focus          thematically literacy and               be based on 1) needs in the
 and limited prioritisation           teacher training. Prioritisation        MS with priority for a) LDCs in
                                      of LDCs and upstream policy             selection of countries and b)
                                      advice less evident.                    needs for policy in selection of
                                                                              themes, and 2) UNESCO
                                                                              capacity deliver prioritising a)
                                                                              countries with UNESCO FO
                                                                              and b) themes where
                                                                              UNESCO has proven
                                                                              expertise and clearly identified
                                                                              experts.

                                                                              Upstream policy advice should
                                                                              be further accentuated.
 Professional capacities and          Cap-EFA added 683 person                The objective of the
 technical expertise especially       months of human resources,              programme should be geared
 in Field Offices need                mainly in the FOs. Not certain          towards strengthening
 strengthening                        if increased professional               UNESCO capacity to deliver
                                      capacities for policy advice            on its mandate and to better
                                                                              serve the Member States in
                                                                              Education for All.
 UNESCO internal cooperation          Through family approach Cap-            There is a need to identify
 and coordination is found            EFA has strengthened service            available expertise in
 weak.                                provision to the MS from within         UNESCO and to ensure that
                                      UNESCO.                                 the identified experts
                                                                              participate in planning and
                                                                              monitoring of the activities.

                                                                              Cap-EFA should build on



                                                                                                           72
                      Evaluation of Capacity Building for Education for All (Cap-EFA)
                                               Final Report

                                                                             already existing expertise in
                                                                             UNESCO. There should be a
                                                                             mechanism to ensure that
                                                                             utilisation of external experts
                                                                             strengthens also UNESCO’s
                                                                             capacity.
There are no standard                Cap-EFA has not addressed.              Cap-EFA programme
definitions for key concepts.                                                document should define
                                                                             UNESCO’s comparative
                                                                             advantage and added-value.
                                                                             Cap-EFA should cooperate
                                                                             with the on-going work to
                                                                             produce UNESCO policy for
                                                                             capacity development.
Overall implementation of            More attention should be paid           More rigorous appraisal (or ex-
Cap-EFA and projects in 11           to ensure relevance,                    ante evaluation) process
countries is found successful.       effectiveness, efficiency and           should be applied in Cap-EFA.
                                     sustainability of the activities.




                                                                                                         73
                           Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                    Final Report



  Annex 11: Management Response Table

              Recommendations                                      Actions taken/to be taken by the
                                                                       Director-General, and/or
                                                              If recommendation is not accepted, give
                                                                        reasons for rejection
Relevance
Recommendation 1: The programme should be                A new Programme document 2008-2009 has been
continued. With some modifications and                   devised in the light of the Evaluation, and a draft
strategic re-orientation, it can play an even            MOU 2008 – 2013 has been prepared jointly with
bigger role in enhancing UNESCO’s mandated               the supporting donors. Further, RP alignment is
service delivery to Member States. Cap-EFA               strengthened through the ED specific strategy
support should continue to be aligned with the           under the UNESCO Additional Programme of
C/4 and C/5 cycles.                                      Targeted/Projected Extra Budgetary Activities

Recommendation 2: The Cap-EFA should be                  This will be ensured partly through UNESS and it is
based on country needs and national education            included as part of the assessment criteria of new
plans, be part of the joint efforts at country level     interventions, The Programme will aim to move to
and invokes ownership. Priority should be                a 100% LDC focus, while previously benefiting
given to LDCs in the selection of countries and          non-LDCs can benefit in the current biennium only
to policy requirements in the way themes are             to phase out; extract lessons learnt for capacity-
selected. Cap-EFA should be based on                     development and ensure UNESCO’s upstream
UNESCO’s capacity to deliver.                            policy role.
Recommendation 3: Aligned with the 34 C/5,
Cap-EFA should further accentuate upstream               The upstream focus is emphasized and clarified
policy      advice.      Downstream          project     as a working definition in the revised Programme
implementation should be allowed only in                 document, in which the balance of up-, mid, and
exceptional circumstances with clear added               downstream is specified. If an activity is
value and feed-back mechanism to support                 downstream then it is to feed upstream.
UNESCO’s policy advisory role.

Results, efficiency and effectiveness
Recommendation 4: A more rigorous ex ante                Ex ante appraisal is being strengthened through
appraisal of the programme logic, options for            UNESS which will be instrumental on a country
design and implementation and expected                   basis.
impacts should be conducted in future.

Recommendation 5: The programmatic nature                The revised Programme document focuses the
of the Cap-EFA programme document should                 programme around five capacity development
be strengthened with clear identification of             priorities as the framework for design,
challenges to be addressed, followed by                  implementation and monitoring of interventions. A
detailed definition of the programme                     results-matrix maps outcomes at programme,
intervention logic and indicators against which          theme and country level with performance
progress can be measured. There must be                  indicators at each level, presenting the CapEFA
clear goals and objectives that permit easily            hierarchy of objectives.
reportable results.
Recommendation 6: The Cap-EFA document                   The revised Programme maps UNESCO delivery
should define UNESCO’s comparative                       capacity for Cap/EFA showing sector expertise and
advantage/added-value with clear identification          services available across HQ, institutes and
of actual service providers in UNESCO. The               offices/bureaux. Internal co-ordination is also being
identified experts should participate closely in         improved by increased consultation in the
the preparation and in monitoring of the support         development of the programme
funded by Cap-EFA to ensure smooth delivery              document.Knowledge-generation for capacity-


                                                                                                        74
                         Evaluation of Capacity Building for Education for All (Cap-EFA)
                                                  Final Report

of expertise from within UNESCO. A                     development is consolidated and further, the family
mechanism is needed to ensure that external            approach will encompass up to three entities to
expertise strengthens also UNESCO’s                    improve co-ordination/ownership at country level.
capacity.
Sustainability
Recommendation 7: Underline the importance             The revised programme document makes clear
of fully using the One-UN processes                    that national ownership, alignment with existing
developing UNESCO support to the education             development frameworks and synergistic
sector locally, and of other existing cooperation      partnerships are both pre-requisites for
mechanisms set up by the local education               interventions and subject to monitoring and
group. It seems logical to move toward a               evaluation.
programme approach in order to ensure
sustainability and greater impact.




                                                                                                     75

								
To top