Docstoc

repen1

Document Sample
repen1 Powered By Docstoc
					      ASSOCIATION FOR SOCIAL INVESTIGATIONS AND APPLIED RESEARCH PRACTICES

1113 Sofia                                                  Tel.: (+359 2) 971 10 02, ext. 346, 347
125 Tzarigradsko shosse                                                     Fax: (+ 359 2) 706 260
bl. 4, fl. 4                                                                       www.asiarp.org




Project:                  Monitoring and Evaluation of
                          Education Modernization Project Activities

Financed by:              Open Society Foundation – Sofia
                          (Contract:    № 198/01.07.2002)




           INTERMEDIARY REPORT




         Analysis and Evaluation of Implemented Activities
            under the Education Modernization Project
                     (for the period January 2001 – September 2002)

                                       Recommendations




                                               Sofia
                                         October 15, 2002
                                                             CONTENTS


Project: Monitoring and Evaluation of Education Modernization Project Activities .............. 5

Project: Education Modernization .............................................................................................. 6

Analysis and Evaluation of Activities under the Individual Components ........................... 10

        G e n e r a l E d u c a t i o n .......................................................................................... 10
        Component National education standards, standards-based curriculum,
                      students assessment and inspection system ................................................ 10
        Component In-service teacher training .......................................................................... 14
        Component Extension of the Delegated Budget System................................................. 15
        Component Optimizing the School Network .................................................................. 18
        Component Education Management Information System ............................................. 20

        H i g h e r E d u c a t i o n ............................................................................................. 23
        Component Improving the efficiency and effectiveness in
                    allocating resources to higher education institutions ............................... 23
        Component Retaining accessibility to higher education institutions ............................. 25
        Component Improving the quality of teaching and the internal
                    management of higher education institutions ............................................ 27

        P r o j e c t M a n a g e m e n t ...................................................................................... 30
        MES capacity for project management ......................................................................... 30
        Communication strategy of the project ......................................................................... 38
        Human resources and institutional memory ................................................................ 40

Key conclusions ......................................................................................................................... 42

Problems in implementing the project and their structural background ............................ 45


Recommendations ..................................................................................................................... 49


Summary of the key conclusions and recommendations of the report ................................ 52

Appendices:
    Appendix 1: Matrix for monitoring and evaluating
               Education Modernization Project activities
    Appendix 2: Questionnaire for depth interviews with experts
    Appendix 3: Component Structure of the Education Modernization Project
    Appendix 4: Translation analysis




                                                                                                                                               2
   Abbreviations used in the text:




    CE         Content of Education
    CC         Curriculum Council
    DB         Delegated Budgets
   EMP         Education Modernization Project
    GE         General Education
    GEI        General Education Institution
 GERMU         General Education Resource Management Unit
    HE         Higher education
HECTMS         Higher Education Competitive Teaching and
               Management System
   HEC         Higher Education Council
   HEI         Higher Education Institution
     IS        Information System
     IE        Inspectorate on Education
   MES         Ministry of Education and Science
  MLSP         Ministry of Labor and Social Policy
    MC         Municipal Council
   NES         National Education Standards
 NGEEU         National General Education Evaluation Unit
   PAD         Project Appraisal Document
   PCU         Project Coordination Unit
    PB         Project Board
   RAC         Resource Allocation Committee
   SNO         School Network Optimization
    WB         World Bank




                                                            3
The Education Modernization Project is
being financed by a World Bank loan. The loan
agreement was signed between the
Government of the Republic of Bulgaria and
the World Bank on September 19, 2000, was
ratified by the National Assembly on
December 12, 2000, and entered into force on
January 10, 2001. The loan amounts to a total
of 62 million US dollars and is an Adaptable
Program Loan. The implementation period is
nine years, divided into three three-year
phases.




                                                The Project Monitoring and Evaluation of
                                                Education Modernization Project Activities is
                                                being financed by the Open Society
                                                Foundation – Sofia, and is being implemented
                                                by the Association for Social Investigations
                                                and Applied Research Practices and the
                                                subcontractor, Previs Consult Agency,
                                                responsible for the financial analysis of the
                                                project. The implementation period is 21
                                                months, starting July 1, 2002; each phase of
                                                the monitoring will be accounted with reports
                                                issued every three months.
                                                Project team: Prof. Doctor of Social Sciences
                                                Georgi Dimitrov (team leader), Associate Prof.
                                                Dr. Petia Kabakchieva (deputy team leader),
                                                Senior Assistant Prof. Dr. Valentina
                                                Milenkova, Prof. Doctor of Philosophy Georgi
                                                Fotev, Assistant Prof. Lyudmila Todorova,
                                                Milena Harizanova, Ivo Hristov, Svetlana
                                                Georgieva, Diana Nenkova.



                                                                                             4
                          Project: Monitoring and Evaluation of
                         Education Modernization Project Activities
                       Implementation period: July 10, 2002 – September 30, 2002.




      Goal:
      To evaluate the relevance between the goals set in the Education Modernization Project and
      the actual results achieved at the ministerial level in the period since the start of the project,
      January 10, 2001–September 30, 2002. All activities included in the MES schedule have been
      monitored.

 Tasks:
      Description, analysis, and evaluation of implemented activities under the Education
      Modernization Project:
      - Evaluation of the institutional conditions for implementing project activities;
      - Analysis and evaluation of the level of implementation of project goals:
          Institutional resources for implementing the project at the MES level;
          Project management;
          Human resources;
          Information resources;
          Funds allocation;
          Public awareness of the project.
      - Recommendations for optimizing activities.

 Methods applied:
         -   Depth interviews with:
              MES leadership;
              PCU members;
              Component leaders;
              Working group members;
              WB representatives.
         -   Documentation systematization and analysis:
              Statutory project documents: Project Appraisal Document, MES Education
                Modernization Project, Project Management Guide, Project Implementation Plan,
                Higher Education Development Strategy;
              Statutory regulations: of MES, PCU, Competitive Center for Higher Education
                Teaching and Management;
              Reports: issued by PCU, WB Memos, internal reports;
              Correspondence: of PCU (out-going and in-coming);
              Tender documents: World Bank Procurement Guide, Procurement Plan;
              Tender Terms of Reference.



    The financial analysis is provided by the Previs-Consult Agency.
                                                                                                      5
        -     Expert Evaluations;
        -     Interviews:
               with social sciences teachers from Sofia, Plovdiv, and Bourgas;
               with representatives of Regional Inspectorates on Education in Sofia, Bourgas,
                 Lovech, Stara Zagora.
        -     Monitoring included in PCU.

 Achieved results:
           Matrix for systematizing monitoring and evaluation information (Appendix 1)
           Questionnaire for depth interviews on the implementation of the project (Appendix 2)
           Inventory and Systematization of the EMP-related PCU Archive (approximately 200
            pages, floppy disc attached)
           EMP Activity Report for the period January 2001–August 2002.

   The team would like to thank the following individuals for their assistance and
    responsiveness:
     The team of the Open Society Foundation - Sofia: Dr. Stefan Popov, Associate Prof.
        Rayna Gavrilova, Ms. Neda Kristanova;
     The MES leadership: Minister, Associate Prof. Vladimir Atanassov, Deputy Minister,
        Prof. Anastas Gerdjikov, Deputy Minister, Associate Prof. Igor Damianov, Mr. Yulian
        Nakov, Mr. Dimiter Mitev;
     The WB team: Mr. Ernesto Cuadra, Yael Duthilleul, Mrs. Boriana Gocheva;
     The former MES leadership: Minister, Associate Prof. Vesselin Metodiev, Deputy
        Minister, Associate Prof. Anna-Maria Totomanova, Deputy Minister, Prof. Borislav
        Toshev, Mr. Ventzislav Panchev and Mr. Kalin Limberov;
     The expert who agreed to give interviews;
     All individuals and institutions that supported our work.




                             Education Modernization Project
   Overall goal:
           To shift from expense-based resource allocation to an investment-based funding
            mechanism that would imply return on invested funds. The purpose of modernization is
            to make Bulgarian education system effective, and above all, flexible – capable of
            adapting to and accommodating the changing social environment.
           The essential feature of the EMP is that it is designed to create structural mechanisms,
            which would help build an education system of new quality. Therefore, the congruity
            between the separate project activities is a key prerequisite for its success. In order to
            prepare future students for the requirements of a competitive higher education system,
            open and responsive to their professional orientations, the school should produce active
            and independent personalities, rather than just carriers of knowledge with a
            passive/receptive attitude to life. This implies not only changes in the content and
            organization of school education, but also reforms in teacher training. That, in turn,
            requires financial resources, which would be secured by improving school management
            through an extension of the delegated budget system and an optimization of the school
            network. Under the new conditions the inspectorates would have a completely new role.
            This efficiency-oriented principle in spending, and above all, managing budget
                                                                                                    6
       allocations on education would be applied in higher education, too, by introducing a
       funding mechanism based on a normative cost formula, rather than on historical patterns.
       The project proposes a mechanism that would prepare HEIs for a competitive academic
       environment by 1) creating internal institutional mechanisms for quality control, and
       responsible and transparent management; 2) encouraging amalgamation of academic
       institutions in order to improve resource management efficiency, as well as introducing a
       mechanism for involving students in the process of funding higher education, which
       would increase demands for a high-quality education.

   The project will be implemented in three phases. Phase I started in January 2001 and would
   continue until the beginning of 2004. It amounts to a total of 14,39 million US dollars.

 Phase One Objectives:
      changing the institutions (norms and regulations);
      establishing the structures (directorates, centers, boards, steering committees,
       independent institutes);
      developing the technical capacity (management information system, methodology for
       optimization of the network of schools and HE institutions, and student assessment);

 Tasks:
   1) successfully introducing new resource allocation mechanisms, and new standards-based
      curriculum, and programs of study;
   2) changing management practices;
   3) changing teaching approaches;
   4) establishing a demand-driven teacher in-service training systems;
   5) establishing a new system of education monitoring and evaluation;
   6) establishing a new student loan and stipend system.


 MES Management Teams:
   The Education Modernization Project was prepared by international experts and a MES
   management team including:
    Vesselin Metodiev – Minister of Education and Science;
    Anna-Maria Totomanova – Deputy Minister, Higher Education Sector;
    Ventzislav Panchev – Deputy Minister, General Education Sector;
    Pencho Nenov – Chief Secretary.


   The loan agreement was signed by a MES management team including:
    Dimiter Dimitrov – Minister of Education and Science;
    Borislav Toshev – Deputy Minister, Higher Education Sector;
    Krassimir Nikolov – Deputy Minister, General Education Sector;
    Hristo Katzarov – Chief Secretary.


  After the parliamentary elections in July 2001 work on the project continued with a MES
management team including:
   Vladimir Atanassov – Minister of Education and Science;
   Igor Damianov – Deputy Minister, Science Sector, responsible for the implementation of
      the project;
                                                                                               7
          Anastas Gerdjikov – Deputy Minister, Higher Education Sector;
          Yulian Nakov – Deputy Minister, General Education Sector;
          Kalin Limberov, Dimiter Mitev – Chief Secretaries.

 Component Structure
The project is developed in a series of documents and annexes, the key of which are the Project
Appraisal Document of August 7, 2000 (PAD) and the Education Modernization Project of 2001
(EMP). The project components are presented in a different way in these documents – the
objectives of the reform are listed with Roman numerals as they have been formulated in the first
document, while in the second document these objectives have been operationalized into
components, listed with Arabic numerals. (Appendix 3).*

І.        Creating conditions for improving the quality of teaching and learning in general
          education:
          1. NES, standards-based curriculum and programs, student assessment, inspection system
          2. In-service teacher training

ІІ.       Creating conditions for improving overall resource management in general
          education:
          3. Extension of the delegated budget system
          4. Optimizing the school network
          5. General Education Management Information System

ІІІ. Creating conditions for improving overall resource management in higher education:
     - Reform of allocation processes for resources and seats
     - Establishment within MES of an automated higher education management information
       system
     6. Improving efficiency in allocating resources to higher education

ІV. Establishing a student loan program and restructuring the existing student stipend
    system:
    7. Retaining the accessibility to higher education in Bulgaria

V.        Creating a competitive fund for improving teaching and resource management in
          HE:
          8. Improving the quality of teaching and the management of higher education institutions

VІ. Strengthening MES capacity for project management and communication:
      - Establishing a PCU
      - Developing and conducting a PR campaign
Comments:
There are significant differences between the English version of the PAD and its Bulgarian
translation, as well as between the PAD and the EMP. The key difference is in PAD objective
VІ, which remained unclear in the EMP and its component structure. This lack of clarity leads to
problems in the management scheme and the communication strategy.


*
    The PAD objectives are presented according to the Bulgarian translation of the document.
                                                                                                 8
The financial reports include six sub-components, some of which again differ from those that
were mentioned up to now.
These differences and their consequences will be examined further on in the analysis.

                                             *   *   *

When the project contract was signed, working groups on the individual components were
brought together. The following table lists the components and the current and former working
group leaders.

                      Working group Current:                         Former:
                             leader:

       Component:
       1. NES, standards-based        Elegia Teodosieva              Pencho Mihnev
       curriculum and programs,
       student assessment, inspection
       system
       1.1. NES, standards-based
       curriculum and programs
       1.2. Student assessment        Alexander Lakyurski            Georgi Simidchiev
       1.3. Inspection system         Mukaddes Nalbant               Velichka Milanova
                                                                     Emil Mihaylov
       2. In-service teacher training    Dinko Neykov
       3. Extension of the delegated     Sergey Magdichev
       budget system
       4. Optimizing the school          Mukaddes Nalbant            Elena Taskova
       network
       5. General Education              Dimiter Tzvetkov            Angel Angelov
       Management Information
       System
       6. Improving overall resource     Nedialka Stoeva
       management in higher
       education
       7. Retaining the accessibility    Dimiter Tanev               Vassil Kitov
       to higher education / Student
       loans and stipends
       8. Improving teaching and         Borislava Kolchagova        Georgi Tzvetkov
       institutional management          (acting)

A number of MES employees and external experts have also worked on the project.

In analyzing the implementation of the project we will stick to the original concept as outlined in
the PAD, but we will also bear in the mind the component structure described in the text of the
working document, the Education Modernization Project.




                                                                                                  9
        Analysis and Evaluation of Activities under the Individual
                              Components


       General Education

І.    Creating conditions for improving the quality of teaching and learning
in general education 

1. COMPONENT National education standards, standards-based                                     curriculum       and
   programs, student assessment, inspection system
   Three sub-components have been structured under this component:
            NES, standards-based curriculum and programs
            Student assessment
            Inspection system

     Goal:
    To improve and perfect the quality of teaching and learning.

    Objectives:
   А) Developing new content of education at the national level and introducing national
education standards and standards-based curriculum in general education.
Comment:
The reform in the content of education aims at promoting the personal development of students,
encouraging critical thinking, and building argumentation skills.

   B) Creating an external system for student assessment capable of providing reliable
information on the quality of education by measuring students’ achievements; organizing
national matriculation exams at the end of general education.
Comment:
Both documents (the PAD and the EMP) provide for the creation of a national system for
“external” student assessment that would overcome localism and subjectivism in evaluating
students’ achievements. This system would consist of a series of national tests at grades 4, 8, and
12, which would be a precondition for continuing to the next level and qualifying for a school
with a higher reputation in the school hierarchy. The “external” student assessment system
would also eliminate unequal opportunities perpetuated by the “parallel system” of private
teaching.

   В) Developing and introducing an improved system for inspecting, monitoring and
supporting schools in the process of implementing the new content of education.
Comment:
The inspection system would provide methodological guidance in pursuing the goal and the
objectives of the project.



  The Roman numerals indicate the objectives as listed in the PAD, while the Arabic numerals – their detailization in
the EMP as components.
                                                                                                                  10
     Activities carried out in the period 2000-2002
    А) Preparatory period:
      (co-financed by the Open Society Foundation - Sofia)
-   Developed national education standards for the content of education - 2000;
-   Developed information packages on the concept of general education reform - 2000;
-   Organized seminars on the information packages for experts from the regional inspectorates,
    headmasters, teachers, professors at teacher training institutes - 2000;
-   Developed curriculum framework - 2000.

    Б) Phase One:
       (of the WB loan)
-   Developed programs for grades 1, 5, 9 and 10 – 2001 (grades ІХ and Х in cooperation with
    the Open Society Foundation - Sofia);
-   Developed programs for grades 11 and 12 – 2001 (in cooperation with the Open Society
    Foundation - Sofia);
-   Developed job descriptions - 2001 and 2002;
-   Constituted Curriculum Council - 2001;
-   Allocated premises for a Resource Center - 2001;
-   Procurement of a consultancy agency to improve the capacity of the General Education
    Directorate – considering the technical and pricing offer of CITO GROUP - 2002;
-   Developed draft programs for the subjects included in the curriculum for grades 2 to 8 -
    2002;
-   Approved programs for grade 2 - 2002;
-   Developed project for a Matriculation Exam Assessment System - 2002;
-   Pilot matriculation exams organized on May 8 and 11, 2002;
-   Amended terms of reference for procuring a consultancy agency to improve the capacity of
    the General Education Directorate - 2002;
-   Proposal for amending the terms of reference for procuring an international consultancy
    agency on sub-component NES, standards-based curriculum and programs - 2002;
-   Established commission to develop the terms of reference for procuring a firm for computer
    training - 2002;
-   Curriculum Council members nominated - 2002.

     What is pending:
-   Introducing the new programs for grades 1 and 12 based on the NES for CE;
-   Discussing the programs for grades 3 through 8;
-   Developing a package of documents to analyze the quality of revised NES for CE, NES for
    students assessment, standards-based programs, textbooks and reference materials;
-   Organizing capacity-building seminars for the General Education Directorate;
-   Enhancing the role of the Curriculum Council in order to involve stakeholders in pursuing
    the goals of the general education reform.
-   Procuring an international consultant to provide technical support for designing and
    developing a new national inspection system;
-   Procuring a national consultant to develop a new national inspection system.

     Current problematic issues related to sub-component NES, standards-based
      curriculum and programs and sub-component Student Assessment:
                                                                                             11
-   The composition of the CC has been determined but the individual members have not yet
    been officially appointed. This was done in 2002 when the new curricula and programs have
    already been adopted. Since the CC is an advisory body to the Minister of Education and
    Science with a wide representation of the MES leadership and the general public, the fact
    that it does not function properly means that there is no public control on the content of
    education – a task that was explicitly provided for in the PAD;
-   The Resource Center was not established although it is envisaged in the project as an
    institution providing reference materials, periodicals, computers, information. Only technical
    equipment and furniture have been procured, but there are not being used as intended;
-   No changes consistent with the goals of the reform have been planned in pre-service teacher
    training;
-   There is no mechanism for evaluating the results of the matriculation exams, as well as for
    introducing similar exams in grades 4 and 8;
-   There is no consensus on whether the test system should be used in the matriculation exams;
-   The results of the pilot matriculation exam have been analyzed by MES.

    Current problematic issues related to sub-component Inspection system:
   The Inspection system sub-component has not been adequately incorporated in the overall
component:
- Activities come down mainly to “developing terms of reference” and “amending
   solicitations” for international and national consultants. All efforts up to now have been
   unsuccessful:
       - The tender for procuring an international consultant did not go through because none
           of the companies approved at the initial stage submitted an offer;
       - The tender for procuring a national consultant was organized but no consultant was
           selected;
- There is no concept on the future development of activities. The key issue whether there
   should or should not be “national and/or international consultants” remains unclear. It should
   be noted explicitly that the WB does not make a difference between national and
   international consultants. But given the still indeterminate strategy, the fact that no
   consultants have been chosen has a negative impact on the implementation of this sub-
   component.

     Evaluation:
Compared to the other components, the component NES, standards-based curriculum and
programs, student assessment, inspection system could be evaluated as successful (except for
the Inspection system sub-component). Although some activities associated with developing
curricula and programs have started in the mid 1990s and not all of them have been completed, it
could be said that the activities, which have been reported according to the formal requirements,
testify that the education reform project is progressing as far as new education standards are
concerned, and the first steps in introducing unified standards for student assessment have been
made. To what extent the new education standard have led to an actual modernization of
education could be established after monitoring teaching based on the new curricula and
programs. It is also unclear what changes have occurred in the methods of teaching and to
what extent education has shifted to developing the potentials, the critical thinking skills,
and the ability of the students to conduct argumented debates.
Meanwhile, however, there is no clear strategy on the future of the testing system, on its
objectivity and reliability – the issue whether students’ achievements should be evaluated with
                                                                                               12
tests or in some other way still remains unresolved. The status of the matriculation exam –
whether it would be a school graduation or a university entrance exam – also involves two
different strategies. The introduction of a national exam at grade 12 without a unified evaluation
system would hardly lead to the desired results. As one of our respondents put it this would be
like “building a house from the roof down.”

The activities under the Inspection system sub-component have not been implemented in
time. This is due not only to the frequent change of the sub-component team leaders, but
also the lack of clear vision on the role and function of Regional Inspectorates in the
process of introducing new education standards, on one hand, and in the potential
decentralization of the school network, on the other. In this respect, efforts should be made to
link activities under the Inspection system sub-component with those under the SNO, Delegated
Budgets and especially IS building component.

The key problem is the poor coordination between the sub-components within the
component, as well as between this and the other components. The team members working
on component NES, standards-based curriculum and programs, student assessment, inspection
system as a whole have no information on the overall status of the project.
The itemization of activities within the sub-components included in a component creates the
risk of over-emphasizing some of them, which does not benefit the quality of education. The
new education standards should be pegged to a unified assessment system, which in turn implies
a new role of the Regional Inspectorates. In the same time the new education and assessment
standards should be closely linked to the reform in HE. If hypothetically the quality of GE
improves radically, this would place new demands to university teaching, and then the issue of
linking school graduation (the matriculation exam) and university admission would become
imminent.

     Recommendations:
   The key recommendation ensues from the key problem: the sub-components within this
    component should be closely linked together, and the component itself should be
    connected with the other GE, as well as HE components – the output of GE directly
    affects access to HE, the quality of teaching in GE influences the quality of teaching in HE,
    and this in turn has an impact on the quality of teacher training. Therefore, it is necessary:
     To develop a clear vision on the student assessment system – tests and/or other types
        of exams, formulating particular requirements, which the national exams at the three
        levels of school education should meet;
     To conduct a debate on the role of inspectorates in the new situation in order to
        clarify their place and functions;
     To conduct joint sessions of all teams working on this component in order to discuss
        completed and pending activities and fine tune the strategy for the development of the
        component;
     To conduct joint sessions with the HE component team leaders;
     It is imperative to coordinate this component with the Teacher Training component
        because change would not come only by introducing new curricula and programs,
        but also by linking them to a real reform in teaching, targeted to developing the
        potentials of the students, encouraging critical thinking, building skills for argumented
        debate, promoting tolerance of opinion;


                                                                                               13
   To involve the CC in the process of developing and monitoring the quality of school
    curricula and programs;
   It is imperative to start monitoring the impact of the new education and assessment
    standards. This could be done by the Institute on Education in partnership with higher
    education institutions; it would be also beneficial to organize seminars for school teachers
    and headmasters across the country with representative of the project working groups, as
    well as to seek contacts with the beneficiaries of the reform;
   To keep the composition of the working groups as stable as possible because frequent
    changes affect work and entail a prolonged adaptation and orientation period for the newly
    appointed team members;
   To explicitly include the responsibilities, which experts have in individual project
    activities, in the job description of each working group member.

                                            *   *   *

2. COMPONENT In-service Teacher Training

     Goals:
    1. To introduce a new in-service teacher training system that is competitive, demand-driven
       and consistent with the current situation;
    2. To create a cascade system for ongoing training based on the training of trainers method;
    3. Improving the capacity of teachers to support the implementation of the new standards,
       curricula, and teaching methods.

     Activities carried out up to now:
-   A tender for procuring an international company to provide technical support to headmasters
    and assistant headmasters has been organized, and a contract with Gopa Consultants –
    Germany has been signed; the international consultants have started work;
-   All procedures for procuring an international company to develop an improved demand-
    driven teacher training system have been completed; a contract with Education Development
    Center – USA is pending;
-   The tender for a national consultant is currently under way; the terms of reference have been
    developed;
-   A tender for a tour operator who would provide facilities for the training is being organized;
-   Information on the number of teachers who would be trained has been collected from the
    IEs.

     What is pending:
   Because work on the component has been delayed, upon a proposal by the Deputy Minister
    for GE some changes in the overall design of the component have been made: the
    development of information packages for teacher training has been cancelled; one of the
    training seminars for headmasters, assistant headmasters and IE experts has been called off;
    the number of teacher training seminars has been increased, and some changes in the training
    schedule have occurred.
   Training packages are being developed.




                                                                                               14
     Current problematic issues:
At this stage the activities within the Teacher Training component are being organized only in
terms of logistics, rather than as part of the overall reform effort. This is also due to the fact that
there is no information on the training needs of the different groups of teachers.

      Evaluation:
There has not been enough progress within this component. Actual teacher training is still
pending. The reform however would be impossible without appropriate teacher training.
Everyone, including IE experts and teachers, agree that teacher qualification is instrumental to
the successful implementation of the goals set in the first component NES, standards-based
curriculum and programs, student assessment, inspection system.
Reasons for the delay:
It is obvious that this component should be developed after component 1, therefore the delay is
to some extent inevitable. In the same time however, if the two working groups had coordinated
their efforts, programs and information packages could have been developed simultaneously.
The delay in the procedure for procuring a national consultant is quite surprising. Probably there
are only a few Bulgarian institutions capable of providing consistent training on the new
education and assessment standards because there is still no real market for training services in
the field of GE. However, some resistance against the consultants is also possible when it does
not come to a radical innovation in the training principles, i.e. when the goal of the reform is not
being pursued.
If teacher training is hindered precisely by the lack of a real market for training services, then
this component should by all means be linked not only to component 1, but also to the SNO
component, the Delegated Budget System, the development of General Education Management
Information System, as well as the creation of a Competitive fund for improving teaching and
resource management in HE.

     Recommendations:
   This component should be given a priority because it would actually spur the activities
    within component 1 and would improve the quality of education;
   Training on the new curricula and programs should be closely linked to the introduction of
    new teaching methods;
   Teacher training could be effective only if:
     there is a system for identifying the training needs of teachers – this could be done by the
       IEs;
     this component is being implemented in close coordination with component 1;
     teacher training is accompanied by an ongoing monitoring of the impact it produces;
     the newly developed Information System includes indicators related to obtained and
       desired qualification.




                                                                                                    15
ІІ. Creating conditions for improving overall resource management in
    general education


3. COMPONENT Extension of the Delegated Budget System

     Goals:
   To create and develop a decentralized management of financial resources in GE, and to
    advance the financial independence of schools.
   To promote effective use of resources; to create a mechanism, but also a tendency to manage
    resources cost-efficiently, which would have an impact on quality, too. I.e. to introduce a
    goals-oriented form of management and use of resources that is dynamic and flexible, and
    provides for active decision-making. This could be made possible by the introduction of clear
    criteria, transparency and universal application of this approach.

     Activities carried out up to now:
А) Preparatory period:
Work started under the Phare Program (1995-1998) – the delegated budget system was piloted in
four municipalities. This is when the General Education Resource Management Unit (GERMU)
was established.

B) Current development:
Under the WB loan the delegated budget system was extended to 20 new municipalities in 2001
and 18 in 2002. Municipalities were chosen based on a voluntary decision by the Municipal
Council (an MC decision is explicitly required).
 The legislative background for implementing the delegated budget system has been created -
   § 25 of the Transient and Closing Provisions of the State Budget Act for 2001.
 Support visits were made to the 20 municipalities, which have been implementing the
   delegated budget system since 2001; meetings with local governments and school
   headmasters were held; support was provided in determining the key elements of the funding
   formula, and in drafting and managing independent school budgets. A municipal forum on
   financial decentralization of schools was organized in Pazardjik.
 A tender has been organized and a 12-seat minivan was purchased for transporting trainers
   and working group members to municipalities implementing the delegated budget system.
 A tender has been organized and PCs were purchased to equip the GERMS Centers.
 A procedure for choosing an international consultant has been conducted.
 A procedure for choosing a national consultant has been conducted.
 The international and the national consultants have analyzed previous experience and have
   developed a program for training local government officials and school headmasters. The
   training package has been updated.
 A two-day training of trainers seminar under the component has been organized.
 A tender procedure was started to select an international company to supply 400 PCs, 400
   printers, and 400 copiers for the schools in municipalities where the delegated budget system
   is being implemented.

     What is pending:
                                                                                               16
   Evaluating solicited offers for the supply of computer and office equipment, and signing a
    contract for the delivery of 400 PCs, 400 printers, and 400 copiers;
   Organizing a visit of experts working on the delegated budget system, trainers and school
    headmasters in a country where decentralized resource management in education is well
    developed – 18 participants for 10 days. It is important to establish a clear procedure for
    selecting participants and to define how they are expected to share the experience they have
    gained;
   Extending and updating the existing software;
   Organizing six seminars for training school headmasters and administrators, and local
    government official on the principles and the implementation of the delegated budget system,
    as well as on the use of the customized software;
   Organizing support visits to the municipalities, which have joined the delegated budget
    system in 2002;
   Conducting evaluation visits to the municipalities, which have joined the delegated budget
    system in 2002;
   Strengthening the capacity of the GERMS – appointing two new people with clear terms of
    reference.

    Evaluation:
This component is among the most successful (along with HECTMS), activities are being
implemented within the planned deadlines, there are no delays. This is also the only
component were the impact of activities (the introduction of the delegated budget system) is
being monitored.
There are several reasons for the successful implementation of the component:
 Firstly, the project was approbated with the support of the Phare Program in 1995-1998, the
   actual difficulties and challenges were identified, and possibilities for overcoming them were
   outlined.
 Secondly, the leader of the component has not been changed, the members of the working
   group have remained relatively stable, team-working is good, the people involved have
   experience in what they do and have no problem developing terms of reference because they
   know what they want and how to write terms of reference.
 Thirdly, this component seems independent and in this respect ostensibly does not depend on
   the progress on the other components.
 The problem, however, is precisely in the ostensible independence of this component
   from the others. Currently, the implementation of the delegated budget system is an
   exception, rather than an established practice. In order for this system to become effective, it
   should be integrated in the more general context of financial decentralization, on one hand,
   and the decentralization of education, on the other. An overall decentralization strategy
   needs to be developed based on a public debate among municipalities and the central
   government, namely the MES. This would be impossible without relevant information
   on the condition of the school network and the local situation. In other words this
   component is closely related to the Information System component (it is quite
   bewildering why the GERMS Center, which collects data and works with advanced
   software, is not perceived as part of a comprehensive information system), the School
   Network Optimization component and the Inspection system component (real
   decentralization eliminates or limits the role of local units of the central administration
   such as the regional inspectorates). If the delegated budget system functions optimally,


                                                                                                17
    funds could be allocated to train teachers on specific issues, thus creating conditions for
    a market for education services to emerge.

      Recommendations:
   The Delegated Budget System component should be closely linked to the Information System
    component with GERMS becoming an integrated part of the overall information system. the
    component should be also linked to the School Network Optimization, the Inspection system,
    and the Teacher Training components;
   The delegated budget system should be analyzed not only in terms of its positive and negative
    features and impacts, but also with respect to the conditions for its implementation – which
    municipalities decide to implement the system and why, which of them refuse and for what
    reasons;
   The existing practices in the municipalities should be monitored more actively;
   An active PR campaign on the benefits of the delegated budget system should be organized;
   The experience of the municipalities already working under the delegated budget system
    should be publicly discussed in order to support the initiated debate on decentralization, on
    one hand, and to transform the delegated budget system from an exotic phenomenon into a
    conscious and effective practice.

                                            *   *   *

4. COMPONENT School Network Optimization

     Goals:
- To support the MES, the regional inspectorates and the municipalities in implementing a
program for closing down and restructuring schools in order to guarantee equal access to quality
education despite the demographic decline and the scarce financial resources.
- In view of the more general goal, formulated in the PAD, the ultimate objective is not to close
down schools, but to achieve cost-effectiveness and quality of education by optimizing the
allocation of school resources, i.e. spending to be linked to specific results.

     Activities carried out up to now:
-   A tender for soliciting an international consultant to provide technical assistance has been
    organized and a contract with the selected expert has been signed. On June 11, 2002
    negotiations were held with the international consultant (Dr. Hobrou from the University of
    Surrey), correspondence with the component leader followed, actual work will start in the
    fall;
-   A tender for soliciting a national consultant has been organized, but no consultant has been
    chosen with the explanation that none of the candidates meets the tender requirements;
-   The working group has sent a letter to the WB suggesting that no national consultant is
    needed. The WB has replied requesting additional details and argumentation.

      What is pending:
It is unclear what is pending because the component leader has not presented at the PCU a plan
for the future development of component activities, nor an estimate on the necessary financial
resources by the end of the first half of 2002.
Comment:


                                                                                               18
There is a discrepancy between the planned development of this component and the views of
new component leader. There is no discord that schools should be closed down, but the working
group members insist resources to be invested in the relevant municipalities – for purchasing
buses to transport children to “consolidated” schools, as well as for renovating and furnishing
“consolidated” school buildings. However, no funds for such activities have been allocated under
the first phase of the Education Modernization Project. Moreover, there is no clarity on the
methodology and the formula, which would be used to identify the schools that should be closed.

        Evaluation:
Almost nothing has been done under this component. The reasons for that are:
- Differences in the institutional culture and inadequate understanding of the WB procedures:
    The correspondence exchanged between the component working group, the PCU, and the
    WB shows that the members of the working group do not know the bank’s procedures and
    violate them by requesting additional documents from the candidates after the notice for
    soliciting a national consultant has been already published, when all conditions and
    requirements should be included in the terms of reference. The failure to comprehensively
    list and clarify, in the terms of reference, all requirements and criteria for selecting a national
    consultant led to delays in the tender procedure and resulted in not choosing a consultant. It
    is probably true that there are no local experts capable of providing consultations on a
    delicate and complex issue such as the School Network Optimization. But then, one can
    hardly understand the opposition, which exists against the international consultant, too.
- The correspondence with the PCU also indicates that project goals and deadlines are not
    being respected. Since there is obviously some problem, it should be publicly discussed and
    solved;
- Those who work on the component and the new component leader agree that the school
    network should be optimized and “consolidated” schools should be created in order to
    improve the quality of education. In this case efforts should be made to develop a clear
    methodology for achieving cost-efficiency and quality of education, rather than just close
    down some schools and transfer the students to others. In this respect, the experience in
    implementing the delegated budget system could be very useful.
- This component is closely related to all other components – the successful implementation of
    the new education standards and assessment standards, the inspection system development,
    the reform in teacher’s training, and most of all the extension of the delegated budget system
    and the creation of an information system. Until the component is perceived as an integrated
    part of the overall project, no real progress on it could be made. Currently, economic
    problems are reduced to merely fiscal and remain detached from social and educational
    issues – what schools, for whom, with a view to achieving what (not because of some current
    concern, but in pursuit of a future goal); it is unclear whether empiric analyses on the status
    of municipalities have been made, and to what extent they are being used.
Comment:
We do realize that the implementation of this component is a formidable task, and that the
different views on school network optimization reflect most distinctly the conflict between the
social protection background of the Bulgarian education system and the drive for an education of
better quality. One should bear in mind however that today’s protectionism has serious
consequences on the quality of education and in effect jeopardizes tomorrow’s professional
development and fulfillment of many children and young people. The deferral of this educational
problem today would lead to a series of social problems in the future.


                                                                                                    19
   Recommendation:
 Active measures should be taken to promptly develop a methodology for improving the cost-
  efficiency and quality of education, and most particularly for introducing clear priniciples for
  closing down schools. In this respect, we believe that consultancy is necessary.
 The information system should be quickly brought into operation. Information on fiscal,
  social and purely educational parameters should be collected from all municipalities and
  analyzed.
 Work on this component should be closely related to all other components in general
  education. The SNO component has a direct bearing on the financial decentralization debate,
  as well as on the decentralization of education management; it is closely linked to the
  successful implementation of the new education standards and assessment standards.
 The immediate development of a special PR campaign is particularly necessary for this
  component. Such a campaign should take into account and consider the concerns of parents
  and teachers, but also explain and clarify the consequences of preserving schools that need to
  be closed down.
 Communication and cooperation with trade unions should be more active to guarantee that the
  criteria for school network optimization will be made clear well in advance to prevent or
  attenuate social tension in school, which are being closed down.

                                            *   *   *


5. COMPONENT General Education Management Information System

    Goals:
To improve the capacity of MES for collecting, analyzing, and distributing information
necessary to implement reforms in general education and optimize education management.
This goal will be pursued by:
       - Developing and establishing a methodology for assessing, analyzing, comparing, and
           making estimates on the status and the development of the general education system;
       - Creating an information infrastructure necessary for the functioning, management and
           modernization of general education;
       - Providing electronic access to MES regulations and bylaws, as well as to other
           education-related legislation through the GEMIS.
       - Creating conditions for strengthening the process of decentralization in general
           education and establishing a market for school information systems, which have
           standardized information exchange protocols with the national GEMIS.

According to the PAD phase I of the project (2002-2003) would include: 1) developing and
implementing capacity for information gathering through the use of periodic formulation of
educational surveys, focus groups, special policies and for the evaluation of studies, auditing,
etc.; 2) developing and implementing capacity for data analysis (system, statistical, financial,
cost, simulation and forecast, dissemination and reporting); 3) providing new and equipped
premises for staff training.
According to the EMP during phase I a working GEMIS prototype should be developed,
approbated and improved, along with the required organization and staff training.

      Activities carried out up to now:
                                                                                               20
       -   Terms of reference for soliciting an international consultant have been finally
           developed after being rejected several times by the World Bank;
       -   A short list of consultants has been drafted.

       What is pending:
       - Choosing a consultant;
       - Studying the standards and the organization technology of information management
         in general education (system of primary data, indicators, statistical procedures and
         criteria applied for diagnosis, comparison, prognosis, and management);
       - Developing, introducing and improving the GEMIS prototype;
       - Creating and developing the regional and national databases and the applied software;
       - Specifying computer systems and communication infrastructure;
       - Delivering and installing equipment and base software;
       - Identifying needs and developing a plan for training GEMIS administrators and users;
       - Testing the prototype.

       Current problematic issues:
At the last WB mission in May 2002 a decision was taken to integrate activities on GE and HE
information systems (there is a recommendation to this effect mentioned in the Memorandum).
But such a development is being discussed since 2001. Currently working groups on general and
higher education are being formed, and a unit is being established to coordinate their activities.
The goal is to speed up work on this component, as well as to compensate the fact that the MES
Education Modernization Project does not provide for establishing a structure to create an
automated information system for higher education management.

        Evaluation:
There is a serious delay in the implementation of activities under this component. In fact,
virtually no work has been done (in July 2001 the World Bank issued recommendations for
amending the terms of reference; in May 2002 there are still remarks on it). This is particularly
disturbing given the key importance of this component for the successful implementation of all
other components.
The reasons:
       а) Organizational – The leader of the working group quit one month after work was started
(April, 2001 г.) and for a long period of time no one was appointed to replace him. The decision
to integrate activities on GE and HE information systems is not administratively regulated yet.
Institutional culture is poor (the continual requests and requirements submitted by the PCU are
not replied to in due time).
       B) The problem however is why a component, which should provide the basis for
implementing all other components, is neglected. And this brings us to the substantive reasons.
One can still feel the opposition of the old institutional environment, of the over-centralized state
administration whose leading principle was that “secrets are the essence of bureaucratic
management”. Information deficit is a background for irrational actions, for ineffective
management, for over-bureaucratization. The transition to a decentralized and competitive
education system based on actual needs requires a shift from bureaucratic management to
management based on information.
       The lack of concern towards this component affects the whole project. The GERMS Center
is not committed to developing specific indicators to collect information from municipalities
regarding school network optimization, or to assess the quality of education. There are no data
                                                                                                  21
on the qualification needs of teachers, the developments on the labor market, etc. sociological
research on education is also disconnected and are not being systematically monitored.
    Since the Information system component should provide the basis for all other
components, one can conclude that until now project activities have been implemented
without an analytical background. Hence, their dependence on accidental factors, which
creates a possibility to arbitrarily speed certain activities up and then delay them without a
valid reason.

    Recommendations:
    The component should be given priority. The ongoing reorganization of information
     systems for general and higher education should be completed as soon as possible to help
     make up for the lost time.
    The team working on the Information System component should cooperate closely with
     the working groups of all other components, requesting and systematizing the
     information that has been collected by them.
    The subordination between the Information Technologies Directorate at MES, the
     Institute on Education, the General Education Resource Management System, and the
     prospective new unit should be made clear. With so many structures already available is
     it really necessary to establish a new unit or should the human resource capacity of the
     existing one be strengthened and its powers and responsibilities clearly defined?
    Criteria for management efficiency should be introduced. Decision-making should be
     based on reliable information for the current status, rather than on arguments such as “my
     experience shows”.
    The indicators on which information is collected should include the widest possible range
     of data to allow for monitoring the dynamics of the education system in relation to the
     other elements of the social environment. Data should be collected on municipalities
     (social and financial), but also on general and higher education institutions; research
     studies on the developments in education; the system for monitoring the implementation
     of reform activities; databases on national education standards, teaching standards, and
     student assessment standards; results from the general exams; legislative documents,
     which have a bearing on education (not only those targeted specifically to education); list
     of experts in this field, etc.;
    The external and internal monitoring of the implementation of education reform activities
     should become an essential part of the information system.
    An information database on sociological surveys in education should be created and
     systematized to allow for monitoring developments in education;
    The information collected by the Project Coordination Unit and the newly established
     information system for general and higher education should be made publicly available;




                                                                                              22
Higher Education


ІІІ. Creating conditions for improving overall resource management in
     higher education


     With sub-components (according to PAD):
     - Reform of allocation processes for resources and seats in higher education;
     - Establishment within MES of an automated higher education management information
         system.
     This goal (PAD) is operationalized in the EMP as:


6. COMPONENT Improving the efficiency and effectiveness in allocating resources to
   higher education institutions


      Goals:
According to PAD:
- Reforming the allocation processes for resources and seats in higher education:
           1) a consultation process for deciding on the principles and priorities that will guide
           the new allocation system;
           2) the establishment and setting into operation of a administrative body to manage
           the allocation of resources and seats
- Establishing within MES an automated higher education management information system.:
           1) create and equip a unit for establishing and managing the information system;
           2) design and validation of data collection instruments and processes;
           3) train personnel to build capacity for analyzing and registering data.

According to EMP:
- Main goal: To improve the efficiency of higher education by targeting state funding to the
   most effective institutions and majors, while retaining the priorities of state policy in higher
   education.
- Strategic goal: To transform the allocation of resources and seats in higher education into a
   mechanism for implementing the state policy in higher education.

    Activities carried out up to now:
     - A working groups has been created under this component (Order No RD 09-
        269/19.04.2000);
     - A Resource Allocation Committee has been established (Order No RD 09-
        291/07.06.2000);
     - A Higher Education Council has been created (Order No RD 14-117/07.07.2000);
     - Terms of reference for the international consulting companies have been developed, a
        tender has been organized, and a contract with Helsinki Consulting Group has been
        signed in the summer of 2002.
     - The members of the working groups and the RAC have undergone training in Finland
        (September 2002).
                                                                                                 23
     What is pending:
-   Developing cost and resource profiles of each HEI;
-   Determining the normative costs for each student and developing a system of normative costs
    for determining allocations and tuition fees;
-   Developing a methodology for allocating seats based on higher education institutions and
    majors;
-   Developing a unified procedure for resource and seat allocation in higher education;
-   Establishing a Higher Education Management Information System.

     Evaluation:
Although the successful implementation of this component is considered as extremely important
for the modernization of higher education (MES even considers it the most important since
resource and seat allocation is an instrument for implementing the state policy in this sector),
most of the activities planned have not been performed.
As early as the summer of 2001 the WB issued a Memo noting that the Resource Allocation
Committee has been established, but is not fully operational and does not meet regularly. The
Memo noted also that the working group should be restructured with new members who are
motivated to contribute (for six months only one of the working group’s members has sent
comments on the terms of reference). In October 2001 the World Bank issued another Memo,
stressing that there has been a serious delay of activities under this component, and noting that
the RAC should be “re-activated” and new terms of reference should be developed. The Memo
also underlined that the members of the working group should undergo training. In May 2002 the
WB mission remarked that a major problem with this component is that the broad stakeholder
community has not been sufficiently involved in its implementation, which could make expected
results difficult to achieve.
Since minutes from HEC sessions have not been made available to us up to now, it is unclear
what is the Council’s contribution to the project. Nothing has been done yet within the MES and
the Rectors’ Council to synchronize seat allocation with the real labor market needs. Such
synchronization requires a long-term strategy and a mechanism for estimating the dynamics in
the cost of education and the demand for particular education of particular quality consistent
with the developments on the labor market. This would be impossible without an education
management information system capable of providing information on the demand for certain
type of education, its quality, and the employment opportunities it would open. None of the
interviewed persons working on this component, however, made a reference to the need to
establish such an information system for higher education (HEMIS).
The explanation that has been provided for this delay is that there are different strategies and
views on how should normative costs per student be calculated, and none of these concepts can
take the upper hand. The problem is that it is not clear whether normative costs per student are
determined based on the current system, or based on international standards. Moreover, this
component should be directly linked with the new Higher Education Act, i.e. with the overall
strategy for developing higher education, which has not been done up to now.
In other words, the delay in the implementation of this component is largely due to the fact that
there is no clear vision on the reform in higher education – how should it be carried out and
what should it be aimed at, – and there is a hidden resistance to change.
Further delay in the implementation of this component would have a negative impact on the
other two components, too, because it is expected that an refined resource allocation would
encourage the process of consolidation within the higher education system, and together with the
                                                                                               24
HECTMS, would boost the efficiency of the system. The budget resources thus freed may be
used to improve the quality of education. The transition to a competitive market-oriented
system could be made only if the three higher education components are considered as a
whole. Otherwise they would make no sense.

    Recommendations:
   - The RAC should be extended to include representative of the academic community,
     students, and Ministry of Finance officials;
   - The activities planned under this component should be speeded up;
   - The establishment of the HEMIS should be speeded up;
   - The members of the RAC and the component group should be provided with all
     necessary information;
   - The duties, responsibilities, control functions, and coordination mechanisms should be
     clearly defined;
   - Stakeholders should be informed and a broad public debate should be organized to
     discuss the structure of normative costs per student, the methodology for allocating seats
     among universities and majors, the resource allocation methodology, and the changes in
     the legislative framework.




ІV. Establishing a student loan program and restructuring the existing
    student stipend system


7. COMPONENT Retaining the accessibility to higher education in Bulgaria

    Goals:
According to PAD:
To develop a financial model to estimate the costs of different financial assistance arrangements.

According to EMP:
- To extend the possibilities for providing student loans;
- To restructure the existing student stipend system;
- To improve the system for determining tuition fees.
Comment:
The different formulation of these two avenues for modernizing higher education bespeaks of the
different understanding of the goals of this component. In the first case the reform effort is
targeted to limiting the protectionism of the state, while in the second case the emphasis is placed
on a more general issue of guaranteeing access to higher education, which involves maintaining
a system of low tuition fees and high stipends. The most important goal is to retain the
accessibility to higher education, although it is mentioned last among the goals listed in the PAD.
It is also necessary to allocate resources for improving the quality of education.

    Activities carried out up to now:
   - The World Bank has approved the terms of reference for the international consultant (on
     March 16, 2002 according to the PCU).
                                                                                                 25
   -   Currently, the commission established under the component is reviewing and evaluating
       the technical offers submitted by the companies applying to provide consultancy.

     What is pending:
Hiring a consultant and studying in detail the optimal model that could be applied in our country:
- Developing options;
- Studying the needs of the beneficiaries and the consequences of the different options;
- Drafting the legislative framework;
- Organizing a media campaign to rally public support.

    Current problematic issues:
   - At this state student loan programs do not fit within the institutional environment in
     Bulgaria – tuition fees are low, there is a stable student stipend system, the labor market
     is still underdeveloped, no legislative background in this respect exists;
   - There is no propensity to seek credits and loans (sociological research shows that most
     Bulgarian tend to lend money from relatives, rather than banks);
   - There is no actual demand for student loans.

     Evaluation:
There has been a serious delay in implementing the activities planned under this component.
This is due to the lack of clarity on the goals that are being pursued and the results to be
achieved. This lack of clarity is manifested in the repeated renaming of the component. In the
PAD the emphasis is placed on student loans, in the EMP – on retaining the accessibility to
higher education. In May 2002 the name of the component was changed to Student Loans and
Stipends in Higher Education. Formulated as Retaining the Accessibility to Higher Education,
the component requires not only to settle financial issues, but also to establish a close connection
with the general education system. If the school network is not optimized, higher education
could become inaccessible to the inhabitants of whole districts. The issue whether national
matriculation exams would be accepted for university admission also has a bearing on this
component.
The fact that in the terms of reference the focus is placed at one point on the stipend system, and
at another on student loans also indicates that the priorities are not clear and there are concerns
that reforms may trigger opposition.
One is left with the impression that at the time being student loans are misunderstood and
underestimated, and do not correspond to the established traditions and the current needs in
society.

       Recommendations:
The MES position on this issue should be clarified. The goals of higher education modernization
should be clearly defined and pursued. We believe that in view of the overall logic of the reform
in education – quality assurance through effective resource allocation – the component should
not be shelved, but conditions should be created for establishing an operational student loan
system. This would require:
- Introducing transparent rules for results-based provision of student loans with a view to
    creating an open competition among lender;
- Drafting a law on student loans, which would change the relationship between state
    institutions and universities, and would make universities fully independent, but also publicly
    accountable;
                                                                                                 26
-    Developing a formula for the actual cost of scientific research;
-    Introducing transparent and cost-effective university management to allow for determining
     the actual cost of education;
-    Studying the experience of countries where conditions are similar to the Bulgarian situation.
     Such information could be collected not only through external consultants, but also through
     the regular channels of communication between MES and similar institutions abroad.

This component should be considered together with the other components in higher education, as
well as in close relation to the student assessment system in general education.




V.      Creating a Competitive Fund for Improving Teaching and Resource
        Management in Higher Education


8.      COMPONENT:             Improving the quality of teaching and
                              the internal management of higher education institutions

        Goals:
According to PAD:
Through the establishment of a competitive fund, to help institutions improve teaching and
equipment, establish operational internal quality assurance and professional management
structures, reform curricula, integrate teaching and research, provide new higher training
opportunities to young academic staff, and encourage the consolidation of higher education
institutions to support university network optimization.

According to EMP:
The Higher Education Competitive Teaching and Management System should support MES and
higher education institutions in their efforts to harmonize Bulgarian higher education systems
with the standards of the European Union. The HECTMS Center will finance specific projects of
individual universities, university networks or key university departments and units, targeted to
developing systems for: 1) institutional and program self-evaluation; 2) quality assurance,
quality management, quality control, and quality improvement; 3) effective higher education
management.

      Activities carried out up to now:
-    A Higher Education Competitive Teaching and Management System Center was established
     (Decree of the Council of Ministers No 148/27.07.2000), Official Gazette, issue 64/2000);
-    The members of the working group were appointed (Order No RD 09-195/19.04.2000);
-    The terms of reference was approved (January 2001);
-    A Regulation on the Structure and the Activity of the Higher Education Competitive
     Teaching and Management System Center was adopted (MES, Official Gazette, issue
     19/2001); Rules of Operation of the HECTMS Center were adopted;
-    A Committee was appointed to evaluate offers and select a national and an international
     consultant under the component (Order No RD 09-998/02.03.2001); A tender for an


                                                                                                27
   international consultant was organized and a contract with the successful bidder, the
   Nottingham University Consultants Ltd., was signed on September 1, 2001;
- The HECTMS Center Board was nominated (Order No RD 09-999/02.03.2001);
- An International Supervising Committee of HECTMS was nominated (Order No RD 09-
   1320/29.05.2001);
- HECTMS Expert Commissions were nominated (Order No RD 09-1396/21.06.2001);
- The first seminar for Rectors and Deputy Rectors was held on October 25, 2001 to present
   the criteria, rules and mechanisms for evaluating projects with which higher education
   institutions could apply before the HECTMS Center;
- Between November 27 and December 5, 2001 seven seminars were organized for
   representatives of MES, HEI, the HECTMS Center Board, and the members of the Expert
   Commissions of the two main sub-components – Self-evaluation, and Quality Management
   in Higher Education. The seminars focused on the guidelines for developing projects, as well
   as on the basic documents – Contracts between the HECTMS Center and the higher
   education institutions. The results of the survey on quality assurance and effective
   management in higher education, conducted during the seminars, were analyzed.
- A Committee was appointed to evaluate offers and select a company to provide technical
   assistance for the HECTMS Center’s web site (Order No RD 09-1923/14.11.2001). A tender
   was organized. The successful bidder is Netplus Company.
- The Board has approved guidelines for participation in tenders for soliciting projects
   (27.11.2001);
- Representatives of MES and the HECTMS Center made a working visit to England to study
   the experience of British institutions in quality assurance and higher education management
   (Order No RD 13-09/23.01.2002);
- Tenders for designing advertising materials (posters, brochures, and pens) to inform the
   general public on the goals, tasks and activities of the HECTMS Center were organized on
   March 7 and 15, 2002. The successful bidders are Intelday Solutions and Compass 2000);
- Four seminars for representatives of HIEs were organized: two on development strategies in
   higher education, two on quality assurance (Order No RD 09-330/10.05.2002 г.);
- A Report on the first session of the International Supervising Committee (ISC), June 3-4,
   2002 was presented;
- A summary of submitted projects under the first call for proposals announced by the
   HECTMS Center was presented:
   1) A total of 183 projects were submitted: 14 by individual HIEs, 11 by university networks,
75 by key university departments, 83 by department networks; 47 projects are targeted to
improving university management, and the remaining 136 focus on quality assurance;
   After the initial check for compliance with the requirements 163 project were admitted for
consideration, after the second check 148 projects were ranked, 107 of them were short-listed.
   On June 6, 2002 the Board granted final approval to 35 project, 27 of which are targeted to
quality assurance in higher education, and 8 to improving university management.
   2) A database of submitted projects was created;
   3) A scheme for tracking projects was developed;
   4) A database of reviewers from HIEs was established;
   5) Consultations to HIE representatives were provided;
   6) Guidelines for writing reviews and monitoring mechanisms were developed;
   7) Submitted projects were evaluated and their implementation has started.
- A second call for proposals was announced on September 30, 2002. The application deadline
   is October 31, 2002. The total project budget is 200,000 Euro, the duration of the projects is
                                                                                               28
   two years, and the completion deadline is March 31, 2004.

    What is pending:
     - Evaluating the results achieved by the project in 1) quality assurance and 2)
       university management;
     - Adopting improved internal procedures for quality assurance;
     - Developing strategies for improving higher education management;
     - Developing national mechanisms for the exchange of experience between different
       HEIs and departments, and integrating them into existing mechanisms (academic
       network for assessment and accreditation);
     - Encouraging the enhancement of information strategies for improving management
       practices.

       Current problematic issues:
This is the least problematic component, but problems do exist here, too. These include:
- Shifting the focus away from improving the efficiency of higher education management (47
   submitted projects) and towards quality assurance (136 submitted projects) – quality in
   higher education can hardly be improved without an effective university management
   structure;
- The fact that the statutory act of the Center eliminates the involvement of stakeholders also
   presents a problem because it opens possibilities for subjectivism in nominating members of
   evaluation bodies.

     Evaluation:
The activities planned under this component have been implemented within the respective
timelines, and this is one of the most successful components under the project. This could be
explained with the following:
 The people who work on the component are qualified and trained on the project. They have
    undergone special training on the component;
 The team of the Center has remained unchanged unit the dismissal of its director in the
    summer of 2002;
 The Center is a separate unit with its own system of internal regulations where the tasks and
    responsibilities of the staff members are clearly defined;
 There is a clear interest within the target group.

In the same time, it should be noted that the high results, which outwardly cover performance
indicators, should not be achieved at the price of overlooking the priorities of the project, which
this component is part of. The reported results testify to the successful operation of the HECTMS
Center as a separate unit, rather than contribute to fulfilling the main goal of the component –
structural reform, optimization of the higher education network, establishment of information
systems for higher education management, introduction of cost-effective management. Until now
the Center has been perceived as independent outside rather than within the Education
Modernization project. The PR strategy is designed to publicize the Center and its project
competition, rather than the goals of the EMP. There is no coordination and correlation with the
teams working on the other two components in higher education.




                                                                                                29
      Recommendations:
      - The component should be integrated in the context of the EMP – overall structural
        reform in higher education. This could be made possible only by the adoption of
        relevant legislative provisions on higher education, as well as of a new Higher
        Education Act, which would regulate the relations between state institutions and
        universities not only in terms of academic autonomy, but also in terms of public
        accountability.
      - Links and coordination should be established with the other two components in
        higher education;
      - Projects targeted to improving the efficiency of higher education management should
        become a priority for the Competitive System since this would result in improving
        quality, too.
      - The activities targeted to establishing a system for internal quality control within
        HIEs, should be renewed because they are an important aspect of university
        management;
      - A system for monitoring the impact of projects supported under the first call for
        proposals announced by the HECTMS should be established. The results of the
        project should be publicized.
      - The activities targeted to establishing information systems for university management
        should be extended and coordinated with the activities on establishing Higher
        Education Management Information System (HEMIS).




       Project Management

The PAD objective Strengthening MES internal management capacity for project
management and communications has not been specified into a separate component in the
EMP. This is particularly crucial given the already mentioned lack of coordination and links
between activities under different components, the lack of common understanding of the overall
purpose of education reform, and hence the purpose of the project, as well as the significant
delays on some of the components.


MES Capacity for Project Management

The key units involved in the management and implementation of the project are:
    Project Board;
    Project Coordination Unit;
    Working groups;
    Units providing consultations (such as the Higher Education Council) and implementing
      specific project activities (such as the National Evaluation Unit).

The responsibilities seem clearly defined in the key documents on the project (Project Appraisal
Document and Project Management Manual): political guidance is provided by the Project
Board, which comprises the Minister and the Deputy Ministers of Education who as a team are

                                                                                              30
supposed to have a uniform concept on the reforms that need to be implemented; daily
coordination is assigned to the Secretary General to whom all MES staff reports; the PCU is
responsible for conducting procurement and controlling compliance with the World Bank
procedures, while project activities are implemented by the MES staff that are supposed to
perform similar activities in their everyday work.


PROJECT BOARD AND PROJECT COORDINATOR

      Current status:
           Project Board
           The Project Board includes the MES leadership team – the Deputy Minister of
           Higher Education, the Deputy Minister of General Education, and the Secretary
           General. The Project Board is supposed to meet twice a year.

              Project Coordinator
              For some time the Project Coordinator was the Secretary General (who was
              changed several times). Currently, this position is occupied by the Deputy
              Minister of Science whose direct responsibilities do not involve activities
              associated with general and higher education, and to whom the employees of
              relevant MES directorates who participate in the working groups do not report
              directly.

       Problems and Evaluation:
No minutes of the Project Board’s sessions are being kept. If this is a real management body that
takes important decisions, these decisions should be registered. Given the dynamics in the
implementation of project components, the Project Board meets at too big intervals.
One is left with the impression that the project management body does not really control the
implementation of project activities.


WORKING GROUPS;
WORKING GROUP LEADERS AND SUB-COMPONENT LEADERS

       Current status:
The key units responsible for the actual implementation of project activities are the working
groups on the individual components and sub-components (according to the Project Appraisal
Document). The members of the working groups vary in number and are mainly MES employees
(the groups sometimes include experts from other institutions). Working groups are established
with an order issued by the Minister of Education and Science. They have developed indicators
to plan and report their activities, they prepare monthly reports (submitted to the Project
Coordination Unit) and action plans. The working group leaders are also appointed with an order
issued by the Minister of Education and Science. The working groups write procurement terms
of reference and evaluate solicited offers.

      Problems and evaluation:
An analysis of documents and interviews outlines the following problems:


                                                                                               31
      There are no regulations on the number and the profile of the working group members;
       there are also no clearly regulated requirements for their leaders who are also sub-
       component leaders.
      For many working group members work on the project is an additional, external
       responsibility, not associated with their everyday tasks. For some of them this is actually
       true insofar as their positions do not involve responsibilities directly associated with
       activities under the respective component.
      It is rather an exception for working group members to have in-depth knowledge of the
       project as a whole. The sources of information include the so-called “purple book” – a
       project description published by the MES, the mass media, discussions with colleagues.
       This is a serious problem because the inadequate knowledge of the purpose and the
       activities of the project in their entirety, especially among its key participants, is one of
       the factors that have led to the current situation.
      There are significant differences in the way education reform is perceived and
       approached. Some place the focus on the quality of education, others on the place of the
       child in the education process, still others on cost-effectiveness, on improving the
       existing system, on accessibility, etc. However, a common understanding of the goals of
       the reform among stakeholders is instrumental for its effective implementation. Such an
       understanding is also important in view of the PR strategy of the project and the need to
       send coherent messages to the general public.
      There is no practice to exchange information and organize coordination meetings among
       working group leaders.
      Working group members are not held responsible for violating the deadlines for
       implementing project activities.
      The documentation requested by the PCU is not always provided in due time.
      There are no clear and objective criteria for the performance of working group members.
      The implementation of each activity requires a sanction by the Minister who is no
       position to know in detail the specifics of each component. This is a structural fault
       because those who are fully aware of the project are unable to make decisions, while
       those who do make decisions, do not know the specifics of the project activities.
      The knowledge of the World Bank procurement procedures is also inadequate. Although
       working group members are not expected to have expertise on that matter, they write a
       significant part of the procurement documents – terms of reference and technical
       specifications, which vary in format and content depending on the procurement
       procedure that would apply. Therefore, it is important for them to know these procedures
       (and the specific time frames that apply) when preparing action plans.
      The project is often reduced to organizing a series of procurement procedures – writing
       terms of reference, choosing consultants, soliciting deliveries, while the link between
       these procedures and the goals they promote, is lost.

On the whole, despite the fact that working group members do work on the project (with
varying intensity in the different groups), there is still much to be done to improve the
organization and the efficiency of their work.




                                                                                                 32
PROJECT COORDINATION UNIT

       Current status:
The PCU was established as a legal entity on March 30, 2000 with Order No RD 14-30 of the
Minister of Education and Science. The functions and the activities of the PCU are described in the
Regulations on its structure and operation, adopted by the MES on July 20, 2000.
The staff of the PCU, according to its Statutory Regulations, consists of nine staff members:
            PCU Leader
            Coordinator for Higher Education
            Coordinator for General Education
            Two procurement experts
            Chief Accountant and accountant
            PR Expert
            Secretary fluent in English

The PCU is responsible for the overall project coordination. It coordinates the technological and
financial implementation of the project (conducting procurement according to the World Bank
guidelines), but being the only structure directly responsible for the implementation of the
project, it also coordinates the preparation of annual work programs and reports.
The legal and actual status of the PCU originates directly from its place in the chain of activities
on implementing the World Bank project. This place could be described as intermediate. The
intermediate position of the PCU between the lender, the World Bank, and the borrower, the
different MES units, the Ministry as a whole, and other beneficiaries within the target group,
predetermines the utterly ambiguous status of the PCU.
The intermediate position of the PCU between lender and borrower results in a series of
problems with the coordination of the project.

    Problems and evaluation:
   Problems arising from the legal status of the PCU:
   - On legal grounds that remain unclear, the PCU, which is a secondary executor of MES
      budget credits, was established as a legal entity financed by the state budget. This is
      precisely where the “time bomb” lies. The PCU is literally torn apart – legally,
      financially, administratively and functionally – between lender (the World Bank) and
      borrower (the MES).
   - The PCU is simultaneously a structure of the lender and the borrower that has a
      peculiar organizational arrangement. The Project Board, which officially is neither a
      managing, nor a representative body of the unit as a legal entity, controls its overall
      activity (article 14, paragraph 2.1 of the Regulations); evaluates its efficiency (article 14,
      paragraph 2.2); approves changes in project activities and issues recommendations on
      improving their effectiveness (article 14, paragraph 2.3 and paragraph 2.4). That is, the
      Project Board, being an “external” informal structure, controls, manages and steers the
      activity of the PCU, which at least formally is an independent legal entity, i.e. an
      independent subject of law. The State Administration Act contains no provisions
      allowing such subordination. On the other hand, the PCU leader is appointed on a
      competitive basis by the Minister of Education and Science, and according to the
      Regulations is the only person authorized to represent and manage the PCU. Being a one-
      man managing body, the PCU leader is responsible for supervising, managing, auditing
      and reporting procurement procedures under the project, but has no authority to sign

                                                                                                 33
    contracts with successful offerors since according to article 26 these require the sanction
    of the World Bank and the Minister of Education and Science.
-   The PCU is financed by the MES and the World Bank, and in the same time manages
    financial resources of the MES and the World Bank. This is provided for in articles 2, 3,
    and 4 of the Regulations. The question is on what grounds does the PCU, which is no
    contracting party under the loan agreement signed with the World Bank, manage
    financial resources allocated under the loan? And how can a legal entity, which is no
    party in the loan agreement, administer payments?
-   The actual organization, support, administration, and implementation of procurement
    procedures are undertaken by the so-called working groups, which are structures with
    unclear legal status. According to the provisions of article 13 of the Regulations, the
    Minister of Education and Science issues orders for appointing working groups on the
    different project components, the members of which, together with the coordinators,
    evaluate solicited offers following the approved terms of reference. In practice, the PCU
    has no authority over the working groups and their leaders, although it is supposed to
    coordinate their activity. According to the Regulations on the activity of working groups,
    the powers of the PCU come down to informing the Minister and proposing solutions.
    That is why no coordination can be achieved. How could the PCU exercise control over
    the working groups? How could it bear responsibility for the implementation of
    procurement procedures when it is in no position to exercise control over them?
-   Since the PCU is designed as a “supporting” unit, its key staff is expected to organize
    procedures. The job descriptions for general and higher education do not contain
    requirements for specific qualification or experience in the education system. Without
    such qualification or experience however they would be unable to coordinate the whole
    variety of activities targeted to reforming the education system.
-   The coordination of the project involves preparation of project implementation plans for
    the individual working groups and the PCU, and submission of monthly reports by the
    working groups and quarterly reports by the PCU. Being a “supporting” unit, the PCU
    does not include in its reports information on the content of activities that have been
    carried out, but only data on the individual steps in the procurement procedures. The
    reasons for the delay in some components and the impact, which this delay might have on
    other components and the project as a whole are not being analyzed.
-   The PCU leader is not a member of the Project Board, which renders the coordination
    between the two structures difficult. If, however, she or he becomes a member of this
    Board, that would assume higher professional competencies, which the current job
    descriptions do not require.
-   Due to the ambiguous status of the PCU, there are no clear requirements for its staff.


Conclusion:
The PCU is a contradictory structure:
- It is simultaneously a unit of the lender and the borrower;
- It attracts funding both from the MES (article 2, 4, ff. of the Regulations) and the World
   Bank (ibid.);
- It audits, administers and reports both to lender and borrower;
- It supervises its own operations;
- It duplicates the activity of the World Bank beneficiaries, which being in the same time
   administrator, coordinator and supervisor of the loan;
                                                                                            34
The Regulations on the Structure and the Operation of the Education Modernization
Project Coordination Unit in effect creates a structure, which duplicates functions of the
lender (the World Bank), the borrower (MES), and the beneficiaries (the other institutions
in the primary, general and higher education system). It could be said that the unclear legal
and administrative philosophy, vested in this document, is a serious obstacle to the future
implementation of the project.
The project is reduced to a series of procurement procedures, while the quality and the
impact of implemented activities are not being monitored.

     Problems associated with procurement and tender procedures:
An important part of the World Bank procedures, namely those guaranteeing the quality of
delivered goods and services, seem to be “forgotten.”1 For example:
    - Terms of reference should be developed by experts in the respective fields. Currently,
       these functions are performed by the working groups and the PCU. Although working
       group members are competent in education policy development, terms of reference for
       hiring consultants to conduct sociological research studies (for example) should be
       developed with the use of professional sociologists, which is not the case. We draw
       attention to this example because a key problem in education policy development, and
       hence the implementation of the project, is that it creates an impression of lack of
       transparency – activities are being undertaken without objective information on the
       parameters of the education system and without regard to and analysis of the opinion of
       stakeholders. There is only one occasion, in which the PCU has sought assistance from
       independent experts in developing terms of reference.
    - The procedures for structuring procurement commissions in case there is no specialist in
       the relevant field within the MES, have not been clarified. Experts have been involved in
       reviewing offers from a purely technical point of view on only one occasion. Offers need
       to be evaluated by at least three professionals in the relevant field.
    - There is no procedure for analyzing tender results even when procurement procedures
       have failed.
    - There is no procedure for evaluating the applicability of consultants’ services, i.e. the
        quality of services does not seem to be an issue. In one case (that of InfoArt) this issue
        was severely underestimated although there were quite a few indicators suggesting poor
        performance of project activities.
    - The consultants’ database is not based on clear principles, and no efforts are being made
        to systematically identify consultants.

    Problems arising from the lack of monitoring and evaluation mechanisms:
    - Apart from the indicators included in the project log-frame, there are no mechanisms for
       monitoring the impact of the project.
    - The monitoring and evaluation of the variety of project activities up to now has been
       manifested only in preparing plans and reports and conducting occasional research on
       some of the components. Although according to the Project Implementation Guide the
       PCU is supposed to include planning and monitoring experts, such activities have not
       been performed up to now.
    - There are no mechanisms requiring transparency and constant external supervision.


1
 Since the key functions of the PCU include conducting procurement and managing the finances of the project, this
activity will be described in detail in the report by Previs Consult.
                                                                                                              35
     Conclusion:
The PCU has in fact undertaken the overall implementation of the project including the
supporting activities, but also the control over the project’s progress and the PCU itself. No
evaluation of the PCU performance is envisaged. The overall design of the PCU implies that it
would parasitize on the successful activities of the MES units, and hence no mechanisms for
solving potential problems have been provided for – apart from reports to the Minister, which are
left unanswered.
The PCU is no position to control the implementation of project activities. The project is actually
being managed by the Minister, but the issue of controlling the quality of performance within the
working groups has not been resolved adequately. The actual implementation of project activities
is thus left to the working group leaders, which is an ineffective arrangement, given the fact that
they change too often. In order to perform its real functions, the PCU should be able to
adequately coordinate project activities. This, in turn, would require wider supervisory powers
but also wider competencies. Currently, the PCU has neither the powers, nor the competencies to
exercise control over the implementation of project activities.
The overall management of the project needs to be dramatically improved with a new
status of the unit directly responsible for its implementation.

       Recommendations:
General recommendations:
The key recommendations on improving project management, which could be made at this stage,
are associated with the need for the Ministry of Education and Science to guarantee an organic
execution of the reform process in education with clear goals, expected results, and mechanisms
for responding to problems and making adjustments:
     - Creating and fostering a common understanding among the MES employees, and
        especially the working group members, of the purpose, the philosophy and the activities
        of the project. This task should be undertaken by the MES leadership because that would
        legitimize the significance and the gravity of the education reform effort.
     - The Project Board should become more active and convene at least every three months;
        the Project Coordination Unit Director should also attend the Board’s sessions.
     - Reports to the Minister should be issued at least once a month to guarantee prompt and
        operative reaction;
     - Minutes should be kept of the Project Board’s sessions, and all decisions on current
        problems should be recorded;
     - Deadlines should be introduced for making and implementing decisions;
     - A system for internal and external monitoring of project activities should be introduced.
        Given the importance of this task and the lack of an appropriate unit at MES, an
        employee could be appointed at the Project Coordination Unit and activities associated
        with collecting and analyzing information could be planned. This, however, would
        require additional financial resources.
     - A structure of all working groups should be developed detailing their composition and
        leaders, the positions of their members, the rules of operation and subordination.
     - The job descriptions of the working group leaders and members should explicitly include
        their responsibilities on the project.
     - Project decisions need to be coordinated among the individual working groups.
     - A clear system for appointment, annual evaluation, and promotion of MES employees
        should be established.


                                                                                                36
    -   The interaction among the individual directorates within MES should be based on a
        common information system, use of statistical data, as well as information on the
        international practice in the various fields.

Recommendations on the different project management alternatives:
Option 1:
- To terminate the existence of the PCU as an independent legal entity because currently it
   does not meet initial expectations. I.e. to transfer the Project Coordination Unit within MES –
   to establish a new independent directorate at MES, which would be responsible for managing
   the funds allocated under the WB loan, administering and supervising activities, and
   reporting to higher authorities within the MES (Minister, Deputy Ministers, Chief Secretary),
   to the World Bank and to the public.
- The incorporation of the PCU as an independent directorate within the MES would achieve
   the following:
1. Clear responsibility for implementing all activities arising from the loan agreement. The
   contracting party under the loan agreement is the MES, and not some ostensible structure
   outside the state administration;
2. Effective subordination and coordination of activities;
3. Avoidance of the unnecessary dissipation of time and financial and human resources now
   invested in coordinating activities between the PCU and the MES;
4. Coordination of MES efforts and project activities;
5. Clear responsibility for implementing project tasks. The contracting party under the loan
   agreement is the MES, which through its independent directorate would bear both the assets
   of implemented activities, and the liabilities in case of failure to fulfill the loan agreement
   signed with the World Bank;
6. Elimination of the existing duplication of functions between the PCU and the MES. The
   funds of the loan are allocated to activities targeted to general and higher education, which is
   a key prerequisite of the MES. A parallel structure such as the PCU in its current status is
   totally unnecessary. The logic is quite simple: the recipient of the loan performs activities
   financed by this loan, benefits from the results of these activities and bears clear
   responsibility in case of failure to perform them.
7. The legal grounds for eliminating the PCU in its current status is that it contradicts the State
   Administration Act and the Public Officials Act. It is questionable whether the status of the
   PCU is consistent with the provisions of the Public Education Act and the Higher Education
   Act.

Option 2:
New status of the Project Coordination Unit based on the model introduced by the Ministry of
Labor and Social Policy. Temporary and small unit financed by the World Bank but with a
requirement for professional capacity. This means competitions for all positions, with high
criteria for professional competence. Elimination of all irrelevant activities of the PCU and clear
rules of interaction with component working groups and MES directorates, as well as
mechanisms for prompt response in case of serious problems. Linking the status of the unit with
the functions of the responsible Deputy Minister. Creating a public council to supervise the
activity of the unit – MES, Rectors’ Council, Student Council, trade unions, Open Society
Foundation - Sofia. The public council should issue public reports each month based on the
monthly reports of the unit.


                                                                                                37
Option 3:
 The Minister to delegate powers, but also responsibilities, to a competent employee of the
   Ministry with adequate administrative and organizational experience who would really
   supervise and manage the implementation of the project and would have the authority to
   impose sanctions for non-performance or poor performance to working group members;
 Clear and strict rules for submitting reports from the working groups to the person
   responsible for the project who would then summarize the information and report to the
   Minister;
 Clearly defined commitment of this person to project management with powers and
   responsibilities listed in detail in his or her job description;
 The selected employee would be directly responsible for observing project implementation
   deadlines set by the MES and the World Bank, and therefore should have the authority to
   impose sanctions for non-performance or poor performance;
 Since the project is not just part of the education reform effort, but its essence, the MES
   directorates should be directly involved in its implementation and project activities should
   become not an auxiliary but a basic responsibility.

General recommendations valid for the three options:
   - To regulate procurement and monitoring procedures.
   - To introduce a requirement for analyzing procurement and monitoring results.
   - To introduce transparency in the financial aspects of the project.
   - To maintain a working and regularly updated project web site.




Communication strategy of the project

     Goals:
   To develop and implement an effective media policy of the MES;
   To train MES employees how to present the reform in the media;
   To study public opinion in order to identify the public attitude towards prospective reforms
    in education;
   To involve a broad range of stakeholders in achieving the goals of the reform.

     Activities carried out up to now:
International consultant on developing and conducting a PR campaign:
 Terms of reference have been developed and a tender has been organized;
 A PR consultant was selected and a contract with COGNOS International was signed on
    March 11, 2002. The first visit of the consultant was organized April 15 through 25, 2002;
 The consultant has submitted a report on the visit;
 The report received negative response, which resulted in a delay of payment, as well as in
    suggestions to change the consultant.

National consultant on studying public opinion:
 Terms of reference have been developed and a tender has been organized;
 A contract for two national representative surveys on education reform was signed with
   InfoArt;

                                                                                              38
   The first survey has already been completed. The report is pending approval by the Minister.

     What is pending:
   Clarifying the situation with the international consultant;
   Publishing the report of the first national representative survey;
   Conducting the second national representative survey – 2004.

     Evaluation:
The implementation of the PR strategy is delayed. Measures should be taken to speed this
process up.
Technically the key activities under this component have been completed. Only the payment to
the consultants remains to be effectuated. However, there are problems associated with some
procedural deficiencies and the lack of criteria for selecting members of commissions
responsible for writing terms of reference and for evaluating the consultants’ services –
problems, which were already mentioned above.
In the case with the international consultant there are obviously discrepancies between the terms
of reference and the evaluation of what has been done. Consultants could be effective only if we
have made clear what we expect of them and we have studied in detail the specifics and the
parameters of the environment in which they would work. Otherwise, their output would be
adequate to the terms of reference, but not to the environment.

The importance of the public opinion survey should not be underestimated. There are a series of
procedural infractions in the procedure for selecting a consultant to study public opinion:
   - In short-listing candidates the commission has explicitly stated that this company has less
       experience than the others but this would not affect quality;
   - In three of the protocols the points awarded to the company have been increased;
   - The commission has awarded the maximum number of points for the company’s ability
       to conduct the survey, although it has specifically mentioned in its protocol that no
       timetable of activities had been submitted.

As a rule, it is unacceptable terms of reference in a specific field to be developed by a
commission, which does not include a single expert in this field. A representative of the
respective professional community should by all means participate in the commission following
strict confidentiality and conflict of interest rules. The lack of criteria in general affects the
quality of services provided under this key component.
A problem in both cases is the lack of a responsible body to evaluate the output of the
consultants, as well as the unclear authority in case of rejecting the product delivered by a
consultant.

The key problem is that public relations have not been included as a priority in the
implementation of project and are in fact missing as a procedure that could guarantee
efficiency of each activity. PR has not been provided for even when this was necessary for
the project, and could have been done. Since education reforms inevitably breed conflicts, the
communication with stakeholders and the general public is instrumental for their successful
implementation. Until now stakeholders have been ignored. Whenever their involvement has
been provided for in the PAD, it has been eliminated in the statutory documents. Whenever they
are supposed to play a leading role in a management body, this body never comes into being


                                                                                               39
(Curriculum Council, Higher Education Council). In fact, the MES has no strategy for
conducting a PR campaign and thus takes a defensive position towards the media.

     Recommendations:
   The PR strategy should become a priority and measures should be taken to publicize in the
    media both the goals of the project from a reform perspective, and the specific achievements
    that have been made – for example, the delegated budget system;
   The PR campaign should be pegged to an on-going monitoring of the impact of the project,
    and whenever problems arise, they should be discussed publicly in an active dialog with the
    stakeholders, rather than shrouded in silence.
   The MES staff should be trained how to present reform efforts, especially when they involve
    highly unpopular measures, in order to raise support from as many stakeholders as possible.
   There should be intensive coordination between the project PR expert and the MES Public
    Relations Department.
   The related MES Directorates should work in close cooperation with the project PR expert to
    guarantee adequate representation of the reform effort in the media.




Human resources and institutional memory

HUMAN RESOURCE
 The interviewed experts as a whole support the reforms in education.
However, there are different views on the extent to which reforms are necessary, on the ways in
which they should be implemented, and on the results that need to be achieved. The reform in
general is considered in view of:
 the specific component and the particular level of education – GE or HE;
 the overall change in the new circumstances;
 the quality of education;
 the need to bring education in line with the European standards.
Some experts see the reform as an on-going process of developments and changes consistent
with the dynamic environment and the dynamic needs, which have to be met. The reform is not
perceived just as a series of ostensible changes at the ministerial level, but as a results-oriented
effort. There is clear awareness of the problems that may arise in its implementation, as well as
of the fact that behind each possible view there are specific interests, which would breed
opposition, if they are not adequately accommodated. There is also understanding of the need to
involve stakeholders by publicizing expected results and explaining possible difficulties, i.e. by
implementing an active PR strategy and informing society at large.
In this respect there is within the MES real volition and real possibility to complete this
phase of the project and proceed to the next.

 In the same time there are problems:
 The irrational mobility, including that of working group leaders, has a negative impact on
  the overall performance and leads to delays in implementing project activities;
 The lack of “specific”, “targeted” (as respondents put it) training on the project for the
  people who are directly involved in its implementation;
 The inadequate information support to those who work on the project;
                                                                                                 40
   The lack of practice to actively seek information and study the preliminary analyses
    pertinent to the project;
   The inadequate knowledge of existing normative documents among some of the MES
    experts who should be clearly aware of:
    - the laws, which regulate the specific activities under a given component and in effect
        make these activities possible;
    - the documents and procedures directly associated with the World Bank requirements and
        the implementation of the loan agreement;
    - the legislative acts related to education in Bulgaria as a whole.
   The lack of coordination among the teams, working under the different components.

 There is a need to provide information on:
 each specific component – the fact that even those who work on the component need such
    information suggests that it has not been adequately provided;
 “the project as a whole” and the link among the different components – this explains why
    there is no coordination among components and why the overall vision of the project has
    been lost;
 the results of the project – i.e. feedback;
To ignore the need for information is to refuse to see, identify, and solve problems.


INSTITUTIONAL MEMORY

 Neither the MES, nor the PCU possess systematic and well-maintained archives on the
    project.
Since there are no “orientation” procedures and accessible information on what has been
done by the previous team, it is only natural that each change within the working groups
would affect the progress and the quality of work. And since there is no possibility to bring
new people in line with the work in the usual institutional way – by giving them access to the
archives – this either does not happen at all or is being done informally through “conversations
in the hallway”, but in all case to the detriment of the quality of performance. This, compounded
by the lack of clearly defined duties and responsibilities, blocks the implementation of activities
under the given component. As a rule, a bureaucratic structure in which all duties and
responsibilities have been clearly defined should work properly as if on autopilot, even when the
leader is absent. In our case this happens only if the composition of the team remains
unchanged, i.e. performance depends on the particular individual, and not on his or her
duties and responsibilities. This is evident by the fact that project activities have been
implemented on time under components where working groups have remained relatively stable
and the leader has been the same from the beginning – the Delegated Budget System and the
HECTMS (it would be interesting to see how would the HECTMS component evolve from now
on, since a new leader has been appointed).
Given the inadequate knowledge of existing normative documents and the lack of a systematic
archive of past activities, each change in the working groups’ composition has a negative impact
on the overall performance. This further intensifies the lack of coordination among the teams
working on different components, and may lead to self-isolation of components. This, in turn,
inevitably affects the overall implementation of the project because the individual components
are in fact closely interrelated and complement each other.


                                                                                                41
 Recommendations:
  - A seminar on the purpose and the goals of the project, the objectives of the reform, and
     the procedures of the World Bank should be organized for everyone working on the
     project. This is imperative in view of the overall analysis, evaluation and conclusions;
  - The responsibilities on the project should be included in the job descriptions of the
     people involved. This, along with well-developed evaluation criteria, would improve the
     motivation of those who work on the project. Meanwhile, additional incentives may be
     considered.
  - The interaction between the World Bank and the MES should be made more harmonious.
     There is a need to improve the knowledge of World Bank procedures. The Open Society
     Foundation – Sofia could help in this respect by organizing training seminars for MES
     experts.




                                                                                           42
                                      Key Conclusions


   The project is well developed from a conceptual and a technological point of view as an
    instrument for strengthening the management capacity of MES in pursuing its education
    reform policy, but there are a series of structural and substantive discrepancies in the project
    statutory documents.
 The breaking down of the education modernization process into separate components with
    specific activities that should be implemented is useful in view of the actual accomplishment
    of these activities and the ability to monitor and control them throughout the process. The
    problem however is not just for an activity to be effective by itself, but rather to guarantee its
    efficiency in correlation with all other activities. The goals of the project could be achieved
    only if there is effective coordination among the activities under the different components.
 The key impression, which monitored activities create, is that currently – two years after the
    start of the project – its cohesion has been lost and it has disintegrated into a series of
    independent sub-projects with their own specifics, which are progressing with a different
    pace and with a different degree of success. This disintegration and loss of cohesion is
    corroborated by:
      - the fact that implementation deadlines are not being met and activities are progressing
           with a different pace;
      - the lack of coordination activities between the different components;
      - the level of coordination among the direct participants;
      - the lack of information support, which has a negative impact on the whole project.
 The virtual lack of integration between the activities within different components indicates
    that the focus on the overall purpose and the strategic goals of the education system reform
    has been lost. An analysis of interviews shows that:
      - There are different visions on the goals of the reform and quite often no connection is
         made between individual activities;
      - There are some cases when the goals have been deliberately shifted away from those set
         in the project;
      - The very fact that a considerable part of those who work on the project consider it an
         external activity to their immediate work indicates that the project is not linked to
         significant and important problems of education, but is rather perceived as an additional
         burden. This results in activities being delayed or performed mechanically, pro forma.
The overall analysis shows that, in spite of all the efforts made, the implementation of the
individual components is inconsistent, which leads to the conclusion that the result at the
intermediary stage could hardly be estimated as satisfactory.

One cannot help but wondering why the reform effort is being wasted by not performing a series
of necessary activities, while implementing others, which are not integrated in the whole project
and seem an end in themselves.

Let’s see again which components are progressing and which are lagging behind.

The components, which are being implemented successfully, include:
- NES, standards-based curriculum and programs, students assessment standards
- Extension of the delegated budget system
- Higher Education Competitive Teaching and Management System
                                                                                                   43
The components, which are lagging behind include:
- Information system
- School Network Optimization
- Inspection system
- Overall resource management in higher education
- Student loan program
- Management capacity and PR

Progress is being made on components:
    which have been developed for some time and where experience and capacity have
      already been developed – Phare Program, Open Society Foundation – Sofia, activities
      under TEMPUS and SOCRATES Programs;
    where units have been already established or which are an intrinsic part of the MES
      activity – National General Education Evaluation Unit, Competitive Center for Higher
      Education Teaching and Management; General Education Resource Management
      System, new curriculum development;
    which are geared towards improving the quality of education;

All components, which require changes in the education management practices as a whole,
are lagging behind, i.e. components targeted towards improving management efficiency
and resource allocation.

On the whole, components involving changes in the management practices are falling back,
while those associated with the quality of teaching mark some progress. This means that issues
related to resource allocation and management, as well as institutional optimization are
considered separately from the issue of quality. In other words, if there are some changes in the
education system, they have not resulted from an improved management and efficiency,
but from something else. The question, however, is why quality is more responsive to change,
and management is not?
There are no mechanisms for external pressure to initiate changes in education
management – still no market pressures exist, and beneficiaries are perceived as a passive
recipient to whom decision-makers are supposed to provide high quality education and access to
education. Thus, problems in funding, resource allocation and management are left at the
discretion of decision-makers.
On the other hand, progress is made on components targeted to quality assurance, i.e.
directly to the beneficiaries themselves – national education standards and programs, student
assessment standards; delegated budgets (effective resource allocation in schools); Higher
Education Competitive Teaching and Management System, i.e. activities where quality
assurance dominates.

Why did changes in general education begin? What brought the obvious crisis in Bulgarian
schools? The low salaries of teachers severely affected their motivation, the material resources
deteriorated, and this had a dramatic impact on the quality of education. The changing social
environment further deepened the crisis. As new media and new models of socialization
emerged, children lost interest in schooling. This underlined the need to seek solutions. It is
obvious that programs have to updated and brought closer to the kids so that they could be
interested to go to school again. In this situation old time teaching is unacceptable, therefore
                                                                                              44
teacher training is needed. Schools are facing serious financial difficulties. Something needs to
be done – delegated budgets could provide some solution. Therefore, consciously or
unconsciously, the needs of the beneficiaries stimulate the change.

Higher education turned out to be more conservative because it is more protected from external
pressure. Universities have certain autonomy, they have academic prestige, students go there on
their own free will. Higher education has clearly defined social functions – it offers a two-way
solution to youth unemployment. On one hand, it postpones the problem – young people study
and do not apply for jobs. On the other hand, it “solves” it – when they graduate, they emigrate
abroad. Academic institutions still enjoy a high reputation and there is no pressure for change.
The only perceived need up to now is to introduce universality and overcome provincialism.
Therefore, the most powerful instrument for change is to bring more universal, European and
global, standard in higher education. This explains the success of the HECTMS – it is managed
by scientists with a European frame of mind, the teams, which participate in the competitive
scheme have experience with the Tempus and Socrates Programs.
The social environment, though the beneficiaries, does exert pressure for change.
Therefore, the reform should be carried out in a closer interaction with the beneficiaries.
Especially when the ultimate goal is decentralization.
This means that active dialog with the beneficiaries should become a norm for the MES.
Not only through an active PR campaign, but also through a permanent exchange in the
form of:
     on-going monitoring;
     initiation of public debates on education reform issues;
     a social background assessment not made just pro forma, but a real instrument to
        collect feedback.
Education management practices would not change if no pressure is exerted on decision-
makers. This pressure apart from the World Bank (and the international institutions) should
come from the beneficiaries themselves. The strategy of the Open Society Foundation – Sofia
should be targeted to involving beneficiaries in the reform effort.
This does not preclude the need to dramatically improve project management, enhance
coordination among the individual components, and implement the reform philosophy
incorporated in the project in its entirety.




                                                                                               45
            Key problems in the implementation of the project
                    and their structural background

The most critical problems in the implementation of the project are not associated with the lack
of adequate human resources or mobility, but rather with a lack of comprehensive understanding
or outright rejection of the project purpose and tasks, as well as with the external relations
between MES and the addressees of the reform. Therefore, the recurrent recommendation to hire
external consultants is inadequate, since until now the hiring of external consultants has not as a
whole contributed to solving the problems. Without a clearly defined problem situation one
cannot expect reform efforts to be effective. The refusal to recognize problems is gradually
emerging as a particular problem along with other challenges.

Most problems arise from the fact that the project for reforming education clashes with the
reality of the inertia-driven education system, which is still clinging to pre-democratization
society stereotypes. This clash can be traced at various levels and is manifested in different
forms:
- in the translation to Bulgarian of the project documents, which have been agreed upon by the
    two parties;
- in the discrepancies between the concept of the initial documents and the wording of the acts,
    which provide the legislative framework for the implementation of the project;
- in the structural drawbacks predetermined by the very design of the project (detailed
    itemization of activities, which reduces project implementation to a technology, thus creating
    a possibility for imitative performance by reporting technical activities without considering
    their actual content);
- the paradoxical situation of MES which should centrally decentralize itself;
- the considerable risk could be avoided through a careful monitoring of the impact of project
    activities, but monitoring has not been adequately developed as a project activity;
- the delay of key activities due to the resistance of the institutional culture (typical examples
    include Student Loans and Information System components);
- the underestimated need to create adequate legislative background (as if the existing
    legislative framework predisposes to reforms, and not to a replication of the existing status);
- mutual misunderstanding of alternative views among stakeholders.


Let us discuss some of the manifestations of the opposition of the social environment:

 The translation:
As a whole the translation is not only precise but very good given the highly professional
specifics of the text. This however means that wherever discrepancies exist, they do not result
from mistakes, misunderstanding or unintentional oversight. “The misunderstanding” is the
result of a conflict of meanings. The discrepancies in the interpretation are remarkably
symptomatic for the neuralgic points in the education system and its management – hence the
management and implementation of the project. The interpretations in the Bulgarian
translation result from the cultural opposition of our social environment to a reform-
oriented project.

                                                                                                46
The alteration of meanings has happened unconsciously in the process of translation so it should
not be surprising that in the normative documents, which institutionalize the supporting bodies
responsible for the implementation of the project, the concept of modernization is even more
pronouncedly “Bulgarianized”.

 Deviations in the normative documents:
In all government decrees for establishing different project units, signed by the Minister of
Education and Science Prof. D. Dimitrov the phrase “education modernization” is replaced by
“education improvement.” The fact that “modernization” is changed to “improvement” suggests
a conceptual shift in the concept of the reform, which might have far-reaching consequences.
Such changes in the wording exist in many other cases.2

 The hidden structural defects:
   Following its practice in countries with no experience in conducting social reforms, the
     World Bank has developed a project, the implementation of which is very precisely
     broken down into activities with specific tasks, timetables, executors, performance
     indicators and control mechanisms. It seems that all comes down to just following the
     prescription. However, this strictly prescribed series of activities, which reduces project
     implementation to an operational technology, creates possibilities for imitative
     performance, that is reporting technical activities with no regard to their content. Thus,
     the people involved in the implementation of the project are prone to introduce their own
     interpretations of the concept – a tendency, which is intensified by the fact that they are
     not familiar with and do not use the project documentation properly. They do not use it
     because it refers to the overall project, and they have problems implementing their
     specific activities in the inertia-driven system;
   Taking intro consideration the lack of experience and capacity, the project relies on
     external consultants, and then on internal institution building. But it is the same people
     who have no capacity to implement the activities that hire consultants and approve their
     output without being competent enough to judge whether it would be effective or not. No
     mechanisms for assessing the effectiveness of consultancy are established. As if the
     whole activity ends with the hiring of a consultant.
   The risks have been very precisely calculated in the project, but all challenges have been
     evaluated as external to the education system and the reform process. Hence, the risks are
     expected to be limited through careful monitoring, on-going active dialog, and
     involvement of the beneficiaries. However, the performance indicators of the individual
     activities as a rule do not include the beneficiaries. It is not incidental that the
     communication strategy (PR) is one of the most underdeveloped activities.
   Although monitoring is a key self-control activity, it has not been adequately developed
     in the project and in practice does not exist because it has not been specifically defined as
     a responsibility and a procedure.
   One is left with the impression that the project would progress by itself because the key
     MES units would commit to its implementation because it corresponds to their main
     scope of activity. This, however, is not the case: the MES has its own bureaucratic logic,

2
 The decree with which the HECTMS was established is quite indicative – changes in the wording have resulted in
excluding the stakeholders from the process of structuring the institution and determining its priorities. The fact that
modernization is perceived as an unclear improvement of the old system, rather than a change in the principles, is
probably the reason for not establishing the Higher Education Council as a key management body.

                                                                                                                     47
       which does not always include education reform activities since there are targeted to
       changing, rather than preserving the status quo.

FACTORS DETERMINING THE EXISTENCE OF PROBLEMS:
 The overestimated potential for change in society and within MES, and the underestimated
  resistance (both among the addressees and within MES);
 The problems with the organizational culture;
 The fact that beneficiaries are not actively involved in the reform effort and there is no
  communication with the public in the process of implementing the project;
 The fact that there is no clear understanding of the implicit systemic link among the
  individual defects in the education practice;
 The fact that the implications of the old social system are not taken into account, nor are the
  specifics of the new system that is being established in the transition period;
 The tradition for strong social protection on behalf of the State, as a result of which education
  modernization becomes subjected to short-term goals motivated by social concerns;
 The underestimated institutional resistance – institutions ostensibly undertake certain tasks,
  which are later on reformulated;
 The researched period is marked by fragmentary management decisions, lack of overall
  vision and coherent strategy, inadequate link between general and higher education
  institutions, lack of substantiated policy, lack of transparency and accountability, adherence
  to the statist model;
 The fact that the project requires universal support but does not provide mechanisms for
  overcoming opposition.

These problems would not be solved without being recognized and clearly formulated.




                                                                                                48
                                    Recommendations


1. New priorities of the institutional capacity of MES:
   А) Development and active implementation of the information system:
 The on-going reorganization of information systems – the creation of a working group on
   general education, a working group on higher education and a third unit to coordinate their
   activities – should be completed as soon as possible to help make up for the lost time.
 The team working on this component should cooperate closely with the working groups of
   all other components, requesting and systematizing the information that has been collected
   by them.
 The subordination between the Information Technologies Directorate at MES, the Institute
   on Education, the General Education Resource Management System, and the prospective
   new unit should be made clear.
 Criteria for management efficiency should be introduced. Decision-making should be based
   on reliable information for the current status, rather than on arguments such as “my
   experience shows”.
 The indicators on which information is collected should include the widest possible range of
   data to allow for monitoring the dynamics of the education system in relation to the other
   elements of the social environment. Data should be collected on municipalities (social and
   financial), but also on general and higher education institutions; research studies on the
   developments in education; the system for monitoring the implementation of reform
   activities; databases on national education standards, teaching standards, and student
   assessment standards; results from the general exams; legislative documents, which have a
   bearing on education (not only those targeted specifically to education); list of experts in this
   field, etc.;
 The external and internal monitoring of the implementation of education reform activities
   should become an essential part of the information system.
 An information database on sociological surveys in education should be created and
   systematized to allow for monitoring developments in education;
 The information collected by the Project Coordination Unit and the newly established
   information system for general and higher education should be made publicly available.

    B) Development and implementation of an active communication strategy:
   The implementation of the project should continue in close partnership with the beneficiaries
    of the reform;
   The cooperation with trade unions on all reform-related issues should be extended;
   The PR strategy should become a priority and measures should be taken to publicize in the
    media both the goals of the project from a reform perspective, and the specific achievements
    that have been made – for example, the delegated budget system;
   The PR campaign should be pegged to an on-going monitoring of the impact of the project,
    and whenever problems arise, they should be discussed publicly in an active dialog with the
    stakeholders, rather than shrouded in silence.
   The MES staff should be trained how to present reform efforts, especially when they involve
    highly unpopular measures, in order to raise support from as many stakeholders as possible.


                                                                                                 49
   There should be intensive coordination between the project PR expert and the MES Public
    Relations Department.
   The related MES Directorates should work in close cooperation with the project PR expert to
    guarantee adequate representation of the reform effort in the media.

2. Specification of management responsibilities and improvement of the new management
    structure:
       А)       New management structure:
Three options of the management structure:
1) To transfer the Project Coordination Unit within MES – to establish a new independent
directorate at MES, which would be responsible for managing the funds allocated under the WB
loan, administering and supervising activities, and reporting to higher authorities within the MES
(Minister, Deputy Ministers, Chief Secretary), to the World Bank and to the public.
2) New status of the Project Coordination Unit: Temporary and small unit financed by the World
Bank but with a requirement for professional capacity. Clear rules of interaction with component
working groups and MES directorates, as well as mechanisms for prompt response in case of
serious problems. Linking the status of the unit with the functions of the responsible Deputy
Minister. Creating a public council to supervise the activity of the unit – MES, Rectors’ Council,
Student Council, trade unions. The public council should issue public reports each month based
on the monthly reports of the unit.
3) Real project coordinator with ministerial mandate to manage the project and sanction
activities with a view to improving project management scheme and achieving effective
coordination among the different components. Clearly defined authority and responsibilities both
for the project leader, and for everyone involved in the project implementation. Strict
accountability of the project leader to the Minister, the task forces and the stakeholders to
guarantee up-to-date information on the actual progress of project activities and the problems in
their implementation.

It is also necessary:
        To regulate bidding and monitoring procedures.
        To introduce a requirement for analyzing bidding and monitoring results.
        To maintain a working and regularly updated project web site.


    B) Specification of management responsibilities
    - Creating and fostering a common understanding among the MES employees, and
       especially the working group members, of the purpose, the philosophy and the activities
       of the project. This task should be undertaken by the MES leadership because that would
       legitimize the significance and the gravity of the education reform effort.
    - The Project Board should become more active and convene at least every three months;
       the Project Coordination Unit Director should also attend the Board’s sessions.
    - Reports to the Minister should be issued at least once a month to guarantee prompt and
       operative reaction;
    - Minutes should be kept of the Project Board’s sessions, and all decisions on current
       problems should be recorded;
    - Deadlines should be introduced for making and implementing decisions;
    - A system for internal and external monitoring of project activities should be introduced.
       Given the importance of this task and the lack of an appropriate unit at MES, an
                                                                                               50
        employee could be appointed at the Project Coordination Unit and activities associated
        with collecting and analyzing information could be planned. This, however, would
        require additional financial resources.
    -   A structure of all working groups should be developed detailing their composition and
        leaders, the positions of their members, the rules of operation and subordination.
    -   The job descriptions of the working group leaders and members should explicitly include
        their responsibilities on the project.
    -   Project decisions need to be coordinated among the individual working groups.
    -   A clear system for appointment, annual evaluation, and promotion of MES employees
        should be established.
    -   The interaction among the individual directorates within MES should be based on a
        common information system, use of statistical data, as well as information on the
        international practice in the various fields.

3. Analysis of project components in their relation to one another and coordination of
   planned activities:
 The coordination among components should be improved to achieve more efficient
   implementation of the project goals and the purpose of the reform;
 Periodical joint sessions of the working groups with a view to improving coordination.
 A seminar on the purpose and the goals of the project, the objectives of the reform, and the
   procedures of the World Bank should be organized for everyone working on the project.
   Such a seminar could be financed by the Open Society Foundation – Sofia.

4. Identifying actual problems in order to overcome the missing link between general and
   higher education:
 Improving the coordination of activities, which build direct links between general and higher
   education – effective functioning of the information system, in-service teacher training at
   universities with a view to improving the quality of teaching, unified national assessment
   system as a prerequisite for admission to higher education institutions;
 Clarification and coordination of general and higher education reform strategies – developing
   a general plan for implementing project goals with a view to accomplishing the education
   reform mission.

5. Link between the project activities and the actual needs of the education strategy:
 On-going monitoring of the implementation of the new programs and assessment standards;
   monitoring of schools where the delegated budget system is being implemented with a view
   to identifying the impact it has on management efficiency and the quality of education;
   monitoring the impact of projects solicited under the Higher Education Competitive
   Teaching and Management System;
 In-service teacher training consistent with the needs of the reform;
 Effective formulas for school network optimization;
 Identifying the efficiency, role, and functions of inspectorates in view of the strategic goals
   of the education modernization effort. Developing a system for teacher assessment.

6. Improving the human resource capacity of MES:
 Introducing strict institutional procedures – job descriptions, procedures for nominating
   working group members and expert committee members, procedures for implementing
   solicited products.
                                                                                               51
   Introducing the practice of involving external experts from the relevant professional
    communities in the process of developing terms of reference and evaluating solicited
    products.

7. Managing the discrepancy between market orientation and social concerns. It should
   be kept in mind that the quality of education is not a problem for the social protection
   system to solve.

8. The legislative framework should be made consistent with the goals of the reform. Where
   regulations are obsolete or do not exist, they should be revised accordingly. To achieve
   this MES should launch an active PR campaign in order to raise broad public support
   and involve stakeholders.

9. An on-going independent monitoring of project implementation should be organized to
   guarantee consistency with the goals of the reform and the needs of the education
   beneficiaries.




                                                                                         52
                        EXECUTIVE SUMMARY

 The Education Modernization Project is being financed by a World Bank loan. The
  loan agreement was signed between the Government of the Republic of Bulgaria and the
  World Bank on September 19, 2000, was ratified by the National Assembly on December
  12, 2000, and entered into force on January 10, 2001. The loan amounts to a total of 62
  million US dollars and is an Adaptable Program Loan. The implementation period is nine
  years, divided into three three-year phases.

The goal of the project is:
 To shift from expense-based resource allocation to an investment-based funding
   mechanism that would imply return on invested funds. The purpose of modernization is
   to make Bulgarian education system effective, and above all, flexible – capable of
   adapting to and accommodating the changing social environment.
 The essential feature of the EMP is that it is designed to create structural mechanisms,
   which would help build an education system of new quality. Therefore, the congruity
   between the separate project activities is a key prerequisite for its success. In order to
   prepare future students for the requirements of a competitive higher education system,
   open and responsive to their professional orientations, the school should produce active
   and independent personalities, rather than just carriers of knowledge with a
   passive/receptive attitude to life. This implies not only changes in the content and
   organization of school education, but also reforms in teacher training. That, in turn,
   requires financial resources, which would be secured by improving school management
   through an extension of the delegated budget system and an optimization of the school
   network. Under the new conditions the inspectorates would have a completely new role.
   This efficiency-oriented principle in spending, and above all, managing budget
   allocations on education would be applied in higher education, too, by introducing a
   funding mechanism based on a normative cost formula, rather than on historical patterns.
   The project proposes a mechanism that would prepare higher education institutions for a
   competitive academic environment by 1) creating internal institutional mechanisms for
   quality control, and responsible and transparent management; 2) encouraging
   amalgamation of academic institutions in order to improve resource management
   efficiency, as well as introducing a mechanism for involving students in the process of
   funding higher education, which would increase demands for a high-quality education.

  The Project Monitoring and Evaluation of Education Modernization Project
   Activities is being financed by the Open Society Foundation – Sofia, and is being
   implemented by the Association for Social Investigations and Applied Research
   Practices and the subcontractor, Previs Consult Agency, responsible for the financial
   analysis of the project. The implementation period is 21 months, starting July 1, 2002;
   each phase of the monitoring will be accounted with reports issued every three months.

 The goal of the project at the current stage is:
To evaluate the relevance between the goals set in the Education Modernization Project and
the actual results achieved at the ministerial level in the period since the start of the project,
January 10, 2001–September 30, 2002. All activities included in the MES schedule have been
monitored.
                                                                                               53
       Tasks:
      1. Description, analysis, and evaluation of implemented activities under the Education
         Modernization Project:
      2. Evaluation of the institutional conditions for implementing project activities;
      3. Analysis and evaluation of the level of implementation of project goals:
      - Institutional resources for implementing the project at the MES level;
      - Project management;
      - Human resources;
      - Information resources;
      - Funds allocation;
      - Public awareness of the project.
      4. Recommendations for optimizing activities.

          Methods applied:
          - In-depth interviews
          - Systematization of PCU Archive and Analysis of EMP Documents
          - Expert Evaluations
          - Interviews
          - Participatory Observation in PCU.

                                                         *    *        *


               Component Structure of the Education Modernization Project

The project is developed in a series of documents and annexes, the key of which are the Project
Appraisal Document of August 7, 2000 (PAD) and the Education Modernization Project of 2001
(EMP). The project components are presented in a different way in these documents – the
objectives of the reform are listed with Roman numerals as they have been formulated in the first
document, while in the second document these objectives have been operationalized into
components, listed with Arabic numerals.

І.        Creating conditions for improving the quality of teaching and learning in general
          education:
          3. National education standards, standards-based curriculum and programs, student
             assessment, inspection system
          4. In-service teacher training

ІІ.       Creating conditions for improving overall resource management in general
          education:
          7. Extension of the delegated budget program
          8. Optimizing the school network
          9. General Education Management Information System

ІІІ. Creating conditions for improving overall resource management in higher education:


    The financial analysis is provided by the Previs-Consult Agency.
                                                                                              54
       - Reform of allocation processes for resources and seats
       - Establishment within MES of an automated higher education management information
         system
       10. Improving efficiency in allocating resources to higher education

ІV. Establishing a student loan program and restructuring the existing student stipend
    system:
    8. Retaining the accessibility to higher education in Bulgaria

V.     Creating a competitive fund for improving teaching and resource management in
       HE:
       9. Improve the quality of teaching and the management of higher education institutions

VІ. Strengthening MES capacity for project management and communication:
    - Establishing a PCU
    - Developing and conducting a PR campaign.

     Key conclusions:
    The project is well developed from a conceptual and a technological point of view as an
     instrument for strengthening the management capacity of MES in pursuing its education
     reform policy, but there are a series of structural and substantive discrepancies in the project
     statutory documents.
    The breaking down of the education modernization process into separate components with
     specific activities that should be implemented is useful in view of the actual accomplishment
     of these activities and the ability to monitor and control them throughout the process. The
     problem however is not just for an activity to be effective by itself, but rather to guarantee its
     efficiency in correlation with all other activities. The goals of the project could be achieved
     only if there is effective coordination among the activities under the different components.
    The key impression, which monitored activities create, is that currently – two years after the
     start of the project – its cohesion has been lost and it has disintegrated into a series of
     independent sub-projects with their own specifics, which are progressing with a different
     pace and with a different degree of success. This disintegration and loss of cohesion is
     corroborated by:
-    the fact that implementation deadlines are not being met and activities are progressing with a
     different pace;
-    the lack of coordination activities between the different components;
-    the level of coordination among the direct participants;
-    the lack of information support, which has a negative impact on the whole project.
    The virtual lack of integration between the activities within different components indicates
     that the focus on the overall purpose and the strategic goals of the education system
     reform has been lost. An analysis of interviews shows that:
-    There are different visions on the goals of the reform and quite often no connection is made
     between individual activities;
-    There are some cases when the goals have been deliberately shifted away from those set in
     the project;
-    The very fact that a considerable part of those who work on the project consider it an
     external activity to their immediate work indicates that the project is not linked to significant


                                                                                                    55
    and important problems of education, but is rather perceived as an additional burden. This
    results in activities being delayed or performed mechanically, pro forma.
The overall analysis shows that, in spite of all the efforts made, the implementation of the
individual components is inconsistent, which leads to the conclusion that the result at the
intermediary stage could hardly be estimated as satisfactory.

The components, which are being implemented successfully include:
- NES, standards-based curriculum and programs, students assessment standards
- Extension of the delegated budget system
- Higher Education Competitive Teaching and Management System
The components, which are lagging behind include:
- Information system
- School Network Optimization
- Inspection system
- Overall resource management in higher education
- Student loan program
- Management capacity and PR

One is left with the impression that progress is being made on components:
    which have been developed for some time and where experience and capacity have
        already been developed – PHARE Program, Open Society Foundation – Sofia, activities
        under TEMPUS and SOCRATES Programs;
    where units have been already established or which are an intrinsic part of the MES
        activity – National General Education Evaluation Unit, Competitive Center for Higher
        Education Teaching and Management; General Education Resource Management
        System, new curriculum development;
    which are geared towards improving the quality of education;
On the whole, the general education system seems more flexible to change than the higher
education system.

All components, which require changes in the education management practices as a whole, are
lagging behind, i.e. components targeted towards improving management efficiency and
resource allocation.

This means that issues related to resource allocation and management, as well as institutional
optimization are considered separately from the issue of quality.

There are no mechanisms for external pressure to initiate changes in education
management – still no market pressures exist, and beneficiaries are perceived as a passive
recipient to whom decision-makers are supposed to provide high quality education and access to
education. Thus problems in funding, resource allocation and management are left to the
discretion of the employees at the relevant MES departments.
On the other hand, progress is made on components targeted to quality assurance, i.e. directly to
the beneficiaries themselves – national education standards and programs, student assessment
standards; delegated budgets (effective resource allocation in schools); Higher Education
Competitive Teaching and Management System, i.e. activities where quality assurance
dominates.
It is obvious that the social environment, though the beneficiaries, does exert pressure for

                                                                                               56
change. Therefore, the reform should be carried out in a closer interaction with the
beneficiaries. Especially when the ultimate goal is decentralization.
This means that active dialog with the beneficiaries should become a norm for the MES.
Not only through an active PR campaign, but also through a permanent exchange in the
form of:
    on-going monitoring;
    initiation of public debates on education reform issues;
Social background assessment should not be made just pro forma, but should become a
real instrument to collect feedback.
Education management practices would not change if no pressure is exerted on decision-
makers. This pressure apart from the World Bank (and the international institutions) should
come from the beneficiaries themselves. The strategy of the Open Society Foundation – Sofia
should be targeted to involving beneficiaries in the reform effort.
This does not preclude the need to dramatically improve project management, enhance
coordination among the individual components, and implement the reform philosophy
incorporated in the project in its entirety.

Key problems in the implementation of the project:
The most critical problems in the implementation of the project are not associated with the lack
of adequate human resources or mobility, but rather with a lack of comprehensive understanding
or outright rejection of the project purpose and tasks, as well as with the external relations
between MES and the addressees of the reform. Therefore, the recurrent recommendation to hire
external consultants is inadequate, since until now the hiring of external consultants has not as a
whole contributed to solving the problems. Without a clearly defined problem situation one
cannot expect reform efforts to be effective. The refusal to recognize problems is gradually
emerging as a particular problem along with other challenges.
Most problems arise from the fact that the project for reforming education clashes with the
reality of the inertia-driven education system, which is still clinging to pre-democratization
society stereotypes. This clash can be traced at various levels and is manifested in different
forms:
- in the translation to Bulgarian of the project documents, which have been agreed upon by the
    two parties;
- in the discrepancies between the concept of the initial documents and the wording of the acts,
    which provide the legislative framework for the implementation of the project;
- in the structural drawbacks predetermined by the very design of the project (detailed
    itemization of activities, which reduces project implementation to a technology, thus creating
    a possibility for imitative performance by reporting technical activities without considering
    their actual content);
- the paradoxical situation of MES which should centrally decentralize itself;
- the considerable risk could be avoided through a careful monitoring of the impact of project
    activities, but monitoring has not been adequately developed as a project activity;
- the delay of key activities due to the resistance of the institutional culture (typical examples
    include Student Loans and Information System components);
- the underestimated need to create adequate legislative background (as if the existing
    legislative framework predisposes to reforms, and not to a replication of the existing status);
- mutual misunderstanding of alternative views among stakeholders.

Key challenges to the project:
                                                                                                57
   The overestimated potential for change in society and within MES, and the underestimated
    resistance (both among the addressees and within MES);
   The concealed problems arising from the organizational culture;
   The tradition for strong social protection on behalf of the State, as a result of which education
    modernization becomes subjected to short-term goals motivated by social concerns;
   The underestimated institutional resistance – institutions ostensibly undertake certain tasks,
    which are later on reformulated;
   The fact that there is no clear understanding of the implicit systemic link among the
    individual defects in the education practice;
   The researched period is marked by fragmentary management decisions, lack of overall
    visions and coherent strategy, inadequate link between general and higher education
    institutions, lack of substantiated policy, lack of transparency and accountability, adherence
    to the statist model;
   The fact that beneficiaries are not actively involved in the reform efforts and there is no
    communication with the public in the process of implementing the project;
   The fact that the project requires universal support but does not provide mechanisms for
    overcoming opposition.

Key recommendations:

10. New priorities of the institutional capacity of MES:
А) Development and active implementation of the information system:
     The on-going reorganization of information systems – the creation of a working group on
      general education, a working group on higher education and a third unit to coordinate
      their activities – should be completed as soon as possible to help make up for the lost
      time.
     The team working on the Information System component should cooperate closely with
      the working groups of all other components, requesting and systematizing the
      information that has been collected by them.
     The subordination between the Information Technologies Directorate at MES, the
      Institute on Education, the General Education Resource Management System, and the
      prospective new unit should be made clear.
     Criteria for management efficiency should be introduced. Decision-making should be
      based on reliable information for the current status, rather than on arguments such as “my
      experience shows”.
     The indicators on which information is collected should include the widest possible range
      of data to allow for monitoring the dynamics of the education system in relation to the
      other elements of the social environment. Data should be collected on municipalities
      (social and financial), but also on general and higher education institutions; research
      studies on the developments in education; the system for monitoring the implementation
      of reform activities; databases on national education standards, teaching standards, and
      student assessment standards; results from the general exams; legislative documents,
      which have a bearing on education (not only those targeted specifically to education); list
      of experts in this field, etc.;
     The external and internal monitoring of the implementation of education reform activities
      should become an essential part of the information system.
     An information database on sociological surveys in education should be created and
      systematized to allow for monitoring developments in education;
                                                                                                  58
      The information collected by the Project Coordination Unit and the newly established
       information system for general and higher education should be made publicly available;

B) Development and implementation of an active communication strategy:
 The implementation of the project should continue in close partnership with the beneficiaries
   of the reform;
 The cooperation with trade unions on all reform-related issues should be extended;
 The PR strategy should become a priority and measures should be taken to publicize in the
   media both the goals of the project from a reform perspective, and the specific
   achievements that have been made – for example, the delegated budget system;
 The PR campaign should be pegged to an on-going monitoring of the impact of the project,
   and whenever problems arise, they should be discussed publicly in an active dialog with
   the stakeholders, rather than shrouded in silence.
 The MES staff should be trained how to present reform efforts, especially when they
   involve highly unpopular measures, in order to raise support from as many stakeholders as
   possible.
 There should be intensive coordination between the project PR expert and the MES Public
   Relations Department.
 The related MES Directorates should work in close cooperation with the project PR expert to
   guarantee adequate representation of the reform effort in the media.

11. Specification of management responsibilities and improvement of the new management
    structure:

А) New management structure
Three options of the management structure:
a) To transfer the Project Coordination Unit within MES – to establish a new independent
directorate at MES, which would be responsible for managing the funds allocated under the WB
loan, administering and supervising activities, and reporting to higher authorities within the MES
(Minister, Deputy Ministers, Chief Secretary), to the World Bank and to the public.
b) New status of the Project Coordination Unit: Temporary and small unit financed by the World
Bank but with a requirement for professional capacity. Clear rules of interaction with component
working groups and MES directorates, as well as mechanisms for prompt response in case of
serious problems. Linking the status of the unit with the functions of the responsible Deputy
Minister. Creating a public council to supervise the activity of the unit – MES, Rectors’ Council,
Student Council, trade unions. The public council should issue public reports each month based
on the monthly reports of the unit.
c) Real project coordinator with ministerial mandate to manage the project and sanction activities
with a view to improving project management scheme and achieving effective coordination
among the different components. Clearly defined authority and responsibilities both for the
project leader, and for everyone involved in the project implementation. Strict accountability of
the project leader to the Minister, the task forces and the stakeholders to guarantee up-to-date
information on the actual progress of project activities and the problems in their implementation.

It is also necessary:
To regulate bidding and monitoring procedures.
To introduce a requirement for analyzing bidding and monitoring results.
To maintain a working and regularly updated project web site.
                                                                                               59
B) Specification of management responsibilities
    - Creating and fostering a common understanding among the MES employees, and
       especially the working group members, of the purpose, the philosophy and the activities
       of the project. This task should be undertaken by the MES leadership because that would
       legitimize the significance and the gravity of the education reform effort.
    - The Project Board should become more active and convene at least every three months;
       the Project Coordination Unit Director should also attend the Board’s sessions.
    - Reports to the Minister should be issued at least once a month to guarantee prompt and
       operative reaction;
    - Minutes should be kept of the Project Board’s sessions, and all decisions on current
       problems should be recorded;
    - Deadlines should be introduced for making and implementing decisions and in case of
       delay specific sanctions should be imposed;
    - A system for internal and external monitoring of project activities should be introduced.
       Given the importance of this task and the lack of an appropriate unit at MES, an
       employee could be appointed at the Project Coordination Unit and activities associated
       with collecting and analyzing information could be planned. This, however, would
       require additional financial resources.
    - A structure of all working groups should be developed detailing their composition and
       leaders, the positions of their members, the rules of operation and subordination.
    - The job descriptions of the working group leaders and members should explicitly include
       their responsibilities on the project.
    - Project decisions need to be coordinated among the individual working groups.
    - A clear system for appointment, annual evaluation, and promotion of MES employees
       should be established.
    - The interaction among the individual directorates within MES should be based on a
       common information system, use of statistical data, as well as information on the
       international practice in the various fields.

    3. Analysis of project components in their relation to one another and coordination of
        planned activities:
   The coordination among components should be improved to achieve more efficient
    implementation of the project goals and the purpose of the reform;
   Periodical joint sessions of the working groups with a view to improving coordination.
   A seminar on the purpose and the goals of the project, the objectives of the reform, and the
    procedures of the World Bank should be organized for everyone working on the project.
    Such a seminar could be financed by the Open Society Foundation – Sofia.

    4. Identifying actual problems in order to overcome the missing link between general
        and higher education:
   Improving the coordination of activities, which build direct links between general and higher
    education – effective functioning of the information system, in-service teacher training at
    universities with a view to improving the quality of teaching, unified national assessment
    system as a prerequisite for admission to higher education institutions;
   Clarification and coordination of general and higher education reform strategies – developing
    a general plan for implementing project goals with a view to accomplishing the education
    reform mission;
                                                                                              60
    5. Link between the project activities and the actual needs of the education strategy:
   On-going monitoring of the implementation of the new programs and assessment standards;
    monitoring of schools where the delegated budget system is being implemented with a view
    to identifying the impact it has on management efficiency and the quality of education;
    monitoring the impact of projects solicited under the Higher Education Competitive
    Teaching and Management System;
   In-service teacher training consistent with the needs of the reform;
   Effective formulas for school network optimization to guarantee equal access to and quality
    of education;
   Identifying the efficiency, role, and functions of inspectorates in view of the strategic goals
    of the education modernization effort. Developing a system for teacher assessment.

    6. Improving the human resource capacity of MES:
   Introducing strict institutional procedures – job descriptions, procedures for nominating
    working group members and expert committee members, procedures for implementing
    solicited products.
   Introducing the practice of involving external experts from the relevant professional
    communities in the process of developing terms of reference and evaluating solicited
    products.

    7. Managing the discrepancy between market orientation and social concerns. It
       should be kept in mind that the quality of education is not a problem for the social
       protection system to solve.

    8. The legislative framework should be made consistent with the goals of the reform.
       Where regulations are obsolete or do not exist, they should be revised accordingly.
       To achieve this MES should launch an active PR campaign in order to raise broad
       public support and involve stakeholders.

    9. An on-going independent monitoring of project implementation should be organized
       to guarantee consistency with the goals of the reform and the needs of the education
       beneficiaries.




                                                                                                61

				
DOCUMENT INFO