Docstoc

Quality Assurance in Higher Education

Document Sample
Quality Assurance in Higher Education Powered By Docstoc
					             Quality Assurance in Higher Education
A Legislative Review and Needs Analysis of Developments in
Central and Eastern Europe


Opinions     expressed       and   arguments    employed     in   this
publication are the views and responsibility of the authors
and do not necessarily reflect those of the European
Commission, the European Training Foundation or the
governments      of    its   member     countries    or   beneficiary
countries.
Preface
This report has been prepared within the framework of the
European Union's Phare Multi-Country Programme in Higher
Education:   Quality    Assurance      in   Higher   Education.   The
Programme is part of the assistance provided by the European
Union, and managed by the European Training Foundation, to
countries of central and eastern Europe in their preparation for
accession to the European Union. The project is supported for the
European Union by a Programme Co-ordination Unit in Bratislava
on behalf of the European Training Foundation. The report
represents the first output of a project being undertaken by a
higher education consortium comprising the Centre for Higher
Education Policy Studies - CHEPS (the Netherlands), the Centre
for Quality Assessment in Higher Education (Lithuania), the Centre
for Evaluative and Strategic Studies (Slovenia), the Comité
National d'Evaluation (France), Evalueringscenteret (Denmark),
QSC: Centre for Higher Education Research and Information
(formerly known as the Quality Support Centre) (UK). Other
activities of the project include a series of 29 pilot evaluations of


                                   1
higher education programmes and institutions in 12 Phare
countries, seminars and training events to promote good practice,
and the preparation of a manual on quality assurance procedures
which is intended for widespread use in higher education
institutions in central and eastern Europe.
The report is in two parts. Part One is a Legislative Review. This is
a comparative review of quality assurance and accreditation
systems within national legislation in the field of higher education
in 12 countries in central and eastern Europe, including
comparative references to EU legislation. Part Two of the report is
a Needs Analysis which specifies the measures necessary and the
scope for common actions, within a multi-country context, to
enhance the compatibility and sustainability of existing quality
assurance/accreditation systems and to ensure that new systems
benefit fully from European experience and best practice. Several
of the needs which are identified will be met by further activities of
the Phare Multi-Country Quality Assurance Project. Others require
responses within individual countries or over a longer timescale.
Needs have been identified by the Consortium's experts following
visits to the countries concerned and taking into account needs
and wishes stated by education ministries and other interested
parties within the countries. Prior to publication, the draft report
was sent to representatives of the beneficiary countries for their
final comment.
The report also contains a series of annexes detailing information
on developments in individual countries of central and eastern
Europe. Higher education quality assurance is a fast developing
area in these countries, in terms of both legislation and operational
practice. The authors have made every effort to check information


                                  2
with the countries concerned and believe it to be accurate as of
May 1998. Where anticipated changes are known, these are
referred to in the text.
The Legislative Review was prepared for the Consortium by Birute
Mockiene (Lithuanian Centre for Quality Assessment in Higher
Education), Malcolm Frazer (QSC) and Don Westerheijden
(CHEPS). The Needs Analysis was prepared by the Consortium's
Project Management Group comprising John Brennan and Ruth
Williams (QSC), Algirdas Cizas (Lithuanian Centre), Sonja Kump
(Centre for Evaluative and Strategic Studies), André Staropoli
(Comité        National    d'Evaluation),     Christian      Thune
(Evalueringscenteret) and Don Westerheijden (CHEPS). The
authors wish to acknowledge the contribution of Anna Scesa
(Project Administrator) of the QSC in preparing the text for
publication, of the countless individuals in the 12 countries who
kindly supplied and checked information and of the members of
the Programme's Steering Committee, the Programme Co-
ordination Unit and the European Training Foundation. However,
responsibility for any inaccuracies remaining in the text rests with
the authors.
John Brennan
Project Director
June 1998
Contents
Page
Part One: A Comparative Review of Recent Legislation
1. Introduction
2.Legislative Framework
2.1 Quality imperative in the higher education systems of Europe


                                 3
2.2 A note on terminology
2.3 Changes in the national higher education systems of the Phare
countries
2.4 New legislation in the Phare countries
2.5 Orientation towards western standards: internationalisation
3. Quality Evaluation Policy and Practice
3.1 Quality issues in higher education systems of the Phare
countries
3.2 Agencies responsible for quality assurance
3.3 Internationalisation of quality assurance
4. Conclusions and Anticipated Changes of Quality Assessment in
Higher Education
4.1 Main features of quality assurance in Phare countries
4.2 From state control to self-steerage?
References
Part Two: A Needs Analysis
1. Introduction
2. The Current Situation
3. Clarification of Purposes of Quality Assurance
4. The Role of the State and the Implications of the Extent of
Institutional Autonomy
5. Getting Started
6. Administrative and Resource Needs of quality Assurance
7. Training
8. Information About Quality Assurance Developments
9. The Balance Between Research and Teaching
10. Programme Evaluation
11. Institutional Evaluation and Management
12. Comprehensiveness of Quality Assurance


                                  4
13. Collaboration Between Quality Assurance Agencies
14. Conclusions
References
Annexes
Annex I Legal documents on higher education and quality
assessment in higher education in Phare countries adopted in
1990-1997
Annex II Some Tempus projects on quality assurance in higher
education in 1996/1997
Annex III Institutions in charge of quality evaluation in higher
education in the Phare countries
Annex IV List of addresses of national institutions for quality
evaluation in higher education (as of December 1997)
Annex V National legislation in twelve countries


Part One: A Comparative Review of Recent Legislation
Introduction
The components of the Legislative Review as described by the
project's terms of reference are:
(i) to make optimum use of existing sources of information, to take
stock of existing legislation and/or the normative basis of academic
quality assurance and accreditation in the Phare countries covered
by the programme, and to ensure that a satisfactory comparative
analysis is available, set within the context of the various
methodologies and systems used by EU countries;
(ii) to identify needs and actions to be undertaken in order to
increase the compatibility of higher education quality assurance
and accreditation systems, and to enhance the European
dimension in the field;


                                    5
(iii) to disseminate the results of the review to all quality assurance
and accreditation bodies, rectors conferences, ministries and other
interested institutions.
A task team was established by the QSC-led consortium to
undertake the Legislative Review and to make proposals on the
needs analysis. The work of the task team included establishing a
framework for the Legislative Review comprising the existing
country legislation on quality assurance in higher education, the
context and the implementation of legislation. Two main sources
of information supported the Legislative Review: first, the country
legislation and existing studies which have been undertaken by
organisations such as UNESCO/CEPES, OECD and Council of
Europe. The second source comprised information gathered by the
designated project experts given the task of liaising with individual
Phare countries.
It should be noted that in discussing the quality assurance systems
of the individual Phare countries, the authors have referred to the
internationally-recognised four stage model of quality assurance
which is (i) the existence of a national quality assurance agency,
(ii) self-evaluation (by institutions, programmes of study etc.), (iii)
peer review and site visits, and (iv) publication of reports.
This review aims to illuminate the situation on quality assurance
and accreditation systems within the countries of central and
eastern      Europe,       including       comparative   references   to
developments in western Europe. The review consists of three
parts: (i) the legislative framework, (ii) quality evaluation policy and
practice, (iii) conclusions and anticipated changes in quality
assurance.
2. Legislative Framework


                                       6
2.1 Quality imperative in the higher education systems of
Europe
Recent documents of the European Commission and other
international organisations have given much attention to issues of
quality in higher education. European pilot projects were launched
in the European Union's participating institutions according to
guidelines for "evaluating quality in higher education" where
general objectives of "enhancing awareness of the need for quality
assessment in higher education", as well as "enriching the existing
national level quality assessment procedures" were pursued.
Recommendations of the Council of Europe Committee of
Ministers (No R (97)1) on the Recognition and Quality Assessment
of Private Institutions of Higher Education (adopted on the 4th
February 1997) state that "the term quality assessment covers a
range of explicit evaluation procedures concerning the quality of
higher education institutions, including its programmes, or only a
given    programme".     This    document     explains    the    term
"accreditation" as having originated in the American system and is
used in some European countries identically to "recognition".
An important international document of two organisations, the
Council of Europe and UNESCO, "Convention on the Recognition
of Qualifications concerning Higher Education in the European
Region", signed in Lisbon on the 11th April, 1997 at the Diplomatic
Conference (33 countries were signatories to the document until
December 1997, including 10 Phare countries) contains explicit
provisions regarding data collection in the field of quality in higher
education. Section VIII contains provisions on "Information on the
assessment of higher education institutions and programmes".



                                  7
Finally, the recent proposal to the Council of a Recommendation
on European Co-operation in Quality Assurance in Higher
Education (presented by the European Commission) on the 2nd
May 1997 (COM(97)159 final) recommends to the European
Union's Member States the establishment of, within their
responsibilities for the organisation of their higher education
systems:
transparent quality assessment and quality assurance systems,
basing systems of quality assessment and quality assurance on
the… principles of autonomy and independence of the bodies
responsible for quality assessment and quality assurance:
· relating evaluation procedures to the profile of institutions while
respecting their autonomy;
· internal and external procedural elements;
· involvement of all players;
· publication of evaluation reports [Proposal, 1997].
The history of legal foundations and practices in the field of quality
assurance in higher education in European countries is rather
short. The European Union's countries vary in their national
contexts, legal basis and implementation of national laws and
experience    in   quality   assurance.   Denmark,      France,   The
Netherlands and the United Kingdom could be mentioned among
those having formalised evaluation systems although these are
themselves of fairly recent origin. European pilot projects were
based on the experience of these countries.
Countries currently establishing systematic quality assurance
(legislative framework and implementation) are as follows: the
university sector in the Flemish-speaking community of Belgium,
Finland, Norway, Portugal, Spain and Sweden. Others could be


                                  8
mentioned only as having some evaluation experience without
having established supporting structures and legislation (Becher
T., Kogan M., 1992; Brennan J. et al, 1993 and 1994; UNESCO
1995).
The fairly recent experiences of quality assurance in the countries
of western Europe have been paralleled by developments in the
countries of central and eastern Europe, most of which have now
established   national   quality   agencies   and   are   testing   or
implementing evaluation methodologies. The recency of these
developments across the whole of Europe should be borne in
mind. Most national systems are themselves evolving and in some
countries remain controversial.
2.2 A note on terminology
The subject of quality in higher education has produced an
extensive and sometimes bewildering lexicon of terms. At the risk
of over-simplification, we distinguish between three separate,
although related concepts.
Evaluation, as used in this report, is a process of judging the
quality, or value of, something e.g. an academic programme, a
faculty, a university. ('Quality assessment' or 'quality measurement'
are sometimes used instead.)
Accreditation is used here to mean the award of a status. It signals
approval and recognition. The accreditation decision may be
based on a process of evaluation, or it may not.
Quality assurance is used as a general term to include not only
evaluation and accreditation but the total actions and mechanisms
through which the quality of higher education is maintained and
developed. ('Quality management' is sometimes used instead.)



                                   9
As will be made clear in the rest of this report, the growing
attention given to quality issues in higher education all over the
world is part of broader changes in the functions of higher
education and, in particular, in its relationship to the state and
other parts of society. Within this broader context, the question of
higher education's autonomy is often raised. Although there is near
consensus amongst the academic community that autonomy is
something to be prized (and something which may be under threat
in some places), there is much less consensus about what is
meant by autonomy. In a recent paper, Frazer has referred to
seven possible meanings of autonomy regarding: legal status,
academic authority, mission, governance, finances, as an
employer, academic decentralisation (Frazer, M. 1997). Frazer's
concern is with institutional autonomy but a further source of
confusion in discussions about autonomy is a failure to distinguish
the institution from the individual. Thus, issues concerning the
academic freedom of an individual professor are sometimes
conflated with issues of institutional autonomy.
As Frazer also makes clear, autonomy - in all of its various
sources - is related to notions of accountability. Accountability (for
the way in which autonomy has been exercised) is the principal
reason why systems of external quality assurance have been
introduced into higher education. Accountability may be to those
who finance universities, those who study in them, and those who
use their services, e.g. by employing their graduates (Frazer,
1997, p350).
Although the terminology of autonomy and accountability is
frequently found in recent higher education legislation in the Phare
countries, its meaning is not always clear. Ideally, the concept of


                                  10
'autonomy' should be qualified by reference to some attribute of
the institution - e.g. its rights to employ its own staff, to devise its
own curricula etc.
Documents written in different legislative and cultural contexts
pose considerable difficulties for interpretation. Explicit comparison
is rendered difficult. Thus, although we believe that certain general
trends are clear from recent higher education legislation in Phare
countries, we would be more cautious in attempting to make
explicit comparisons between individual countries. This should be
borne in mind when reading the rest of this report.
2.3 Changes in the national higher education systems of the
Phare countries
Two major trends that are reflected in legislation and practice of
higher education in Phare countries are worth mentioning:
diversification of institutions and programmes, and globalisation of
higher education. Both have major implications for quality
assurance.
In each Phare country the growth of higher education institutions,
changes to their names (and missions), the establishment of
private ones, have become constant features. New institutions,
new programmes, new qualifications create increasing complexity
for persons who try to evaluate, for one or another reason, a
strange diploma or a study programme. These may not be new
problems but they are accentuated with diversification and
globalisation.
The problem of compatibility of awards within a national higher
education system is one thing. But the problem of securing
compatibility of awards within an individual country poses



                                  11
considerable challenges for achieving compatibility of awards
between countries.
The usual instruments in the hands of a state to solve the
problems are regulations, rules, standards and registers. An
additional element included in the system of regulations is the
quality evaluation that may be used to ensure state control and a
wider credibility within the diversified system.
In order to prove that minimum requirements are fulfilled, a
licensing and/or accreditation procedure is carried out. Some
countries have experienced the mushrooming of private institutions
(e.g. Bulgaria, Estonia, Hungary, Latvia, Poland, Romania) which
may be less amenable to traditional mechanisms of state control.
But even where a private sector of institutions has not been
established, systems of accrediting existing state institutions have
commenced in many places, e.g. in the Czech Republic and the
Slovak Republic.
Diversification of higher education increases when a binary system
is introduced, with the creation of a sector of non-university
institutions. This has occurred in Estonia, Latvia, Poland and
Romania where accreditation procedures form a basis for the
normative evaluation of study programmes and institutions along
binary lines.
Globalisation of the higher education system firstly implies
increasing trans-border transactions among peoples, assets and
services, and secondly deepening the economic interdependence
between and among entities (public and private).
The growing number of institutions, programmes and students
requires more teachers. Some professors in Phare countries work
at several institutions. They are forced to do this for economic


                                  12
reasons (comparatively low salaries). The potential damage is
obvious in terms of the quality of teaching. Therefore qualitative
and quantitative requirements for permanent staff of higher
education institutions are set out in legal documents of each Phare
country.
Higher education institutions, both state and private, started
competing for students paying for tuition. Legal provisions
regarding fees have been made in several countries (e.g. Estonia,
Hungary, Lithuania) which force governments and legal entities
(institutions) to be more accountable to those paying fees.
Political considerations concerning application for membership of
the European Union (EU) have made it necessary for Phare
countries to transpose legal provisions of the EU into national
legislation. Comparisons of systems and standards became a vital
obligation of governments of all countries having agreements on
associate status with the European Union. The importance of
comparing higher education systems and standards within central
and eastern Europe increased with the possibilities of participating
in EU programmes SOCRATES (ERASMUS in particular),
LEONARDO, Youth for Europe, Framework IV, ACE (Action for
Co-operation in Economics) and others. Quality assessment and
recognition in higher education became an important element and
prerequisite for internationalisation: development of international
academic mobility, creation of mutually beneficial research links,
transfer of academic awards (credits) in the Phare countries.
Several Phare countries used and are using Tempus programmes
to develop good quality assessment and recognition practices in
their respective higher education systems in co-operation with their
western partners. Several ongoing Tempus projects are worth


                                 13
mentioning in this context, in particular those co-ordinated by
higher education institutions in Bulgaria, Czech Republic, Hungary,
Poland, Romania, Slovak Republic (see annex II).
2.4 New legislation in the Phare countries
New laws on higher education have been enacted in most Phare
countries since 1990, thus creating a new legal basis for
transformation and modernisation of higher education systems.
(See annex I).
New legal acts introduced the following basic changes to the
systems:
· ensured autonomy of higher education, its independence from
the state, institutional autonomy and independence of departments
within the structure of the institution - in most countries;
· guaranteed academic freedom - in all countries;
· integrated research and education - in most countries;
· introduced new rules of administration based on competitive
financing or with elements of competitive financing - in several
countries.
Autonomy of higher education institutions has even been declared
in the constitutions of several states, namely Bulgaria, Estonia,
Lithuania, Romania, Slovenia, and has become a slogan in
discussions on higher education in all of the countries of central
and eastern Europe. However, some countries have found it
necessary to determine the limits of autonomy in their legal
documents (e.g. Czech Republic, Estonia, Hungary, Poland,
Slovak Republic). Some countries with a long history of state
control still have de facto state control, even if their legislation may
proclaim autonomy of higher education. In some cases autonomy



                                   14
has been described in the legislation in a quite vague way (e.g.
Lithuania, Slovenia).
Administrative initiatives have been taken to establish closer links
between research and university sectors, to make science and
education indivisible. In Lithuania joint doctoral studies under the
leadership of university level higher education institutions have
been encouraged, involving intellectual resources from state
research institutes. Hungary's new PhD programmes provide a
similar example.
Attempts have been made to introduce more flexible and fairer
systems of financing higher education and research based on
qualitative and quantitative indicators. In most countries of central
and eastern Europe means of financing imply a form of steering (in
some cases the only form of steering) for the higher education and
research system.
Legislating in central and eastern Europe in the nineties in general
was a challenging issue in view of the dramatic macro-changes in
the societies. This also had implications for higher education
legislation. New laws shaped new rules, procedures and measures
at the level of decision making, points of reference for steering and
policy instruments.
In general, policy-makers concentrated on the foreseen and
desirable effects of the policy system keeping in mind several
shortages or hampering circumstances in the higher education
systems of the countries concerned. The main difficulties arose
from
i) lack of a coherent and comprehensive legislative framework;
ii) lack of well-structured reporting systems giving reliable data and
information on institutions and systems of higher education;


                                 15
iii) lack of experience in the higher education institutions in
formulating decisions and plans (in the centralised system
decisions tended to be made outside institutions or, if made by
institutions, to be influenced or determined by non-academic
factors) (Ryan L. 1993);
iv) lack of funding for higher education;
v) lack of trust of governmental bodies in higher education
institutions, both in those receiving funding from the state and
those providing private higher education.
2.5        Orientation          towards           western       standards:
internationalisation
Countries        have   faced       numerous       impediments        to     the
implementation of the huge tasks laid out in the new laws. Working
together    in     international    organisations,     representatives        of
academic communities from central and eastern Europe as well as
western countries have initiated several programmes to support
eastern     partners     in   the    field   of   legislation   and        policy
implementation in higher education.
Several of these have touched directly upon quality issues in
higher education. International events have been organised in
Europe and world-wide. It would be worth mentioning at least two
of them. UNESCO/CEPES, the Ministry of Education in Romania
and the University of Romania organised the International High-
level Consultation on Policy Issues of Quality Assessment and
Institutional Assessment and Institutional Accreditation in Higher
Education in Oradea (Romania) in 1993, which was particularly
directed at the countries of central and eastern Europe. The above
mentioned International High-level Consultation adopted the
Oradea Statement which has formulated recommendations on


                                      16
issues of quality assurance, assessment and improvement as well
as those of institutional accreditation as priorities in central and
eastern European systems of higher education.
Furthermore,     the   Council      of    Europe   Legislative     Reform
Programme for Higher Education (LRP) organised a Multilateral
Workshop "Accreditation and Evaluation in Higher Education" in
Bratislava in 1993. Later activities of the LRP (missions, seminars,
study visits) made a significant impact on almost all Phare
countries introducing quality assurance systems in higher
education. One important outcome of these activities was the
establishment of the regional co-operation in the Baltic States
creating an international body "Baltic Higher Education Co-
operation Committee" mainly in charge of quality assurance
developments in the respective higher education systems.
Some international organisations covering a world-wide audience
have also influenced reform of higher education in Europe. The
OECD Institutional Management in Higher Education (IMHE)
programme has contributed to raising awareness on the topic
through a series of seminars held in several countries of central
and eastern Europe. The International Network for Quality
Assurance      Agencies    in    Higher    Education    has      facilitated
dissemination of information in regular newsletters and organises
conferences world-wide.
Developments within countries in central and eastern Europe
must, therefore, be viewed in a much wider international context
which is itself rapidly evolving.
3. Quality Evaluation Policy and Practice
3.1 Quality issues in the higher education systems of the
Phare countries


                                    17
A uniform system of quality evaluation cannot be found in Phare
countries. Similarities and differences can be detected in existing
legislation,    existing      quality evaluation    bodies      and existing
practices. In most countries during the transitional period from
centralised to more autonomous open and free higher education
systems,       quality   assurance      was   considered     through    "first
generation" legislation, rules and requirements. Clarification of the
distinction between quality assurance through laws, which regulate
and restrict behaviour within higher education, and quality
assurance through evaluation, which makes public judgements
about such behaviour, is a current task for higher education
systems in the Phare countries.
Methodology and practice of quality assurance in higher education,
both in western, central and eastern European countries, in
general conforms to a four stage model, comprising:
· a national institution (agency, commission) in charge of quality
assessment;
·   self-evaluation      of    higher   education   (faculty,    department,
programme, institution);
· external peer review;
· public reporting (Van Vught F. A., 1994).
However, there are considerable differences in emphasis and
implementation, as can be seen in the national descriptions
contained in Annex V. As indicated in Annex V, the meaning of
quality assurance, quality assessment and accreditation is found in
legal papers dealing with the following issues: description of
requirements of staffing; setting up levels of qualifications;
introduction of accreditation and evaluation procedures. However,



                                        18
there is a lack of consistency in the usage of these terms between
countries.
Evaluation with accreditation
The vast majority of Phare countries have introduced legal
provisions   on     accreditation.      Although   the    notion   of   the
accreditation purpose and procedures is almost the same in the
following countries: Bulgaria, Czech Republic, Estonia, Hungary,
Latvia, Poland, Romania, Slovak Republic, the focus and the
outcomes differ.
Accreditation in the countries mentioned above serves as a two-
tiered procedure of recognition of higher education institutions
(and/or programmes, or faculties), when the state introduces a
formal licensing procedure based on a process of quality
evaluation of the institution (or a programme). This system was
applied to the establishment of new (often private) higher
education institutions in Bulgaria, Estonia, Latvia and Romania.
The methodology of evaluation contains mostly elements of
normative and summative assessment (standards based) of the
mission of an institution (or a programme), input (staffing,
resources, student admission procedures), throughput (curricula
and organisation of study process), data on research activities and
management structures (Kogan M. 1993). Some approaches to
evaluation emphasise output (student achievement, employment,
etc.) (Frazer M. 1997).
Hungary is to be considered a pioneer in the development of
accreditation of all doctoral programmes. In addition, tasks of
accreditation      of   institutions,     evaluation     of   qualification
requirements for a field of study as well as assessment of



                                     19
applications for a full professorship, are all delegated to the same
body - the Hungarian Accreditation Committee.
The Czech Republic and the Slovak Republic, both having legal
provisions on accreditation since 1990, carry out evaluations of
institutions, faculties and programmes (mainly post-graduate in the
Czech Republic). Poland's recent (1997) Act on Schools of Higher
Vocational Education contains provisions on accreditation as well,
although the process has not started yet. The need to introduce
accreditation in this country was strengthened by the growing
number of private higher education institutions.
Outcomes of accreditation in the countries mentioned above are
leading to recognition of the institution by the state. In the case of
the Czech Republic, Romania, and the Slovak Republic, the right
to organise state or diploma examinations and the right to award
nationally   recognised    diplomas     are   also   determined    by
accreditation. When a negative or partly negative accreditation
decision occurs, advice or recommendations for improvement are
given and a new evaluative and accreditation process is planned.
In certain cases, when the accreditation is negative, administrative
measures are taken to close institutions or programmes, but the
state takes responsibility for the students' rights to pursue their
studies in a recognised institution of the country concerned.
Evaluation without accreditation
Lithuania and Slovenia are distinctive as countries tending to
develop improvement oriented systems of quality assurance in
higher education and research without direct links at present to an
accreditation system. The Swedish model of quality assurance as
support to higher education institutions is considered the most
appropriate in Lithuanian academic society which has carried out a


                                   20
broad and long-lasting debate regarding the introduction of
accreditation procedures. (Quality assurance as support for
processes of innovation, 1997). Slovenia has recently invested in
special research projects to verify the methodology to be applied in
its small and autonomous higher education system. Academics in
these countries are trying to make a clear distinction between
normative evaluations, based on rules and regulations, and
formative evaluations, reflecting the notion of Trow's "academic
culture of excellence" (Trow M. 1994).
Poland also seems to be quite reluctant to accept strict rules of
accreditation. A wide academic discussion on the issue is
anticipated in 1998. The Rectors' Conference of the Polish
universities has proposed (January 1998) the establishment of a
University Accreditation Committee. In these and several other
countries (Latvia, Estonia) legal documents of higher education
institutions,   e.g.   statutes,   regulations,   provide   norms   and
mechanisms of quality assurance. Special bodies are established
to guarantee quality assurance on the institutional, faculty and
department level. Attention is paid to regular self-evaluation of
teaching and research (and management).
3.2 Agencies responsible for quality assurance
Agencies organising quality assurance have been established in
almost all Phare countries. In most cases in central and eastern
Europe, similarly to western countries, these agencies are
regarded as buffer institutions. Their role in legal provisions is to
advise, to recommend, to serve as an expert, to give consultations,
information and support. Their constitutions and powers vary
considerably from country to country.



                                    21
In the countries of the European Union, quality assurance
agencies differ in the roles they play in higher education. An
important difference between agencies in the European Union lies
in the source of their authority. In some cases (e.g. Denmark) this
resides in the state, in other cases (e.g. The Netherlands) this
resides   in   the   academic   community     of   higher   education
institutions. Most of the agencies in the Phare countries are state
sponsored although there is awareness of the need to keep them
operationally distinct from education ministries and to obtain the
full support of institutions and the academic community.
Various attempts are made to strike an acceptable balance
between the interests and requirements of the state and the values
and wishes of the academic communities in these countries. The
Phare countries are not alone in facing challenges to achieve a
mutually acceptable balance.
Agencies differ in the focus of quality assurance. For example, the
Comité National d'Evaluation (CNE) in France is responsible for
the assessments of the quality of both teaching and management
of all universities; the Association of Universities (VSNU) in The
Netherlands is responsible for co-ordination of all evaluation at
programme level in Dutch universities; the Centre for Quality
Assurance and Evaluation of Higher Education in Denmark has
responsibility for the evaluation of programmes of study at degree
level and the evaluation, in conjunction with the research councils,
of research activities (Brennan J. et al. 1994).
Several Phare countries have established agencies, councils,
committees or centres for more or less operational purposes: to
organise quality evaluation in higher education. In some cases
committees or councils give advice or judgement to their


                                  22
respective governments on the quality of higher education. These
institutions are listed in annex III.
Some of these institutions (predominantly councils or committees)
are bodies of elected, nominated and appointed members and
they play a political role in developing new legislation and
implementing quality assurance practices in the countries
concerned.
Most of the agencies were established by the parliaments,
governments, or ministries of education. States, or states together
with higher education institutions, finance agencies in charge of
quality assurance.
Representatives of the labour market play only an insignificant role
in quality assurance. In only a few countries (e.g. Czech Republic,
Estonia, Slovak Republic, Slovenia) do representatives from the
society at large, or from the professional and employment sector,
take part in the work of the councils, committees or commissions
or in the policy making or programmes of evaluation of higher
education and research.
3.3 Internationalisation of quality assurance in higher
education
All countries, whether in the west, centre or east of Europe,
consider internationalisation as an objective and a challenge in the
development of higher education systems. European Commission
programmes encouraging academic co-operation and mobility can
be successful and reach beyond European Unions' borders if
higher education communities speak on equal terms and share a
common higher education 'language'. This makes it necessary to
deal with the quality of international co-operation in higher



                                    23
education and with internationalisation of the procedures of quality
assurance.
Policy and practice in this field of internationalisation are taken into
account in most Phare countries, but are still weak in Phare
countries (as in other parts of Europe). A limited number of foreign
peers are invited to take part in the evaluation of programmes; in
some countries (e.g. Hungary) foreign representatives are
members of advisory committees; international discussions have
been organised on quality evaluation goals and criteria (Baltic
States). National Phare programmes in Bulgaria, Latvia, Lithuania,
Romania, as well as Tempus programmes are making serious
investments in developing east-west networks in the field of
internationalisation of quality assurance in higher education.
4.   Conclusions      and    Anticipated     Changes      to   Quality
Assurance in Higher Education
4.1 Main features of quality assurance in the Phare countries
Most of the Phare countries are in the process of establishing
quality assurance systems based on a four stage model
comprising: creation of a national co-ordinating agency, self
evaluation, external peer review, published reports. As such, the
approach being adopted follows established practice elsewhere in
the world.
But there are some important differences. The emphasis placed on
accreditation in most Phare countries reflects the concerns of
governments to impose control over what have been rapidly
changing and, in many ways, unstable higher education systems.
This is understandable in the context of transition.
Quality assurance is being developed at both programme and
institutional levels. The former reflects concerns about international


                                  24
recognition of qualifications. The latter reflects concerns about
system    management     and,   in    some   cases,   a   need    for
rationalisation.
There is some use made of academic peers from other countries
in the quality assurance processes of most Phare countries
although this is not yet being implemented in a systematic way.
4.2 From state control to self-steerage?
Legal documents on higher education and regulations on quality
assurance in most Phare countries contain provisions which are
predominantly oriented towards control. Only a few countries of
central and eastern Europe follow the line taken in western higher
education systems to place emphasis on the improvement of
academic life and of the quality of teaching and learning in
particular. The rules in Phare countries are written to establish
procedures and mechanisms within the higher education system,
including intermediate bodies and ministries in charge of the
system. The concept of the self-governing, self-critical higher
education institution is given somewhat less emphasis.
The primary purposes of legal provisions are to ensure recognition
of awards and diplomas; to exercise state control (accountability to
the state); to stimulate change, innovation and improvement.
Legislative acts on higher education and provisions for quality
assurance take into account academic, historical and cultural
traditions of the countries in central and eastern Europe. National
as well as international policy dimensions have been taken into
consideration, keeping in mind the general concern to assure a
minimum degree of compatibility of higher education systems and
the notion of European standards and values of quality assurance
in higher education. It is worth mentioning that European


                                 25
standards in central and eastern Europe in the period of political
recognition as well as academic recognition were concerned with
achieving international validity. But it should be borne in mind that
standards and criteria are implicit characteristics of the academic
community and tend to be defined by higher education institutions
in accordance with both national traditions and international
developments. The relationship between what can be achieved
through legislation and what may already exist in the international
culture of academic life would repay further consideration.
Different approaches to legislation in the field of quality assurance
have been highlighted. Several countries found it necessary to
promulgate     special   legislation    on   quality   evaluation   and
accreditation, others included quality provisions in the framework
of laws on higher education and research or laws on education.
Where measures of accreditation have been adopted, the idea of
minimum standards and summative evaluations have been
elaborated and implemented in legal provisions and actual
practices. Consequences of accreditation are a legal right to
perform certain activities and a direct link with funding.
From a time perspective, the legal basis for higher education, and
quality assurance in particular, should be considered as an attempt
to fit into a transitional policy where autonomy of higher education
institutions   is   accompanied    by    adequate      mechanisms    of
accountability. Considering the process from a short term
perspective (approximately seven years) the transitional period
has revealed a need to deal with quality issues in higher
education, to perceive and legitimise these issues in the academic
communities, to institutionalise practices. Different countries are at
different stages along this line of development.


                                  26
Although the emphasis is placed on improvement in only a few
countries of central and eastern Europe, for all countries
evaluation has the potential to stimulate improvements in higher
education. Evaluation involves a learning process for all of those
who participate in it with change and improvement the desired
outcomes. However, a framework of state control through
accreditation is not necessarily the best context for the
achievement of improvement goals.
The self-evaluation process, if done well, has proved an
appropriate way to look at the strengths and weaknesses of a
higher education institution, or a programme, or an individual
member of the academic community. The self-evaluation process
is strengthened and balanced by the involvement of external peers
who bring broader knowledge and objectivity.
The Phare countries are implementing methods of quality
assurance in higher education which are quite similar to those
being adopted elsewhere. But the contexts are different. The
contexts in the Phare countries are of societies in transition and
these bring uncertainties, mis-trust and doubts at all levels of
society and at all levels in higher education. Thus, in the short run,
perhaps inevitably quality assurance is about imposing order and
certainty. It is about control. But in the long run, serious
consideration of the self-regulating potential of a professional
organisation (higher education institution) will be required. In such
a self-regulating organisation, many things are needed, as was
stressed in the first volume of the Council of Europe Legislative
Reform Programme:




                                  27
Self-steering of universities necessitates leadership by inspiration
besides the use of methods for pricing and costing, planning and
evaluation (Veld, R. et al, p.90).
The place of quality assurance within systems of effective internal
management of higher education institutions rather than as part of
systems of external control is the longer term challenge for higher
education.
References
Becher, T. and Kogan, M. (1992) Process and Structure in Higher
Education, 2nd ed. London, N.Y, p.157-171.
Brennan, J. Goedegebuure, L.C.J. Shah, T. Westerheijden, D.F.
Weusthof, P.J.M. (1993) Comparing Quality in Europe, Higher
Education in Europe, Vol. XVIII, No 2. p.129-146.
Brennan, J. El-Khawas, E. Shah, T. (1994) Peer Review and the
Assessment of Higher Education Quality: An International
Perspective, Quality Support Centre, London, 71 p.
COM(97)159 final (1997) Proposal for a Council Recommendation
on European Co-operation in Quality Assurance in Higher
Education, (Presented by the Commission), Brussels, 11 p.
Frazer, M. (1997) Report on the Modalities of External Evaluation
of Higher Education in Europe: 1995-97, Higher Education in
Europe, Vol XXII, No 3, p.349-401.
in't Veld, R. Füssel, H.P. Neave, G. Eds (1996) Relations Between
State and Higher Education, Council of Europe.
Kogan, M. (1993) The Evaluation of Higher Education: An
Introductory Note, in Dimensions of Evaluation: Report of the
IMHE Study Group on Evaluation in Higher Education, in U.
Dahllöf, J. Harris, M. Shattock, A. Staropoli, R. in't Veld Eds,
London, p.11-25.


                                     28
Quality Assurance as Support Model in Comparative Perspective:
The Swedish Model in Comparative Perspective, Stockholm, 1997.
108 p. (Högskoleverket Studies, 1997:I S).
Ryan, L. (1993) Prelogomena to Accreditation in Central and
Eastern Europe, Higher Education in Europe, Vol. XVIII, No 3,
p.81-90.
Scott, P. (1996) Internationalisation and Quality Assurance. Goals,
Strategies and Instruments: EAIE Occasional Paper, De Winter U.
Ed., 55 p.
Tempus: Compendium. Academic Year 1996/97, Phare European
Commission        European   Training    Foundation,     Brussels;
Luxembourg, 1997. 381 p.
Trow, M. (1994) Academic Reviews and the Culture of Excellence,
Stockholm, 40 p.
UNESCO (1995) Quality Assurance and Institutional Accreditation
in European Higher Education. Bucharest, Higher Education in
Europe, Vol. XX, Nos. 1-2. 201 p.
Van Vught, F. A. (1994) Intrinsic and Extrinsic Aspects of Quality
Assessment in Higher Education, in Changing Contexts of Quality
Assessment, D. Westerheijden, J. Brennan, P. Massen Eds.
Utrecht, p. 31-50.
Vlasceanu, L. (1993) Quality Assurance: Issues and Policy
Implications, Higher Education in Europe, Vol. XVIII, No 3. p. 27-
41.
Part Two: A Needs Analysis
1. Introduction
The project's terms of reference state that the aims of the Needs
Analysis are to specify the measures necessary and the scope for
common actions within a multi-country context, to enhance the


                                29
compatibility and sustainability of existing quality assurance and
accreditation systems, and to ensure that new systems benefit fully
from European experience and best practice.
This Needs Analysis is based partly on the Legislation Review
undertaken for this project (see Part One) and partly upon needs
identified   within   the   twelve    countries    themselves   and
communicated to the Project Management Group through the
country liaison expert members of the project team.
This Needs Analysis identifies both needs to be met within the
context of the Multi-Country Project and needs which are longer-
term or outside the terms of reference of the project. It
concentrates on needs which are common to all or several of the
eleven countries. Common needs will be discussed under the
following headings:
· Clarification of purposes of quality assurance
· The role of the state and the implications of the extent of
institutional autonomy
· Getting started
· Administrative and resource needs of quality assurance
· Training
· Information about quality assurance developments
· The balance between research and teaching
· Programme evaluation
· Institutional evaluation and management
· Comprehensiveness of quality assurance
· Collaboration between quality assurance agencies
2. The Current Situation
As the Legislative Review makes clear, almost all countries have
enacted legislation which includes provisions concerned with the


                                 30
quality of higher education. A minority of countries have
operational national quality assurance systems with two or more
years of experience. Others have established systems more
recently or are engaged in continuing debates about the nature
and purpose of such systems. For the latter, there is an evident
problem of 'getting started'.
These first steps in the establishment of quality assurance reflect a
diversity of external influences as well as differences in national
contexts and traditions. Developments in some countries show a
strong influence of US models of accreditation, e.g. Hungary and
Romania. In other countries, there is a clear influence by individual
European countries, e.g. the Higher Education Funding Council for
England has been training external experts within all three Baltic
States.
In all countries, developments are compatible with wider
international experience insofar as they reflect a general model
which comprises:
· a national co-ordinating body
· self-evaluation by institutions (programmes, faculties etc)
· external evaluation by expert peers
· published reports.
But within this general model, there are considerable variations in
approach, in part reflecting the kinds of influence and advice
referred to above. There is a clear need in all countries for a
greater awareness of alternatives, and for choices to be made in
the light of national purposes and conditions. The importation of
systems and practices from individual western countries is unlikely
to be successful unless tailored to local circumstances.
3. Clarification of Purposes of Quality Assurance


                                  31
    The literature on quality assurance in higher education refers to
    two key purposes of accountability and improvement (e.g.
    Vroeijenstijn 1995). Recent international studies (e.g. Brennan and
    Shah 1997) have referred to a wider range of purposes, e.g.
         to ensure accountability for the use of public funds
         to improve the quality of higher education provision
         to inform funding decisions
         to inform students and employers
         to stimulate competitiveness within and between institutions
         to undertake a quality check on new (sometimes private)
    institutions
         to assign institutional status
         to support the transfer of authority between the state and
    institutions
         to assist mobility of students
         to make international comparisons.
    Governments and national quality bodies frequently list several
    such purposes in their public statements. But to these must
    sometimes be added hidden or less public purposes, e.g. closing
    down certain kinds of institutions or programmes, increasing
    productivity within the academic profession, regulating the content
    of higher education to meet changing societal needs.
    Among the twelve Phare countries, we have found some confusion
    as to the purposes of quality assurance. This is not to imply that
    multiple purposes cannot be achieved but it is to imply that
    purposes should be recognised explicitly by all involved, that
    priorities should be agreed and systems and practices should be
    designed and resourced accordingly. This represents an important
    need in all countries.


                                       32
A specific need in relation to purposes is a clarification of the
difference between accreditation and evaluation. They are not the
same. Several countries (e.g. Bulgaria, the Czech Republic,
Hungary, Romania) have emphasised accreditation processes in
their developments to date. While there may have been good
reasons for this emphasis, there is now a need to consider
whether the approaches adopted so far are likely to achieve the
broader purposes of quality assurance and, in particular, whether
they will support or hinder the growth of a wider culture of
evaluation within higher education.
The clarification of purposes, the distinction between accreditation
and evaluation, reflect differences in approach between 'top down'
and 'bottom up' quality systems and practices. As such, they are
connected with wider issues of the role of the state and the
meaning of institutional autonomy in higher education.
4. The Role of the State and the Implications of the Extent of
Institutional Autonomy
'Autonomy' is a much-abused term in discussions about relations
between higher education and the state. It is taken to be a
desirable goal. However, in practice it is usually severely limited. It
may refer to any of the following (Frazer 1997):
· Legal status
· Academic authority
· Mission
· Governance
· Financial
· As an employer
· Academic



                                  33
In the post-1989 reforms, many of the traditional forms of control
and regulation exercised by the state were removed in the eleven
countries. New institutions and programmes - and the rapid
expansion of existing ones - have been a feature of developments
in most countries. In some countries, the creation of private
institutions has further confused the relation between the state and
institutions.
Governments now see a need to regulate expansion and, in
certain cases, to act to remedy the worst excesses of previous
expansion. Accreditation is frequently seen as an important
mechanism for doing this. But there are also other mechanisms,
e.g. legislation, planning and funding. There appears to be a need
for national governments, with their respective higher education
communities, to debate the role of quality assurance as one
among a range of mechanisms through which the state can
regulate the development of higher education according to national
purposes and priorities.
For higher education institutions, there is a need - urgent in some
places - for a greater recognition of the implications of greater
institutional autonomy for management and decision-making.
Quality assurance can play an important role in achieving greater
levels of institutional autonomy and accountability. But, to be
effective, it requires clarity and acceptance of the respective
responsibilities of rectors, deans, professors and other categories
of staff within institutions, and of the decision-making powers of
academic councils, committees and boards at all institutional
levels. Without this, autonomy is unlikely to be achieved in any real
sense and it can be expected that governments will gradually re-
assume the external control and management of institutions.


                                 34
Quality assurance, in these latter circumstances, is likely to
become important as an instrument of state control rather than a
force for institutional improvement.
5. Getting Started
There is a minority of countries where the most pressing need is to
move beyond debate to action. In some countries, e.g. Albania and
Poland, legislation on higher education quality does not yet exist.
In other countries, e.g. Slovenia, legislation has been in existence
for some time but has not yet led to any significant action.
The absence of legislation should not be a barrier to action.
Indeed, its absence allows greater opportunity for flexibility and the
testing out of a range of quality assurance methods. In those
countries which have not yet done so, there is a need to gain
experience of self- and external-evaluations at both institutional
and programme levels. The subsequent formulation of national
and institutional policies will only be assisted by such experiences.
Indeed, continuing debate and policy formulation in the absence of
such experiences is likely to prove unsound and to lead to
instability in whatever quality systems are established.
6. Administrative and Resource Needs of Quality Assurance
Failure to implement quality assurance policies and procedures
may reflect a failure on behalf of government and institutions to
recognise the resource implications of quality assurance. This is
not just about financial resources but about time and expertise
necessary to support the processes involved.
Recognition of the time demands on academic staff is necessary.
In the short-run, these demands will be accentuated by the need
for discussion and training.



                                 35
Additionally, there is a need for a recognition of the administrative
resources required. Professionally competent administrators play
an important role in any quality assurance system, helping to
safeguard the integrity of the processes involved and helping to
reduce the burdens that would otherwise fall on academic staff.
Adequate numbers of appropriately trained administrators are
necessary both within national quality bodies and within higher
education institutions. The professionalism and credibility of such
administrators - many of whom will themselves have academic
backgrounds - will be a key determinant of the success of any
quality assurance system, whatever its nature.
A further important resource need will be for information. Effective
self-evaluation at the institutional level will be assisted substantially
by the availability of information on student progression rates and
satisfaction, on employment destinations, on course curricula,
bibliographies and assessment schemes, on staff research and
publications and so on. Good institutional information systems and
the administrative supports to ensure their maintenance and
effective use can reduce substantially the load on academics in
preparing self-evaluations and other forms of institutional quality
assurance. Also at the national level, sound information can assist
in providing benchmarks and reference points for the evaluation of
individual institutions and programmes, as well as providing a
useful record of the 'state' of higher education in the country as a
whole.
In emphasising the importance of resource for quality assurance,
there is clearly need for realism about what individual countries
can afford. But the following points should also be borne in mind.
First, the resources necessary to support quality assurance will be


                                   36
    a tiny proportion of both national and institutional budgets. Second,
    the outcomes of quality assurance will represent an important
    resource to the country and the higher education community,
    providing improved quality and standards and better decision-
    making throughout the higher education system. The input of
    resources for quality assurance should therefore be seen as an
    investment.
    7. Training
    Inevitably, the establishment of new systems and procedures will
    be secured more firmly if training is supplied. This is recognised in
    the project's terms of reference. Training of institutional staff,
    academic and administrative, can assist both in the preparation of
    self-evaluations and in the preparation for external evaluations.
    Training is also essential for those academic staff who will perform
    the role of external experts in the evaluation process. There is a
    continuing need for training of staff in quality assurance agencies.
    However, the need for and the nature of training will depend, to
    some extent, on the kinds of quality assurance activities that are
    envisaged in a particular country. Whereas quite specific training
    can be given by the project for those activities which are part of the
    project, training to support quality assurance generally should be
    of a more generic awareness-raising nature. The most important
    needs are for
         practical experience
         confidence building
    rather than specific skills - which might vary between systems - at
    this stage.
    8. Information About Quality Assurance Developments



                                     37
    Linked to training is a need for greater awareness and
    understanding of developments in quality assurance in higher
    education and, in particular, of the wide variety of current national
    and international activities in the quality field.
    In more specific terms, we believe that there is a need for better
    information on
         current quality assurance projects sponsored by a range of
    international organisations
         a glossary of key terms, identifying variations in usage.
    Such information should be disseminated widely throughout the
    higher education communities of the twelve countries.
    9. The Balance Between Research and Teaching
    This is a wide-ranging issue facing higher education systems
    throughout the world. To some extent, it is an issue still to be
    addressed in central and eastern Europe countries where higher
    education cultures and values are still predominantly within a
    'Humboldtian' model. Yet the issue is one which underpins most
    current debates about the future of higher education: about
    diversity, about equality, about higher education's links with the
    economy, about how universities are organised and resourced.
    The balance between research and teaching affects both the
    design of a quality assurance system (e.g. should each be
    evaluated, separately or together?) and the likely impact of quality
    assurance. There is now quite a lot of evidence from western
    countries that the introduction of quality assurance of teaching can
    increase the attention given to teaching within higher education
    institutions. But much depends on the rewards (including prestige)
    which attach to the different activities.



                                        38
We believe that a debate about the balance and the relationship
between research and teaching cannot be divorced from issues of
quality assurance. For quality assurance to achieve its potential
benefits, it needs to be accompanied by a wider debate about the
functions and purposes of higher education as these affect
research and teaching.
10. Programme Evaluation
We note that the needs perceived in most of the twelve countries
are for programme evaluation. This may reflect the importance
attached to the international recognition of diplomas and the
greater ease with which peers may be identified at the subject or
programme level. It may also reflect the relative weakness of the
institutional level of decision-making in many countries.
The development of programme evaluation will present difficulties
in the smaller countries of the region. The creation of international
networks of subject experts may be needed to make programme
evaluation effective in these countries, although problems of
language and contextual knowledge will be considerable. This will
be an area on which the pilot evaluations undertaken as part of the
present project will make a significant contribution to determining
what will be feasible in the longer term.
11. Institutional Evaluation and Management
There is a strong link between institutional evaluation, autonomy at
the   institutional   level,   and    the   capacity   for   institutional
management and decision-making.
Institutional evaluation which is not linked to effective institutional
decision-making is unlikely to provide assurance to those outside
the institution (e.g. funders and users) and nor is it likely to
contribute to institutional change and improvement. In addition,


                                     39
therefore, to measures directed towards the development of quality
assurance, there is a need for complementary measures to assist
in the enhancement of institutional management and decision-
making.
Although, as indicated in 10 above, most countries have initiated
evaluation and accreditation at the programme level, several have
begun to embark on quality assurance activities at the institutional
level as well. This raises questions of the linkage between
programme level and institutional level processes, the purposes of
each and the additional costs involved.
12. Comprehensiveness of Quality Assurance
There is a view prevalent in many of the twelve Phare countries
that quality assurance is only necessary for recently established
higher education institutions, for private institutions, for non-
university institutions, for small institutions. We believe that such a
view represents a misunderstanding of both the nature of quality
assurance and the pressures and demands facing all higher
education institutions at the present time. Reputations based on
past achievements in quite different circumstances provide little
assurance of achieving quality in today's conditions.
There is a need, therefore, to ensure that there is recognition that
quality assurance in higher education must be comprehensive: it
should apply to all institutions and programmes. (This does not
mean, however, that procedures of quality assurance should be
identical between institutions and programmes.)
13. Collaboration Between Quality Assurance Agencies
There is already considerable experience of quality assurance
through the work of national quality agencies in many of the Phare
countries. These agencies have grown up in parallel with their


                                  40
counterparts in western countries. There is a need to share and
compare experiences of the more established of the national
agencies, both within the Phare countries and with agencies
around the world, as for example through the International Network
of Quality Assurance Agencies (INQAA).
There may also be some scope for sharing resources, e.g.
publications or networks of peer experts. Given the importance
attached to internationalisation in the higher education policies of
many Phare countries, quality assurance can play an important
part in opening up higher education systems to external influences,
to supporting students and staff mobility and the international
recognition of qualifications. Within the smaller countries of the
region, there are also good operational reasons for collaboration
between national agencies, e.g. providing neutral expertise in the
evaluation of subjects which exist in only one or two of a country's
institutions.
14. Conclusions
Many of the needs identified above are interconnected. Few will be
achieved quickly but the present project can make an important
contribution to beginning to meet them.
Needs that can be addressed by the present project, at least to
some extent, include
· clarification of the purposes of quality assurance
· getting started
· training
· information about quality assurance developments
· the balance between research and teaching
· programme evaluation
· comprehensiveness of quality assurance


                                  41
· collaboration between quality assurance agencies.
Needs that will need to be addressed outside the present project
include
· the role of the state and the meaning of institutional autonomy
· administrative and resource needs of quality assurance.
References
Brennan, J. and Shah, T. (1997) Quality Assessment Decision
Making and Institutional Change in Tertiary Education and
Management, Vol. 3, No. 2.
Frazer, (1995) M. Report on the Modalities of External Evaluation
of Higher Education in Europe: 1995 - 1997, Higher Education in
Europe, Vol. XXII, No. 3.
Vroeijenstijn, A. I. (1995) Improvement and Accountability:
Navigating Between Scylla and Charybdis. Guide for External
Quality Assessment in Higher Education, Higher Education Policy
Series 30, Jessica Kingsley Publishers, London.
Annex I
LEGAL DOCUMENTS ON HIGHER EDUCATION AND QUALITY
ASSESSMENT         IN    HIGHER       EDUCATION        IN   PHARE
COUNTRIES
(adopted in 1990-1997 and available in English)
Country Title of the legal document Year of adoption
1 Albania Law No. 7810, of Higher Education in the Republic of
Albania 1994
2 Bulgaria 1. Higher Education Act 2. Statute for the Functioning of
the National Evaluation and Accreditation Agency 1995 1996
3 Czech Republic Act No 172 Concerning Institutions of Higher
Education 1990



                                 42
4 Estonia 1. Law on Universities 2. Law on the Amendment of the
Law of Universities. 3. Law on the Organisation of Research and
Development Activity 4. Law on the University of Tartu 5. Law on
Education 6. Standard of Higher Education 1995 1996 1997 1995
1992 1996
5 Hungary Act LXXX of 1993 on Higher Education as amended by
the Act LXI of 1996 1993, amendment of 1996
6 Latvia 1. Law on Higher Educational Establishments 2.
Methodological Recommendations for Assessment of Higher
Education   Institutions   and   their     Study    Programmes     3.
Recommendations      for   Accreditation    of     Higher   Education
Institutions and Study Programmes 4. Accreditation Regulations
for Higher Educational Establishments 5. An Amendment to the
Accreditation Regulations 1995 1995 1997 1996 1996
7 Lithuania 1. Law on Research and Higher Education 2.
Qualitative Regulations of Higher Education 3. Regulations for the
Establishment and Assessment of Institutions of Higher Education
4. Regulations of the Lithuanian Centre for Quality Assessment in
Higher Education 5. Rules of Quality Assessment for Institutions of
Research and Higher Education 1991 1993, Amended 1996 1993,
Amended 1996 1995 1996
8 Poland 1. Acts on Schools of Higher Vocational Education 2. Act
on Schools of Higher Education 3. Act on the Academic Titles and
Academic Degrees 1997 1990 1990
9 Romania 1. Law on the Accreditation of Higher Education
Institutions and the Recognition of Diplomas 2. Law on Education
1993 1995
10 Slovak Republic Higher Education Act 1996
11 Slovenia Higher Education Act 1993


                                 43
    Note: Information on the Former Yugoslav Republic of Macedonia
    is not included in annexes I-IV. At the time that information was
    being gathered in May 1998 (the Former Yugoslav Republic of
    Macedonia joined the project only at a later stage), new legislation
    was being drafted but had not yet been presented to Parliament.
    The current situation in the Former Yugoslav Republic of
    Macedonia is described in Annex V.
    Annex II
    SOME TEMPUS PROJECTS ON QUALITY ASSURANCE IN
    HIGHER EDUCATION IN 1996/1997
         Accreditation   System     Development     for   New    Degree
    Courses; Complementary Measures project (CME) co-ordinated by
    Technical University of Sofia, Bulgaria;
         Accreditation of Agricultural Higher Education; CME project
    co-ordinated by Higher Institute of Agriculture in Plovdiv, Bulgaria;
         The Development and Assessment of a New Primary Care
    Oriented Medical Curriculum; Joint European Project (JEP) co-
    ordinated by Charles University in Prague, Czech Republic;
         Strategic and Internal Management of Czech Universities:
    Design and Implementation of a Quality Assurance System at
    Institutions of Higher Education in the Czech Republic; JEP co-
    ordinated by University of West Bohemia in Plzen; Czech
    Republic;
         Quality in Post -Graduation Courses in the Czech Republic,
    CME co-ordinated by Instituto de Soldadura e Qualidade, Oeiras,
    Portugal;
         Implementation of Quality Assurance in three Hungarian
    Polytechnics, JEP co-ordinated by Donat Banki Polytechnic in
    Budapest, Hungary;


                                      44
         Quality Control at the Polytechnic of Dunaujvaros, JEP co-
    ordinated by Miskolc University, Faculty College of Metalurgy,
    Dunaujvaros, Hungary;
         Quality     Assessment      and   Facility   Management   in
    Architecture, JEP co-ordinated by Silesian Technical University in
    Gliwice, Poland;
         Development of an Internal Quality Assurance System at
    Bialystok Technical University, CME co-ordinated by above
    mentioned university in Poland;
         Quality Assurance in Engineering Degree Programs; JEP co-
    ordinated by University Dunarea de Jos', Galati, Romania;
         Quality Management in Higher Education, JEP co-ordinated
    by University of Transport and Communication in Zilina, Slovak
    Republic;
         EQATU - Education Quality Assessment at Technical
    University, JEP co-ordinated by Slovak Technical University in
    Bratislava.
    Annex III
    INSTITUTIONS IN CHARGE OF QUALITY EVALUATION IN
    HIGHER EDUCATION IN PHARE COUNTRIES
    Country Name of the Agency, Committee, Council Year of
    Establishment
    1 Albania -
    2 Bulgaria      1. National Evaluation and Accreditation Agency
    (NE&AA) 2. Accreditation Council 1996 1996
    3 Czech Republic Accreditation Commission 1990
    4 Estonia 1. Higher Education Quality Assurance Council 2. Higher
    Education Quality Assessment Centre 1995 1997
    5 Hungary Accreditation Committee (HAC) 1992


                                      45
6 Latvia 1. Council of Higher Education 2. Higher Education
Quality Evaluation Centre 1995 1995
7 Lithuania 1. Science Council of Lithuania 2. Centre for Quality
Assessment in Higher Education 1991 1995
8 Poland 1. Main Higher Education Council 2. Accreditation
Commission of Schools of Higher Vocational Education 1990 1997
9 Romania National Council for Academic Evaluation and
Accreditation (CNEAA) 1994
10 Slovak Republic Accreditation Commission 1990
11 Slovenia 1. Council for Higher Education 2. Higher Education
Quality Assessment Commission 1994 1997
Annex IV
List of addresses of national institutions for quality evaluation
in higher education in Phare countries
(as of December 1997)
Bulgaria
National Evaluation and Accreditation Agency (NEAA)
at the Council of Ministers
Tzarigradsko shosse Blvd., bl.5,
1113 Sofia
Telephone: (359 2) 971 20 39
Fax: (359 2) 971 20 68
Czech Republic
Secretariat of the Accreditation Commission
Secretary: Professor Vaclav Vinđ
Ministry of Education, Youth and Sports of the Czech Republic
Karmelitska 7, 11812 Prague
Telephone: (42 0) 51 93 457
Fax: (42 0) 51 93 790


                                   46
Estonia
Higher Education Quality Assessment Centre
Foundation ARCHIMEDES
Kohtu 6, EE 0100 Tallinn
Telefax: (372) 696 2425
Fax: (372) 696 2427
GSM (372 51) 10 450
E-mail: mart@euedu.ee
Hungary
Hungarian Accreditation Committee
Ajtosi Durer sor 19-21
H-1146 Budapest
Telephone: (36 1) 251 2951; 344 0314
Fax: (36 1) 344 03 13
E-mail: rozsnyai@fki.huninet.hu
Homepage: www.hac.huninet.hu
Latvia
Higher Education Quality Evaluation Centre
Latvian title: Agstakas Izglitibas Kvalitates Novertesanas Centrs
Valnu iela 2, LV-1098 Riga
Telephone: (371 7) 213 870
Fax: (371 7) 212 558
Lithuania
Lithuanian Centre for Quality Assessment in Higher Education
Lithuanian title: Studiju kokybes vertinimo centras
Suvalku g. 1, 2600 Vilnius
Telephone: (370 2) 23 25 55
Fax: (370 2) 23 25 53
E-mail: acizas@skvc.ktl.mii.lt


                                  47
Homepage: http://neris.mii.lt/research/kokybes/prisista.htm
Poland
Secretariat of the Main Higher Education Council
Chairman: Professor Jerzy Osiowski
Ministry of National Education
Szucha 25, 00-918 Warszaw
Telephone: (48) 26 218 478; 22 25 46 39
Fax: (48) 26 218 478; 22 252 300
Romania
National Committee for Accreditation and Academic Evaluation
(CNEAA)
Professor Paul Sterian General Secretary
Bucharest Polytechnical University
SPL Independentel 313, Sector 6
Bucharest
Telephone: (40) 1 410 45 85
Telefax: (40) 1 312 7135
Slovak Republic
Secretariat of the Accreditation Commission
Ministry of Education
Hlboka 2, SK-813 30 Bratislava
Telephone: (42 1) 7 49 65 72 (Head of Dept.) 395 257 (Professor
M. Melnik)
Telefax: (42 1) 7 49 89 89 (Head of Dept.); 393 198 (Professor M.
Melnik)
Slovenia
National Higher Education Quality Assessment Commission
Kongresni trg 12
1001 Ljubljana


                                 48
Telephone: (386) 61 125 41 17
Fax: (386) 61 125 40 53
E-mail: Alojz.Kralj@fe.uni-lj.si
Annex V
National legislation in THE TWELVE PHARE Countries
1. Albania
1.1 National and international context of higher education
Albania covers an area of 28,748 sq. km. and has a population of
3,303,000.
The higher education sector consists of 10 higher education
institutions, seven of which have university status. Others provide
higher professional education programmes.
All institutions are subordinated to the Ministry of Education and
Science. The Higher Education Council is an elected body which
decides on all policy issues in the field. The higher education
legislation of 1994 (Law on Higher Education) made provisions for
the creation of a new National Council of Higher Education
Council, which is composed of 42 members and chaired by the
Minister of Education and Science. The Council has wide-ranging
responsibility for the structure of the higher education system as
well as for teaching, research and evaluation. It is also responsible
for developing higher education funding plans which are submitted
to the Finance and Economic Affairs Minister for approval.
1.2. Legislation on higher education and the provision for
quality assurance
The legislation on higher education was promulgated on 6 April
1994. Some amendments are being considered in order to make
the law more responsive to the needs of higher education
development in the fields of private and professional education.


                                   49
The 1994 legislation provides the framework for autonomy (Article
7)   and   Article   16   states   that   "The    composition,    tasks,
competencies and functioning of the senates, councils and their
units in universities and other higher education institutions are
defined in the statutes of higher education institutions". Article 41
says that, "the draft statutes of the higher education institutions are
approved by the Minister of Education". (No information is
available on these statutes and their provisions regarding quality
issues).
In addition, legislation on science and technological development
was promulgated on 22 December 1994. This defines the basic
principles of organisation, management, operation and evaluation
of scientific and technological development activities.
In 1996 the Council of Europe suggested the creation of an
independent Quality Council to deal with the issues of quality
assurance and recognition, but no steps have been made in this
direction yet.
1.3 Implementation of quality assurance in higher education
Article 26 of the 1994 higher education legislation states that: "the
Commission for Scientific Qualification approves, on the basis of
the proposal of the National Council of Higher Education, the
classification of the teaching staff in the field of Science". Article 43
provides that: "The academic accreditation and quality assurance
of higher education institutions are made on the basis of special
decisions of the Council of Ministers".
The legislation on science and technology development contains
several provisions for the evaluation of scientific and technological
activities in Chapter III Article 27/d, Article 28/b. Chapter VI,
entitled Evaluation of the Scientific and Technological Activities,


                                   50
    makes provisions for the evaluation of national programmes as
    well as the evaluation of scientific institutions.
    Sources:
    Law on Higher Education in the Republic of Albania, 6 April 1994,
    No 7810.
    Law on Science and Technological Development, 22 December
    1994, No 7893.
    Council of Europe (1996) Report of the Legislative Reform
    Programme Advisory Mission on the integration of higher
    education and research legislation.
    Bock, K.H. (1997) Student Handbook: A Directory of Courses and
    Institutions in Higher education for 29 Countries Which are Non-
    members of the European Union, Council of Europe, Bonn.
    2. Bulgaria
    2.1 National and international context of higher education
    Bulgaria covers an area of 110,000 sq. km and has a population of
    8.5 million.
    In Bulgaria there are 29 state universities, 42 state non-university
    institutions and five private universities.
    The following state organisations have responsibilities for higher
    education (Articles 8 - 11 of the Higher Education Act of 1995):
         The General Assembly has powers to establish, transform
    and close higher schools, and allocate funds for public higher
    education. Proposals relating to these powers come from The
    Council of Ministers.
         The Council of Ministers determines the national policy for
    higher education and also approves the state requirements for
    obtaining degrees, approves numbers of students in each state



                                       51
    higher school, and determines fees and scholarships, etc. for
    students in public higher education.
         The Ministry of Education and Science (MES) is responsible
    for implementing the national policy on higher education, and
    makes proposals to the Council of Ministers.
         The National Evaluation and Accreditation Agency (NEAA) at
    the Council of Ministers is a state subsidised legal entity and
    "specialised governmental authority" for the quality assurance and
    accreditation of the activities of the higher schools.
    Article 9 of the NEAA gives the responsibility for representing
    Bulgarian higher education internationally and entering into
    international contracts to the Council of Ministers. However, Article
    21 gives the right to higher schools to sign international contracts
    and to become members of international organisations. MES is
    responsible for the legitimisation of the diplomas of students who
    have studied abroad (Article 10).
    2.2 Legislation on higher education and provision of quality
    assurance
    The Higher Education Act (HEA), passed by the General Assembly
    on 12 December 1995, introduced major reforms for higher
    education in Bulgaria. It introduced new regulatory mechanisms for
    state control of higher education through the state organisations
    listed above. Article 8 provides for the state to take "care of the
    quality of the training process and research through organising the
    activities of the NEAA and defining the conditions for recognition of
    the higher education diplomas issued in our country and abroad".
    Chapter 10, entitled Accreditation of Higher Schools (Articles 75 -
    88), provides the legislative basis for quality assurance and



                                      52
    defines accreditation as the recognition of compliance between the
    activities of a higher school and the state requirements.
    The managing bodies of NEAA are the Accreditation Council and
    its Chairperson. The Accreditation Council consists of:
           fourteen representatives of the higher schools elected by the
    Rectors
           seven representatives of the Bulgarian Academy of Sciences
    and the Agricultural Academy elected by their Boards of Directors
           two representatives of MES.
    The structure and activities of the NEAA are regulated by the
    NEAA Statute.
    The Accreditation Council through secret ballot proposes to the
    Prime Minister the appointment of one of its members as
    Chairperson, who then becomes President of NEAA. The principal
    responsibilities of the Accreditation Council are to:
           establish expert committees which are responsible for
    reporting the results of each evaluation and accreditation and
    approve their report
           evaluate existing institutions and programmes or projects to
    open or reform institutions and programmes, make accreditation
    decisions and inform MES
           adopt the Regulations of NEAA and submit them for approval
    to the Council of Ministers.
    There are detailed provisions in the HEA and the NEAA Statute
    about     applications   for   accreditation,   timescales   for   their
    completion, consequences of positive and negative outcomes, and
    provision for appeals.
    NEAA is required to:



                                       53
         develop criteria and standards for accreditation, while
    observing the state requirements
         develop and approve the procedures and documentation for
    the accreditation process
         assess the condition and activities of higher schools, their
    faculties and specialities on the basis of which accreditation shall
    either be given or refused
         establish an information system about accredited higher
    schools
         publish (i) a bulletin giving criteria and requirements for
    accreditation, (ii) examples of best practice and (iii) an annual
    report.
    The costs of accreditation are to be covered by the applicant at
    rates approved by the Ministry of Finance (Article 83 of the HEA).
    The NEAA Statute gives detailed procedures about attendance,
    voting and members remuneration of the Accreditation Council.
    NEAA has published, after consultation with higher schools, its
    fundamental documents, approved by the Accreditation Council on
    20 November 1997. It is this paper which describes the methods to
    be used for institutional and programme accreditation.
    2.3 Implementation of quality assurance in higher education
    The Decree establishing the Accreditation Council and NEAA was
    issued in August 1996, the first staff of NEAA were appointed in
    October, and the first formal meeting of the Council was held in
    November. The Agency became fully operational by December
    1997. The Decree makes provision for a staff of 27, but following
    the State Budget of June 1996, this was reduced to 17.
    The HEA and Decree make it clear that the purpose of quality
    assurance is one of accreditation (approval or non-approval) of


                                     54
programmes       and   institutions    (Decree,    Article   13).   Quality
improvement is also mentioned in the Decree, which states that
the Executive Director of NEAA has among his responsibilities to
"organise the fulfilment of consultancy services to the higher
schools..." (Decree, Article 7).
With regard to the methodology of quality assurance, the published
NEAA procedures are based on the four stage model. It should be
noted, however, that this model is not a requirement either of the
HEA or the Decree.
Although NEAA is required to carry out both institutional and
programme evaluation, it has indicated in its published work plan
that it will concentrate on programme evaluation initially. NEAA
has published criteria and indicators for programme accreditation,
and on the work and responsibilities of external experts (peers)
and expert committees.
Since its creation, NEAA has organised many workshops and
seminars for staff from the higher schools and provided training for
external experts. Eleven programme accreditations have been
granted. The NEAA management body organised over 15
meetings with the academic community of the higher education
institutions in order to explain the accreditation principles and its
documents.
With regard to reporting procedures, Article 23 of the Decree
requires that an accreditation document be published in the NEAA
Bulletin, and that it shall confirm that ".... the quality of the activities
of the accredited institution or study course are in conformity with
the unified state requirements and the criteria and state
requirements     developed      by     NEAA".     The   validity    of   the
accreditation document is five years.


                                      55
    Sources:
    Decree of the Council of Ministers, 1 August 1996, No 189.
    Higher Education Act, 1995.
    State Budget of June 1996.
    NEAA      (1997)        Evaluation,      Accreditation,     Opening      and
    Transformation     of     Higher      Schools,   their    Main   Units   and
    Programmes, Fundamental documents of NEAA.
    3. Czech Republic
    3.1 National and international context of higher education
    The Czech Republic covers an area of 78,886 sq. km. It has a
    population of 10,3 million.
    As at 1996 there were 23 public higher education institutions, three
    military higher education institutions and one police academy.
    There is no private higher education in the Czech Republic.
    Czech higher education institutions are established (merged,
    divided and abolished) with an Act by the Czech Parliament. The
    Academic Senates of higher education institutions are responsible
    for establishing faculties while the Accreditation Commission
    expresses its expert point of view on this act.
    The following bodies play an important role in higher education:
         Ministry of Education, Youth and Sport
         Accreditation Commission
         Council of Higher Education Institutions
         Czech Rectors Conference
    The Ministry of Education, Youth and Sport is entitled by law to co-
    ordinate the activities of higher education institutions and create
    conditions for their development. It is responsible for the allocation
    of the state budget among the institutions and for the supervision
    of its effective use. Further responsibilities of the Ministry are


                                           56
    mentioned in connection with the functions of the Accreditation
    Commission later in the text.
    The Accreditation Commission and the Council of Higher
    Education    Institutions   were   established    by   the   Act.   The
    responsibilities of the Accreditation Commission lie mainly in
    ensuring the quality of higher education whilst the Council of
    Higher Education Institutions works in partnership with the Ministry
    in taking all important decisions and measures concerning higher
    education. For instance, it discusses with the Ministry the
    proposals of the rules according to which funds are to be
    distributed from the State Budget.
    The role, rights and responsibilities of the Czech Rectors
    Conference are not explicitly prescribed in the law. Nevertheless,
    due to its prestige, it has the possibility to influence the functioning
    of the whole higher education system as an important partner of
    the state authorities.
    3.2 Legislation on higher education and provision on quality
    assurance
    The Higher Education Act 172/1990 which was adopted in the
    former Czechoslovakia is still valid in the Czech Republic.
    As concerns quality in higher education this Act states:
         in Article 15: "with the approval of the Accreditation
    Commission the Ministry shall decide on the removal of the right of
    a higher education institution or faculty to perform state
    examinations, the procedure for nomination of professors and the
    competence of associate professors, or on the return of those
    rights to an institution of higher education and faculties..."
         in Article 17: (1)"The Accreditation Commission shall be
    established by the Government, as its advisory body after noting


                                       57
the advice of the Council of Higher Education Institutions. The
members of the Accreditation Commission shall be nominated
from outstanding members of higher education, professional and
scientific institutions. (2) The Accreditation Commission shall
publish its views on the proposals for establishing, merging,
dividing or abolishing higher education institutions and faculties..."
The proposal for the new higher education act was submitted to
the Czech Parliament in December 1995. At that time the
Parliament refused to approve it and asked the Ministry to have
additional discussions and improve on several controversial
issues. After about two years, the new draft law is currently again
under discussion at parliamentary level and it is hoped that it will
be approved this year.
If approved and implemented the new act will bring about several
important changes. It makes provision for a better balance
between rights and responsibilities both at the state level (between
state authorities and institutional management) and at the
institutional level (relationship among rectors, academic senates
and scientific councils). It also introduces higher integrity of the
higher education institution. The most important change is the
proposal to transfer the state property to the institutions which will,
as a consequence, transform state higher education institutions
into public ones. To ensure their proper functioning, mainly with
respect to the property ownership, the law makes provision for the
establishment of a new body - the Council of Public Higher
Education Institution appointed by the Minister of Education.
Another important issue is the proposal to introduce tuition fees.
However, whether there is political will to accept this remains to be
seen. The new draft also makes room for the diversification of


                                  58
higher education. The introductory paragraphs mention the
possibility of establishing new types of higher education institution
providing mainly bachelor study programmes. Any educational or
scientific institution has the possibility to ask, in co-operation with a
higher education institution, for the accreditation of its study
programme. Finally, there are articles prescribing the possibility of
establishing private higher education institutions under the same
requirements for accreditation that apply for public institutions.
The draft law gives more rights and responsibilities to the
Accreditation Commission. This is very important with respect to
the diversification process leading to the so-called "mass" higher
education.
3.3 Implementation of quality assurance in higher education
Two major aspects are worth mentioning in the development of a
quality   assurance     system     in   higher   education:     national
accreditation and international evaluation.
In addition to the duties prescribed by the Act 172/1990 (see
Section 3.1) the Accreditation Commission evaluates continuously
the quality of higher education by means of evaluating faculties.
This activity is based on evaluation of groups of faculties dealing
with the same or similar study fields and the procedures followed
do not differ significantly from those adopted in many other
countries. The faculty which undergoes the evaluation is asked to
write a self-evaluation report which is submitted to the working
group of the Accreditation Commission. The next stage is the visit
of external experts at the faculty under scrutiny followed by the
preparation of the experts' report. Once discussed with faculty
representatives, the report is usually made partly public. The report
usually contains recommendations on how to overcome the


                                   59
weaknesses highlighted as well as noting the positive aspects of
faculty functioning.
During the last five years or so the Czech universities have been
involved in several international evaluations. For example the
University of Agriculture in Prague has been evaluated by the
University of Wagenningen, The Netherlands; the University of
Veterinary Medicine and Pharmacy in Brno by the Commission of
European Veterinary Universities; the Czech Technical University
within the CRE project. In co-operation with international
organisations such as the OECD and the International Association
for the Evaluation of Educational Achievement (IEA), an extensive
assessment of the whole educational system was carried out in the
Czech Republic. Some Tempus projects involve higher education
institutions in partnerships with United Kingdom, Swedish and
Danish institutions.
In general quality assessment has been accepted by the academic
community in the Czech Republic as a component of higher
education management, and universities try to incorporate it into
university self-government. Self-evaluation activities are carried
out in practically all institutions, though they have not always been
systematic, and the application of evaluative procedures differs
widely.
Sources:
Centre for Higher Education Studies (1992) Act No 172
Concerning Institutions of Higher Education, Prague.
Freibergova, E. Sebková, H. Skuhrova (1996) Overview of the
current situation within higher education in the Czech Republic,
Prague, 4 p.



                                 60
Ondracek, E. Sebková, H Hanzl, S. (1996) Czech Republic,
Relations Between State and Higher Education, R. int Veld , H-
P.Cssel, G. Neave Eds., Council of Europe, Vol. I.
Bock, K.H. (1997) Student Handbook: A Directory of Courses and
Institutions in Higher education for 29 Countries Which are Non-
members of the European Union, Council of Europe, Bonn.
Ministry of Education, Youth and Sports of the Czech Republic
(1994)    Quality   and    Accountability:   The     Programme    of
Development of the Education System in the Czech Republic,
Prague, p. 20
4. Estonia
4.1 National and international context of higher education
Estonia covers an area of 45,200 sq. km and has a population of
1.6 million.
In October 1994 the Declaration of Intentions of Co-operation for
Quality Assurance in Higher Education in the Baltic States was
signed by the three Ministers of Education for Estonia, Latvia and
Lithuania. The Baltic Higher Education Co-ordination Committee
was established in January 1995.
There are two types of higher education institutions in Estonia: i)
universities, offering academic higher education and diploma
programmes and ii) applied higher education institutions, offering
diploma programmes. Estonia has 14 public and 18 private higher
education institutions.
General higher education policy is defined by the Ministry of
Education. The Ministry is assisted by a number of management
and consultative bodies which fulfil the following tasks or roles at
administrative or teaching level:



                                    61
         the Higher Education Advisory Chamber - a consultative
    body of representatives of universities under the Ministry of
    Education regarding problems related to university education
         the Research and Development Council - a consultative
    body chaired by the Prime Minister
         the Estonian Science Foundation - a body of experts,
    representatives of universities and of the Ministry of Education
    which deals with the financing of science projects
         the   Higher   Education     Quality   Assurance    Council     -
    responsible for the accreditation of higher education institutions.
    4.2 Legislation on higher education and provision of quality
    assurance
    The legislation of March 1992 (Law on Education) which provides
    a general framework for the activities of higher education
    institutions and the legislative basis for quality assurance was set
    out in the Law on Universities of 1995, amended in 1996. In
    addition, legislation on private schooling was passed in 1993, and
    contains provisions on accreditation.
    The legislation concerning quality assurance in state and private
    higher education institutions defines accreditation as an "Activity
    during which an evaluation is made, and a decision is taken, on
    the basis of the accordance of the university and its curricula to the
    requirements determined by the law and standards" (Article 2 of
    the Law on Universities).
    Article 10 legislates on issues of accreditation and describes the
    mission of the Higher Education Quality Assurance Council, which
    shall be formed by the Government of the Republic and shall
    operate in the administrative jurisdiction of the Ministry of
    Education: "The Higher Education Quality Assurance Council shall


                                      62
accredit universities and their curricula at least every seven years.
The Accreditation decisions shall be made public".
Article 11 has provisions on the accreditation of universities and
Article 12 on the accreditation of curricula. Article 55 (Accreditation
and Status) states that "The Higher Education Quality Assurance
Council shall organise the accreditation of the curricula of all the
universities, within four years of the time the present Law comes
into force". Accreditation is the basis for the state recognition of
diplomas as well as for the state commissioning fields of study and
setting priority areas.
According to the Law on Universities, the principles of organisation
and governance of a university are set down in bye-laws - the
university constitution. The highest collective body of a higher
education institution is the council and the constitution of the
council is determined by the institution itself.
In 1995 the Higher Education Quality Assurance Council was
established as a quality assurance agency. Recently, the office of
the Higher Education Quality Assessment Centre within the
foundation ARCHIMEDES has been set up.
In 1996 the total state funding of accreditation did not exceed 0.3
per cent of the total state budget for higher education. The funds
for   accreditation   are   at   the    disposal   of   the   Foundation
ARCHIMEDES, but all arrangements for allocation are made by
the Chairman of the Higher Education Quality Assurance Council
in co-operation with the Chairman of the Higher Education Quality
Assessment Centre.
Organisational, management and financial issues are described in
the Statutes of the Higher Education Evaluation Council (approved
by the Government on 11 April 1995, Decree No 179) and


                                   63
Standards for Accreditation of a Higher Education Institution or its
Academic     Unit   (Institutional    Accreditation).   Statutes   make
provision for quality assurance (e.g. Tallin Technical University has
to conduct evaluation of study programmes every four years).
4.3 Implementation of quality assurance in higher education
With regard to the methodology of quality assurance, the four
stage model is used and the focus is on higher education
institutions and curricula (programmes), but not on research.
In 1996 Higher Education Evaluation Council carried out the
process of accreditation of study programmes in economics and
business (in six higher education institutions) and in law (in three
institutions). Foreign experts were used which provided the
Council and the academic community with some experience of
internationalisation in the field of quality assurance.
Sources:
Law on Organisation of Research and Developmental Activity, 26
March 1997.
Law on the Amendment of the Law on Universities, 26 June 1996.
Law on the University of Tartu, 16 February 1995.
Law on Universities, 12 January 1995.
Law on Private Schools, 2 June 1993.
Republic of Estonia Law on Education, 23 March 1992.
Statutes of the Higher Education Evaluation Council, 11 April
1995, Decree No 179.
Standard of Higher Education (Estonia), Validated on 26
November 1996.
Standards for Accreditation of a Higher Education Institution or its
Academic Unit.



                                     64
    Estonian ENIC/NARIC (1997) Private Higher Education Institutions
    in Estonia.
    Bock, K.H. (1997) Student Handbook: A Directory of Courses and
    Institutions in Higher Education for 29 Countries Which are Non-
    members of the European Union, Council of Europe, Bonn.
    Olm, T. (1996) Situation of Higher Education Quality Assurance in
    Estonia, Council of Europe Legislative Review Programme
    Seminar, Budapest.
    Sillamaa, H. (ed.) (1995) A Manual on Quality Assurance in Higher
    Education in Estonia,
    Tallin, Ministry of Education.
    Sillamaa, H. (1997) First Experiences on International Expert
    Evaluation in Estonia, Council of Europe Legislatve Review
    Programme Seminar, Lisbon.
    Vaht, G. (1997) Higher Education System in Estonia, Estonian
    ENIC/NARIC.
    5. Hungary
    5.1 National and international context of higher education
    Hungary covers an area of over 93,000 sq. km and has a
    population of some 10 million people.
    In Hungary there 25 state universities, five non state universities,
    31 state colleges, 23 church colleges, five private colleges and
    colleges operated by foundations.
    National higher education bodies are described in sections 76 - 81
    of the Law on Higher Education (1994, amended in 1996). In
    summary these are:
         The Ministry of Culture and Education.
         The Higher Education and Research Council which makes
    proposals and advises the Minister. Amongst other things it is


                                     65
    required to work out a system of performance indicators for higher
    education which are to be published by Government decree.
         The Hungarian Accreditation Committee which is a legal
    entity and an independent body appointed by Government and
    financed by Parliament (Section 80 {6}). Its functions are described
    in section 5.3 below.
    5.2 Legislation on higher education and the provision of
    quality assurance
    Higher education legislation was passed in 1993 (Act LXXX), and
    was amended by the Act LXI of 1996 (hereinafter "Act" refers to
    the 1996 Act). There is also a Parliamentary decree 107/1995
    which refers to the importance of quality in higher education.
    The Act states that there may be state funded universities and
    colleges and non-state universities and colleges. Section 3 of the
    Act describes the general requirements for the human and
    physical resources necessary for the establishment of a university.
    State higher education institutions will need to meet these
    requirements by 31 December 1998, by which time the
    Government will lay before Parliament their "founding charters".
    The spheres of state authority connected with higher education are
    described in Sections 69 - 75 of the Act. In summary:
         Parliament lays down the development plan for higher
    education, determines the annual operating budget, establishes
    (and can abolish) state higher education institutions, and
    recognises (and can withdraw recognition) non-state higher
    education institutions.
         The President appoints (and dismisses) university professors
    and mandates (and dismisses) rectors.



                                     66
         Government approves in state institutions the establishment
    of faculties, lays down the rules for the habilitation procedure, lays
    down the rules for the organisation and operation of the Hungarian
    Accreditation Committee, regulates the numbers and fees for state
    financed students, and lays down conditions for recognition of
    foreign qualifications.
         The Prime Minister appoints (and dismisses) college
    professors, mandates (and dismisses) college director-generals,
    and mandates the chairman and members of the Hungarian
    Accreditation Committee.
         The Minister of Culture and Education provides advice to the
    state bodies listed above, permits (and withdraws permission) for
    the launching of fields of study and accredited higher vocational
    education,    determines    the    qualification   requirements     of
    postgraduate education and regulates the conditions for awarding
    doctoral degrees, regulates the Higher Education and Research
    Council (see below) and has legal supervisory rights over the
    Hungarian Accreditation Committee (see below).
    Section 59 of the Act requires that the council of each higher
    education institution annually evaluates the effectiveness of the
    qualification requirements, the existence of the personnel and
    material conditions, and summarises the results of educational and
    research activities. The report of this evaluation, including
    proposals for actions, is sent to the Hungarian Accreditation
    Committee and is made public.
    The Hungarian Accreditation Committee (HAC) is described in the
    Act (Sections 80 and 81). It is established for "...ongoing
    supervision of the standard of education and scientific activity in
    higher education, for the perfecting of classification there, and for


                                      67
    the supporting of quality assurance...". Specifically, HAC is
    required to:
         assent to doctoral programmes and decide the branches of
    knowledge in which a university may conduct doctoral education
         adopt a standpoint on matters that involve the quality of
    higher education
         advise the Minister of Culture and Education on the
    establishment or recognition of institutions and faculties, doctoral
    and habilitation requirements, regulations about a credit system,
    and on the operation of foreign higher education institutions in
    Hungary
         at least every eight years assess the standard of education
    and scientific activity in each institution
         receive annual reports from institutions
         develop and publish detailed procedures for its evaluation
    tasks.
    Accreditation is defined in the Act (Section 124E) as "the
    attestation of educational and research activity conducted in higher
    education institutions and in their faculties, and the attestation of
    the results of these activities in respect of quality".
    From 30 June 2000, no university or college will be permitted to
    operate unless certain requirements are met and these will be
    evaluated by the HAC (Section 122).
    The HAC has 30 full members (15 from higher education, 10 from
    research institutes and five from professional organisations). There
    is also one non-voting student member. Appointments are for three
    years, and may be renewed once. HAC elects its President. It also
    has an international advisory body with 11 (maximum) experts
    from other countries.


                                        68
The management and administration of the affairs of HAC is
performed by a secretariat. The staff of the secretariat are public
employees.
5.3 Implementation of quality assurance in higher education
The HAC began its operations in 1994. The secretariat currently
has 13 members of staff. The Committee has adopted the four
stage model for quality assurance and the focus is on PhD
programme evaluation. The HAC provides guidance and training
on self-evaluation and prescribes a list of performance indicators.
With regard to peer review and site visits, the Chair for each team
is selected by the HAC, who then selects about four experts in the
field, sometimes including one from another country. The HAC
keeps a register of suitable peers. A member of the HAC staff acts
as secretary. The visit lasts three days on average and peers are
provided with a check list. With regard to reporting procedures the
report is published after the institution has checked a draft for
factual errors. The Minister of Education and Culture receives the
HAC resolution on whether the legal requirements have been met.
The Parliamentary decree 107/1995 refers to improvement of
teaching and research, and particularly the need to improve the
quality of PhD training. One of the stated functions of HAC is to
disseminate good practice, including publishing.
Sources:
Amendment to Higher Education Act 1996, Act N. LXI.
Higher Education Act 1993, Act N. LXXX.
Filep, L. Institutional responses to quality assessment. Developing
a quality assurance system in an institute. The case of Bessenyei
Teacher Training College. Unpublished paper.



                                69
Filep, L. and Balogh, A. Management problems in Hungarian
higher education institutions. Unpublished paper.
6. Latvia
6.1 National and international context of higher education
Latvia covers an area of 64,600 sq. km and has a population of 2.5
million.
In 1994 the Declaration of Intentions on Co-operation for Quality
Assurance in Higher Education in the Baltic States was signed by
the three ministers of Education of Latvia, Lithuania and Estonia.
The Baltic Higher Education Co-ordination Committee was
established in January 1995. The first year of activities of the
Committee ran under the leadership of Latvia. Two protocols have
been signed and implemented in the field of co-operation of quality
assurance in higher education.
The higher education system includes 19 state institutions, five of
which are called universities, eight academies which specialise in
a small range of subjects, and six institutions which are called
higher schools.
The legislation on higher education institutions (17 November
1995) does not distinguish between the different types of higher
education     institutions,   but   rather       between      academic   and
professional higher education. Academic education is described as
a general higher education based upon fundamental and applied
science, and professional higher education provides career-
oriented knowledge based upon applied science.
The Education Act of 1991 allows for the establishment of private
higher     education    institutions.        However,   the    degrees   and
certificates of the private institutions must be recognised by the
state through licensing followed by quality assessment and


                                        70
accreditation (as with all higher education institutions) of the
institutions and their study programmes. In 1997, there were 13
private institutions.
Policy making and development is the responsibility of the
Department of Higher Education and Research of the Ministry of
Education and Science. The Department works with the Higher
Education Council and the Latvian Council of Rectors.
6.2 Legislation on higher education and provision of quality
assurance
Article 1 of the 1995 legislation stipulates that "accreditation of
higher education institutions is an examination of the organisation
of work and the quality of resources in an education institution as a
result of which the latter is awarded the status of higher education
institution recognised by the state".
Article 5 (7.3) describes the tasks of higher education institutions
and provides that "the quality assurance of study programmes of
universities shall be carried out with the help of international
experts; different methods of written final assessment of the study
programmes shall be used to assess students' achievements".
Article 9 describes the general accreditation principles as follows:
"only those higher education institutions which have been
accredited and which offer state-accredited study programmes
have the right to issue to its graduates certificates of higher
education recognised by the state. The accreditation process is
carried out in accordance with the regulations on accreditation
approved by the Cabinet of Ministers and is organised by the
Ministry of Education and Science. Study programmes are
accredited at least once every six years."



                                  71
The requirements of the study programmes and higher education
institutions as well as the most important organisational aspects
related to accreditation are reflected in the Accreditation
Regulations for Higher Education Institutions (AREG) approved by
the Cabinet of Ministers on 28 November 1995. Recommendations
for application of these rules were elaborated by the Baltic Higher
Education Co-ordination Committee in 1996. According to the
1995 legislation and AREG, accreditation is a component of the
higher education quality assurance system which, together with
self-evaluation and peer evaluation, constitutes the system of
continuous quality assurance.
The Higher Education Quality Evaluation Centre Ltd. was
established by the Ministry of Education and Science on 28
December 1994. The Centre is a non-profit making organisation.
As representatives of the Council of Rectors, the University of
Latvia and Riga Technical University are among the shareholders
of the Centre. The statute capital of the Centre in 1996 amounted
to Ls 3000, 48 per cent of which belonged to the Ministry of
Education and Science, 26 per cent to the University of Latvia and
the remaining 26 per cent to Riga Technical University. All three
parties are entitled to vote at the meetings of the shareholders.
The Centre does not receive a direct budget but raises funds by
providing services which include the organisation of the external
evaluation of higher education institutions, consultancies for higher
education institutions, and applied research in the field of higher
education quality issues.
The statutes of the Centre provide its objectives which are:




                                 72
         to establish and co-ordinate the procedures for quality
    assurance of higher education institutions and study programmes,
    and prepare peer visits in compliance with legislation
         to organise quality assurance procedures on behalf of the
    Ministry of Education and Science
         to set up commissions and working groups responsible for
    solving problems related to quality assurance and accreditation
         to invite foreign experts as a part of peer visits
         to take stock and make public the experience acquired.
    6.3 Implementation of quality assurance in higher education
    In 1992-1993 an international evaluation of Latvian research was
    carried out by Danish scientists.
    The four stage model of quality assurance is applied.
    The evaluation of teaching in Latvia covers the following aspects:
    content; teaching methods used; resources; assessment methods
    used and students achievement; research, scholarships and
    professional involvement of staff; student services and social
    facilities; external interaction with programmes; employability of
    graduates; and quality control and assurance of programmes.
    The   evaluation    of   institutions    focuses   on   governance   of
    institutions; management; financial viability and control; resources;
    library and computing and staff training and development; student
    services; international activities; relations with the local community;
    and quality control and assurance systems.
    The peer review system operates by selecting peers who are the
    leading experts in their field drawn from higher education
    institutions. One peer is usually from Latvia and at least two others
    are invited from other countries. Training in evaluation is provided.
    Peers are usually selected by the Centre which keeps a database.


                                        73
Approximately three to four peers are used for each site visit over
two to four days. The evaluation includes observation of teaching
and students' work and both students' and employers' opinions are
taken into account.
With regard to reporting procedures, the team of peers prepares a
report in consultation with the institution evaluated which is then
published. The report will state whether or not accreditation has
been granted (there are no links to funding). Mechanisms for
appeal have been established.
Sources:
Cabinet of Ministers (1995) Accreditation Regulations for Higher
Educational Institutions, Reg. No 370, 28 November 1995.
Cabinet of Ministers (1996) Amendment to the Accreditation
Regulations adopted by the Cabinet of Ministers on 28 November
1995, Riga, 2 July 1996.
Education Act, 1991.
Law on Higher Educational Institutions, 1995.
Methodological Recommendations for Assessment of Higher
Education Institutions and their Study Programmes.
Ministry of Education and Science (1997) Recommendations for
Accreditation   of    Higher   Education   Institutions   and   Study
Programmes, draft.
7. Lithuania
7.1 National and international context of higher education
Lithuania covers an area of 65,300 000. sq. km and has a
population of 3.7 million.
In October 1994 the Declaration of Intentions of Co-operation for
Quality Assurance in Higher Education in the Baltic States was
signed by the three Ministers of Education for Estonia, Latvia and


                                  74
    Lithuania. The Baltic Higher Education Co-ordination Committee
    was established in January 1995.
    Lithuania's higher education system consists of 15 state higher
    education institutions: eight universities, six academies and one
    institute. There is no non-university sector at present and no
    private institutions.
    Principles of university autonomy, academic freedom and the
    integration of research and higher education are enforced. All
    institutions may organise academic and professional studies.
    The following bodies play an important role in higher education:
         The    Ministry    of   Education   and   Science    which   has
    responsibilities   for   higher    education    policy    making   and
    implementation.
         The Science Council which is an independent science and
    research institute. It provides advice to the Parliament and the
    Government on the organisation and financing of science and
    research and is responsible for the quality assurance of doctoral
    studies.
         The Conference of Rectors which represents the interests of
    higher education institutions and plays a very important role in the
    policy making process.
         The National Union of Students which represents and
    protects civil and social rights as well as the legal interests of
    students.
         The Centre for Quality Assessment in Education (see below).
    7.2 Legislation on higher education and provision of quality
    assurance
    The legislation on research and higher education promulgated in
    1991 stated that "qualification requirements for institutions of


                                       75
    higher education and research institutes shall be established by
    the Government of the Republic of Lithuania upon the proposal of
    the Science Council of Lithuania" (Article 15). New legislation on
    higher education is in preparation.
    Qualitative Regulations of Higher Education as well as Regulations
    for the Establishment and Assessment of Higher Education
    Institutions were approved by the Government in 1993 and
    amended in 1996. The General Regulations of the System of
    Research Degrees and Pedagogical Research Titles of the
    Republic of Lithuania makes provision for doctoral studies. In 1996
    the Ministry of Education and Science approved the Rules of
    Quality Assurance for Institutions of Research and Higher
    Education (herein Rules). All these documents form the legislative
    basis for quality assurance in the Lithuanian higher education
    system.
    Each higher education institution has its statute and as a result of
    the adoption of the Rules, each institution (higher education and
    research) is entitled to nominate persons in charge of quality
    assurance, and to create mechanisms for evaluation of studies
    and research.
    The evaluation of basic studies, master's courses and research is
    organised by the Centre for Quality Assessment in Higher
    Education established by the Ministry of Education and Science in
    January 1995. The Centre fulfils the following functions in the field
    of assessment of research and higher education institutions:
        organises peer review of institutions
        organises the expertise of basic studies and master's
    courses, included on the register of the Ministry of Education and
    Science


                                     76
         co-ordinates    the   regular    self-evaluation   process    of
    institutions using a uniform methodology
         organises the quality expertise (peer review) of research
    activity and higher education institutions, combining it with the self-
    evaluation process
         accumulates, analyses and publishes information on the
    conclusions of the self-evaluation and peer review process
         formulates and summarises recommendations on quality
    enhancement for the scientific and pedagogical activities of
    institutions.
    The Centre if financed by a state budget with some additional
    financial support supplied by Lithuanian and foreign organisations.
    7.3 Implementation of quality assurance in higher education
    The evaluation of research and higher education in the country
    commenced in 1994 and as a result some recommendations and
    proposed rankings of institutions were made. In 1995 research in
    Lithuania was evaluated by foreign peers during a project
    organised by the Research Council of Norway. Both exercises
    recommended the creation of a national structure for regular
    assessment and this led to the establishment of the Centre for
    Quality Assessment in Higher Education.
    In 1995 the Centre organised the self-evaluation and peer review
    of 11 research institutes. Evaluation procedures based on the four
    stage model and the experience of western European countries
    were approved by the Conference of Rectors and the Science
    Council. In 1996-1997 the first assessments of study programmes
    have been focused on management studies and master's studies.
    Experience of assessments has highlighted issues related to the
    peer review system. These include conflicts of interest of the peers


                                      77
given that Lithuania is a small country and this has shifted the
focus to foreign experts. However the use of foreign experts has
been restricted because of issues relating to translation needs and
expense.
Two kinds of the evaluation of study programmes are foreseen:
comprehensive and partial evaluation. Partial evaluations check
that all qualitative requirements are satisfied. If they are not, a
comprehensive assessment is made, and both the strongest and
weakest aspects of the programme are highlighted and a strategy
for improvement proposed.
Research is evaluated in terms of efficiency, significance, and
durability. Research is considered in terms of its significance to the
field of science or to the economy and culture of Lithuania. The
durability of research is evaluated on the basis of its use of a
modern theory, modern equipment, qualified research staff, and
the demand for such research.
Sources:
Cizas, A. Current state of quality assurance in higher education in
Lithuania, Unpublished paper.
Documents on Higher Education and Research of the Republic of
Lithuania, Vilnius: Leidybos Centras, 1995.
The Research Council of Norway (1996) Evaluation of Research in
Lithuania, Vol. 1 and Vol. 2, Oslo.
8. Former Yugoslav Republic of Macedonia
8.1 National and international context of higher education
The Former Yugoslav Republic of Macedonia covers an area of
25,713 square kilometres with a population of two million (66
percent Macedonians, 22 percent Albanians, 4 percent Turks, and
various others).


                                 78
    There are two higher education institutions in the Former Yugoslav
    Republic of Macedonia. Both are public universities: Ss. Cyril and
    Methodius University in Skopje and University St. Kliment Ohridski
    in Bitola. Both offers diplomas at the undergraduate and the
    graduate levels, the former in all fields and the latter in a few fields.
    The new law, which is still in draft form, expects the creation of
    private institutions.
    The Ministry is in charge of the higher education budget, legal
    framework, and rules regarding fund distribution and budget
    management. General higher education policies are set by the
    Ministry. Currently, there are no intermediate bodies between the
    ministry and the institutions. The new law expects the constitution
    of two boards:
         The accreditation and evaluation board will be involved in the
    creation of new institutions and the evaluation of existing ones.
         The funding board will assist the Ministry in setting funding
    levels for institutions. It will also develop short- and long-term plans
    for higher education.
    8.2 Legislation on higher education and provision of quality
    assurance
    The legislative framework regarding higher education in the
    Former Yugoslav Republic of Macedonia is in a transition state.
    The old law dates back to the Republic of Yugoslavia and has
    fallen in disuse since some of it is now unconstitutional. Very little
    from that law is used in practice. The new law, which has been
    drafted but has not been presented to Parliament yet, is the basis
    for current practice. This law was developed under the Council of
    Europe's supervision.



                                       79
The Ministry approves the funding of new programmes but this
approval is merely a financial feasibility study procedure. The
academic aspects of new programmes are developed and
evaluated by the institutions themselves, through a university
committee.
At the moment, government regulations guide the promotion and
recruitment of academic staff. This, with the evaluation of research
through grant proposals, are the two formal activities currently
linked with quality assessment.
Academic staff, depending on their level, are reviewed every 3 to 6
years. There is no tenured position. Rather, positions are
advertised, applications collected and evaluated by a referee
commission (consisting of 3 professors) who write a public report.
A council of professors (including all professors and teaching
assistant representatives) conducts elections on the basis of this
widely-distributed report.
Curricula should be reviewed every five years, but this is not the
current practice.
The new law refers to minimal standards but states that it is the
universities which will define them. In this respect, it is important to
point out that, because of the small size of the country, the Ministry
and the institutions work together to develop legislation. Thus, the
legislative culture requires that the law set a legal framework and
then detailed procedure be developed by the institutions.
Standards regarding the physical environment, equipment and
laboratories have now been developed and adopted by the Ss.
Cyril and Methodius University. They will probably be adopted by
the University St. Kliment Ohridski as well and thus become
national standards.


                                  80
8.3 Implementation of quality assurance in higher education
The new accreditation board will accredit all new institutions and
evaluate current ones. The process can start at the initiative of the
Ministry, the institutions or the board itself. There will be no link
between funding and evaluation.
The board will be autonomous from the Ministry and the
institutions. It will be appointed by the Parliament for four years. It
will be funded from the State budget. Its technical staff will be
drawn from the Ministry. It will be a fifteen-member body consisting
of 9 academics, 3 members of the Academy of Science and 3 from
government.
The new law does not refer to accountability, except incidentally.
The precise evaluation and accreditation procedures will be
developed by the board. The law does not specify what these
should be. In the short term, this is the primary need of the
country.
Sources:
Interview,   Atanas   Avramovski,      Undersecretary,    Ministry   of
Education and Physical Culture, Former Yugoslav Republic of
Macedonia
Interview, Vladimir Dukovki, Professor of Mechanical Engineering,
Ss. Cyril and Methodius University
Interview, Violetta Panovska, Vice-Rector, University St. Kliment
Ohridski
Interview, Vladimir Pavlovski, Vice-Rector, Ss. Cyril and Methodius
University
9. Poland
9.1 National and international context of higher education



                                  81
Poland covers an area of 312,520 sq. km and has about 39 million
inhabitants.
Poland has 90 higher education institutions (1993 data) in the
public sector. A separate sector of 'polytechnic' type higher
education institutions has been approved in an amendment to
legislation in June 1997. Alongside regular courses, non-regular
and extension courses exist. Higher education institutions gain
additional money through these: students on regular courses do
not pay fees, others do. Most of the higher education institutions
are concentrated in 22 larger cities and towns ('academic centres'),
where about 75 per cent of students, teachers and researchers are
found.
The 1990 Act on Higher Education allows for private higher
education. In September 1994, there were 49 private higher
education institutions, in 1997 there were more than 130. The
number of students in the private sector is much smaller than the
number of students in the public sector of higher education. Private
higher education institutions are mostly found outside the
'academic centres', indicating that they have an outreach function
to the regional student markets (together with distance education,
organised also by public higher education institutions).
Private higher education institutions may charge tuition fees; the
Ministry of National Education may also grant state funds to private
higher education institutions.
Of the 90 public higher education institutions, 11 are general
universities. The others are more specialised institutions, such as
technical universities (18), medical academies (10), etc. (Note that
some 35 of these are under tutelage of other ministries such as
Health,   Culture   and   Transport).   Private   higher   education


                                 82
institutions mostly specialise in the areas of business studies, law,
etc.
The legal framework is set by the Act on Schools of Higher
Education (1990). It confers wide autonomy on the higher
education    institutions.   Lower-level   regulations   (ministerial
executive regulations) and the higher education institutions'
individual statutes provide the details. The Minister of National
Education's actions are to some extent overseen by the Main
Council on Higher Education.
Higher education institutions may be founded by natural or legal
persons, both Polish and foreign, under authorisation by the
Ministry of National Education.
The degree of autonomy enjoyed by public higher education
institutions depends, under the Act, on some criteria, mainly: the
number of professors (a personal title, controlled by a national
body) and the number of faculties or units having the right to award
second doctorates (doctor habilitatus). Larger universities are
independent with respect to statutes, study regulations, admission
requirements, and they may also establish and dissolve faculties.
In smaller universities such matters require approval by the
competent minister. Students are admitted by individual faculties,
which may set their own admission exams, and decide the
numbers of students to enroll in regular and 'extra-mural'
programmes (in agreement with the rector). Faculties set their own
curriculum, and decide on their academic posts. In internal
decision-making, collective bodies (faculty and university councils)
play a major role.
The Main Higher Education Council (Rada Glówna Skolnictwa
Wyszego) is a 50-member body, elected by the academic


                                  83
community. It advises the Ministry of National Education on,
amongst others, funding criteria and draft acts. Also, it sets criteria
for establishing new faculties.
The national Committee for Scientific Research (Komitet Badan
Naukowych), the state organ for science and research policy as
well as the national research council, is mainly known for its
competitive distribution of research funds to researchers in higher
education. The majority of its members are elected by the
academic community.
The Central Commission for Academic Titles and Degrees is the
national body that controls the conferment of the professor-title
(given by the President of the Republic), and supervises the
academic titles and degrees granted by the higher education
institutions and research institutions.
9.2 Legislation on higher education and provision of quality
assurance
The Acts on Schools of Higher Education (12 September 1990)
states that the establishment of higher education institutions must
be authorised by the Ministry of National Education, and that this
includes input criteria, especially academic staff numbers. In
addition, the system retains control over minimum programme
requirements through the Rada Glówna and conferment of the
highest academic degrees are controlled by a national body. The
Central Commission for Academic Titles and Degrees directly
controls the conferment of the professor-title and authorises
faculties to award the habilitation degree. Quality is the
responsibility of the autonomy of the higher education institutions
(or faculties).



                                  84
In an amendment to the 1990 Acts of June 1997, a binary system
was created, recognising professional education (up to the level of
licentiat degree) as part of higher education. The intention is to
start teaching in these new institutions in September 1998.
New draft legislation is being developed and its submission to
Parliament (the Sejm) is expected in early 1998. It may contain
provisions for an Accreditation Body or separate legislation may be
developed.
9.3 Implementation of quality assurance in higher education
At present no quality assurance agency exists in Poland. However,
the White Paper on professional higher education makes provision
for an accreditation system which may be extended to the whole
higher education system at some stage.
A number of Tempus projects at universities have explored models
of quality assurance and graded accreditation systems for
programmes of study have been proposed. However, these have
not been followed up at national level. A number of networks on
quality assurance have been established throughout Poland
comprising interested people from various centres such as the
Centre for Science Policy and Higher Education at Warsaw
University and the Centre for the Development of Education in
Economics at the Warsaw School of Economics.
Sources:
Act on Higher Education, 12 September 1990.
Ministry of National Education (1994) Acts on School of Higher
Education, The Academic Titles and Academic Degrees, Warsaw.
Bialecki, I. et al. (eds.) (1996) Education in a Changing Society,
2nd revised edition, Background report for OECD review of Polish
education, TEPIS Publishing House, University of Warsaw.


                                85
    Council of Europe (1994) Report of the Advisory Mission on
    Quality Assessment and Accreditation.
    Dbrowa-Szefler, M. & Wójcicka M. (1997) Poland in: Education for
    the Transition, Part III. Higher education policy in Central and
    Eastern Europe, Country Reports, Civic Education Project,Institut
    für die Wissenschaften vom Menschen.
    10. Romania
    10.1. National and international context of higher education
    Romania covers an area of 237,500 sq. km. and has a population
    of 23.3 million.
    Romania has 35 state universities, 13 state non-universities
    institutions and numerous private education institutions.
    Higher education institutions are established by law, as non-profit
    making organisations, subject to prescribed academic evaluation
    and accreditation standards (1993 Accreditation Law, Articles 1
    and 2). There are two stages to the process of accreditation
    (Article 3):
         provisional functioning - the right to organise entrance
    examinations (Article 5)
         accreditation - the right to organise the graduation
    examination and to award diplomas recognised by the Ministry of
    Education (Article 6).
    Article 3 also specifies that accreditation must (sic) be obtained
    two years after the graduation of the first class, but does not
    explain what value the diplomas have if accreditation is not
    obtained.
    The Ministry of Education is the principal state organisation with
    responsibility for higher education. It has several councils and
    advisory bodies such as:


                                     86
        National Council for Higher Education Research
        The National Council for the Recognition of Academic Titles,
    University Diplomas and Certificates
        National Higher Education Funding Council
        National Education Reform Council
        National Rectors' Council.
    The control of the Ministry of Education over institutions is shown
    by Article 14 of the Accreditation Law which states that "The
    university graduation examinations... carried out in accredited
    institutions...will be organised in conformity with the methodology
    approved by the Minister of Education". However, it should be
    noted that the accreditation body, the National Council for
    Academic Evaluation and Accreditation (CNEAA) is not part of the
    Ministry of Education, or even the Government, but is responsible
    to Parliament (see below).
    10.2 Legislation on higher education and provision of quality
    assurance
    The legal framework is provided by the Accreditation Law of 1993,
    the 1996 Law on Education and the CNEAA Statutes.
    The CNEAA was set up under Article 4 of the Accreditation Law.
    Its procedures are governed by the Accreditation Law and the
    Statutes. The CNEAA consists of 19 - 21 members, who are
    appointed by Parliament. The membership of CNEAA is renewed
    every four years on a rotating basis. The President, Vice-President
    and Secretary of CNEAA are elected by the Council on the
    recommendation of the Minister of Education.
    Evaluation commissions are appointed by CNEAA of seven - nine
    members who are replaced every four years by rotation. There are
    eight commissions:


                                      87
         fundamental sciences
         social sciences
         human sciences
         economic and management sciences
         engineering sciences
         medicine and pharmacy
         agricultural and veterinary medicine
         arts and sports
    The operational costs of the CNEAA and its evaluation
    commissions are covered by its own budget and fees charged to
    institutions for accreditation (Accreditation Law, Article 13).
    10.3 Implementation of quality assurance in higher education
    The Accreditation Law was introduced as a result of the rapid
    expansion of higher education from 1990. Other factors included
    new financial mechanisms, increased institutional autonomy,
    increased international student mobility, and the insistence of
    major universities for a hierarchy of universities to be introduced
    (Mihailescu).
    The four stage model of quality assurance is adopted with some
    modifications at both stages of the accreditation process, that is,
    provisional functioning and accreditation. The four stages are:
    (1) the CNEAA
    (2) an applicant institution provides a self-evaluation report to the
    CNEAA
    (3) the appropriate evaluation commission analyses the report and
    checks it against standards and makes a report. (A site visit is not
    required by the Accreditation Law, although it is referred to in
    CNEAA Statutes.)



                                      88
    (4) CNEAA issues a report and makes a recommendation to the
    Government. (At the first stage of the accreditation process, if the
    recommendation         is   positive,        the   Government   authorises
    provisional functioning of the faculty, college or academic unit.
    Reports are sent to the applicant institution.)
    A number of Articles (6 - 12) exist relating to the timing of
    applications for provisional functioning, and the consequences of a
    negative decision result in closure and possible transfer of
    students of the academic unit concerned. Article 9 states that
    CNEAA must evaluate or re-accredit academic units every five
    years.
    Whilst the focus of evaluation is at the programme level, both
    research and institutional management are also considered. Much
    reference is made to the criteria and standards for academic
    evaluation in the Accreditation Law. References include:
         students and graduates,
         academic staff and the educational environment,
         curricula,
         infrastructure, material and financial means,
         research staff and the research plan,
         research strategy and results,
         infrastructure, material and financial means concerning
    scientific research,
         services within the institution,
         public services.
    In 1993 CEPES-UNESCO organised an international conference
    in Oradea on policies for quality evaluation and accreditation of
    higher education institutions. In compliance with the agreement
    made between the Government of Romania and the World Bank in


                                            89
November 1993, pilot projects were launched under the auspices
of CEPES in order to test the instruments and the academic
assessment procedures for the accreditation of higher institutions.
The pilot projects were carried out between November 1993 and
March 1994.
Sources:
Decree for the Promulgation of the Law on the Accreditation of
Higher Education Institutions and the Recognition of Diplomas,
Bucharest, 1993.
Draft Statutes of the National Council for Academic Evaluation and
Accreditation - undated.
Law on Education, 1996.
Law on the Accreditation of Higher Education Institutions and the
Recognition of Diplomas, 16 December 1993.
Bruckner, I. Case study of an external evaluation, unpublished
paper.
Dinca, O. and Radu, D. Financing of Higher Education in Romania,
unpublished paper.
Mihailescu, I. The System of Higher Education in Romania,
unpublished paper.
Mihailescu, I. Glossary of Terms on the System of Higher
Education in Romania, unpublished paper.
Quality in Higher Education, From Pilot Project to Academic
Assessment and Accreditation, Bucharest, 1994.
Sterian, P. Quality Assurance Systems in Romanian Higher
Education, unpublished paper.
11. Slovak Republic
11.1 National and international context of higher education



                                90
     The Slovak Republic covers an area of 49,000 sq. km. and has
     nearly 5.4 million inhabitants.
     There are 21 higher education institutions (including one police
     and two military academies, which are officially part of the higher
     education system in Slovakia). There is no private higher
     education at present. There are both general and specialised
     universities, five and more technical or specialised engineering
     (agriculture, wood techniques, etc.) higher education institutions.
     Other specialised institutions focus on medicine and economics.
     Bodies concerned with higher education include:
1.         The Ministry of Education which creates conditions for the
     development of higher education institutions and of higher
     education,   coordinates    the   activities   of   higher   education
     institutions, allocates the finances allotted from the state budget
     and checks on their use in compliance with the state policy,
     registers the statutes of higher education institutions and faculties,
     decides upon the proposal or withdrawal of the right of higher
     education institution to hold the state examinations, 'examina
     rigorosa', to perform dissertation examinations, to hold PhD
     studies and PhD examinations etc.
2.         The Higher Education Council, which is composed of
     representatives of all higher education institutions. It is a self-
     governmental body of the higher education institutions that
     represents the institutions, especially in relation to the Ministry.
     The Council gives opinions on proposals of the Ministry concerning
     the allocation of funds and measures concerned with higher
     education institutions. Membership is a matter of honour.
3.         The University Student Council which represents the
     students' interests from the outset, gives opinion on essential


                                       91
     questions, proposals, and measures concerning higher education
     studies. Membership is a matter of honour.
4.          The Accreditation Commission is set up by the Government
     of the Slovak Republic as its advisory body. The Commission acts
     directly as an advisory body for the Minister of Education. It mainly
     gives its opinion on proposals of the establishment, merger, split
     and dissolution of the higher education institutions and faculties
     and on proposals concerning fields of study at those institutions. It
     also gives opinion on the application of the founder of a non-state
     higher education institution for financial support.
5.          The Academy of Sciences which may be seen as an
     intermediary body, being a collection of the academic oligarchy. In
     relation to higher education it is an external scientific workplace
     which can (upon the proposal or statement by the Accreditation
     Commission) upon approval by the Ministry of Education have the
     right to provide PhD study at least in one scientific discipline or
     field of art.
6.          The Slovak Rectors Conference is a non-governmental body
     which includes all of the universities and can represent the
     interests of higher education institutions especially in relation to the
     Ministry and Government.
     11.2 Legislation on higher education and provision of quality
     assurance
     TheHigher Education Act No 172/1990 of the Law Code of 4 May,
     1990 as recently amended by changes and supplements
     implemented by the Act of the National Council of the Slovak
     Republic of 1996, represents a legal norm for the activities of the
     higher education institutions (both private an state-operated) in the
     Slovak Republic and came into force on 20 November, 1996.


                                       92
Quality in higher education is guaranteed through
· the Article 15 of the Higher Education Act No. 172/1990:
"Ministry of Education decides upon the proposal or statement of
the Accreditation Commission on acknowledgement or withdrawal
of the right of the higher education institution or faculty to hold the
state examinations, examina rigorosa, to hold PhD study and PhD
examinations to perform dissertation examinations and the
defences of dissertations, to award the scientific/academic degree,
as well as to perform habiltation procedures and procedures for
nominating professors".
· the Article 17 of the Higher Education Act No. 172/1990:
"The Accreditation Commission shall
- give its opinion on proposal of the establishment, merger, split,
and dissolution of the higher education institutions and faculties,
and on proposals concerning fields of study at HE institutions and
faculties,
- give its opinion on the application of the founder of a non-state
higher education institution for financial support of its activities
from the state budget."
The    Accreditation   Commission      derives       from   the    former
Czechoslovak Accreditation Committee. It was (re-)founded in
1993 and is located in Bratislava. It is an independent body,
consisting of 21 academics from Slovak universities and from the
Academy of Sciences and representatives from non-academic
institutions and industry. The Commission is concerned with
accreditation and quality assurance of all state universities. With
regard to accreditation the foci of the Commission's activities are
programme quality, leading to programme and institutional
accreditation.   Quality   assurance     is   also     concerned     with


                                  93
programme quality leading to funding categories. The Ministry of
Education financially provides for activities of the Commission by
special allocation of funds for its secretariat. Membership of the
Accreditation Commission is a matter of honour.
11.3 Implementation of quality assurance in higher education
The Accreditation Commission evaluates periodically the quality of
higher education institutions, based on evaluation of faculties (not
whole higher education institutions). The faculty which is under the
process of evaluation prepares a self-evaluation record/report
which is submitted to the respective working group of the
Accreditation Commission. The working group has to prepare
conclusions according to the written report of the faculty and the
results of a peer review (peers are from the working group) and
submit them to the Accreditation Commission. The Commission
votes on the recommendation.
The stated purposes of quality assurance are accountability to the
government, funding decisions, to help maintain standards (hence
transparency of degrees for employers etc.), and to improve
educational provision.
With regard to the method of quality assurance, the four stage
model is applied, although there is greater emphasis on
accreditation than on quality assurance as follows:
(1) there is an agency
(2) the higher education institutions provide information (although
this is not a self- evaluation to the agency)
(3) peer committees review the information (there are site visits for
self-evaluation and for accreditation)
(4) decisions, i.e. summative ratings, are published.



                                  94
    Other features of quality assurance in the Slovak Republic are as
    follows:
         Non-academic peers can be involved in the work of the
    Commission as well as international peers (but they can work only
    in working groups, not directly in the main body). The possibility of
    involving non-academic peers is stated in the higher education act.
         Students' opinion is taken into account but students are not
    present in the decision making process of the Accreditation
    Commission.
         The    Accreditation      Commission   prepares    reports   in
    consultation with the institution evaluated. Each report is submitted
    to the Ministry of Education.
         The Commission has an obligation to submit a report to the
    Government every two years.
    During the last three to four years the Slovak higher education
    institutions have been involved in several international evaluations.
    For instance the University of Veterinary Medicine in Kosice was
    evaluated by the Commission of European Veterinary Universities.
    At present the two biggest universities - the University of
    Comenius in Bratislava and the Slovak Technical University in
    Bratislava are in the process of evaluations by CRE. In addition the
    Slovak Technical University was involved in a process of quality
    assurance by a Group of experts from IESC. Technical universities
    took part in an evaluation performed by FEANI.
    Some specific projects focused on evaluation and quality
    assurance processes in higher education institutions have been
    prepared through the TEMPUS programme.
    Higher education institutions and the academic community have
    accepted the process of self-evaluation and accreditation as a part


                                       95
of quality management in higher education. However, the process
of implementation of accreditation procedures is sometimes very
slow, and it is felt within the country that the administrative skills of
the Commission need to be improved as well as quality assurance
and accreditation procedure skills. On the other hand, the
Accreditation Commission is quite a new element in academic life
in the Slovak Republic. The Commission works very independently
and recognises that the quality of higher education is the most
important topic of its scrutiny.
Sources:
Ministry of Education (1990) Higher Education Act No 172/1990
amended by changes and supplements of the Act of the National
Council of the Slovak Republic No 41/1994 and the Act of the
National Council of the Slovak Republic No 324/1996.
Bock, K.H. (1997) Student Handbook: A Directory of Courses and
Institutions in Higher education for 29 Countries Which are Non-
members of the European Union, Council of Europe, Bonn.
Frazer, M.J. (1997) Report on the Modalities of External Evaluation
of Higher Education in Europe: 1995-97, Higher Education in
Europe, Vol XXII, No 3, p.349-401.
Ministry of Higher Education (1998) The Concept of the Higher
Education Development, Ministry of Education of the Slovak
Republic, Bratislava.
Melnik, M. Background to the Quality Assurance System in the
Slovak Republic, unpublished paper.
12. Slovenia
12.1 National and international context of higher education
Slovenia covers an area of 20,000 sq. km and has a population of
2 million.


                                   96
Slovenia has two state universities, the University of Ljubljana and
the University of Maribor, and seven free-standing higher
education institutions.
The Articles of the 1993 Higher Education Act (HEA) states that a
higher education institution may be established by "Slovene or
foreign natural (sic) or legal persons" and that the state shall
establish public higher education institutions. Article 10 states that
a "university is a legal entity", and Article 11 indicates that non-
public higher education institutions shall also be legal entities.
Articles 10 and 15 refer to a charter of public higher education
institutions (adopted by Parliament), and according to Article 19 all
HEIs have to possess a constitution. Article 16 refers to a register
of higher education institutions kept by the Ministry of Education
and Sport. Entry into this register requires there to be proper
premises and facilities, adequate academic staff and programmes
which have been adopted according to Article 32. The Ministry is
responsible for procedures and terms relating to entry (and
removal) from the register.
12.2 Legislation on higher education and provision of quality
assurance
The Higher Education Act (HEA) was adopted by Parliament on 7
December 1993. It contains 100 articles, of which six refer directly
to quality. Articles 16, 32 and 35-42 refer to approval of higher
education institutions and programmes.
The Act refers to two bodies with responsibility for quality:
The Council for Higher Education (Articles 48 - 51)
· Its members are appointed by Government for four years .
· It consists of a President and 11 members (six are professors
and scientists nominated by higher education institutions, ex-officio


                                  97
are university rectors, President of Academy of Sciences and
Arts).
· Among its tasks are that "it shall determine the criteria for
assessment of study programmes with regard to the international
comparability and duration of studies and that it shall determine
criteria for assessing the quality and effectiveness of teaching,
research, art and professional activities".
The Quality Assessment Commission (Article 80 and 94)
· The Commission is "created by higher education institutions"
within one year of the Higher Education Act coming into force.
· Its task is to monitor and assess the quality and effectiveness of
teaching, research, art and professional activities of higher
education institutions.
· It is composed of "representatives of all scientific and art
disciplines and professional fields".
· It shall obtain the advice of students.
· It shall conduct its business according to rules determined in co-
operation of the senates of higher education institutions and
criteria defined by the Council for Higher Education and the
Council for Research and Technology
· It shall publish an annual public report to senates of higher
education institutions and the two Councils.
In May 1994 the Council for Higher Education adopted the Criteria
and Procedures on the Accreditation of Study Programmes and
Higher     Education      Institutions.     In   March   1995    the
Recommendations on Financing Study Programmes, which
mention quality as one of the basic principles, were adopted.
Although the Act does not mention funding, the Council for Higher
Education is funded directly through the state budget whereas the


                                   98
Quality Assessment Commission is funded through the budget for
public services in higher education.
12.3 Implementation of quality assurance in higher education
The   work    of   the   Higher   Education   Quality   Assessment
Commission is still at an early stage. The Ministry of Education
and Sport, however, began quality-related activities soon after the
adoption of legislation. In March 1994 Slovenia took part in an
international seminar on quality assurance and accreditation in
higher education organised by OECD and EU and in December
1994 Slovenia was represented in a national workshop on self-
evaluation of higher education and peer review. In addition
Slovenia attended all seminars, workshops and study visits
organised by the Council of Europe - Legislative Review
Programme, and in spring 1996 the University of Ljubljana
completed the institutional quality audit by CRE.
It is also worth noting that the Centre for Evaluative Research at
the University of Ljubljana is carrying out two projects concerning
quality assurance in higher education: "Evaluation of Higher
Education" and "Evaluation of Research in Slovenian Higher
Education". Both projects are financed by the Ministry of Science
and Technology.
The first project aims to prepare a draft model for introducing a
system of quality assurance in Slovenian higher education. The
Slovenian system of quality assurance will take into consideration
foreign experiences and adapt them to national circumstances and
Slovenian higher education. It is suggested that the new system
will consist of elements that are common to other developed
European quality assurance systems, that is a co-ordinating body,
self-evaluation undertaken within institutions, external peer review,


                                  99
publication of reports, and an indirect link to funding. Since 1996
some institutions of higher education and study programmes have
participated on a voluntary basis in the project. Once this
experimental phase has been completed, it will be possible to
analyse and assess whether the quality assurance system which is
being proposed will meet the expectations of the stakeholders and
its initial goals (Kump 1997).
The second project is looking at the methodology for quality
assurance in the field of research and in particular aims to
contribute to:
· the transparency of research activity in Slovenian higher
education
· elaborate methodological approach to quality assurance of
research in higher education
· co-ordinate the instruments and procedures for quality assurance
of research with those for quality assurance of teaching in higher
education.
Finally, the Ministry of Education and Sport has financed a project
on Quality Assessment and Planning in Higher Education and
another on Information Support of Quality Assessment in Higher
Education, implemented by the University of Ljubljana. The former
produced the handbook "Self Evaluation in Higher Education", the
latter developed indicators for quality assessment.
Sources:
Higher Education Act, 7 December 1993. (Article numbers refer to
this Act).
Kump, S. (1997) Introduction of systemic quality assurance in
Slovenian Higher Education, Mediterranean Journal of Educational
Studies, Vol. 2, Number 1.


                                 100
Kump, S. (1997) Country Liaison Report, Multi-country Programme
in Higher Education, unpublished paper




                              101

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:15
posted:9/9/2011
language:English
pages:101