Synthesis Evaluation Study
on Technical Cooperation Projects
in the Primary and Secondary Science and Mathematics
Japan International Cooperation Agency
BACKGROUND AND OBJECTIVES
(1) BACKGROUND AND OBJECTIVES
As part of the effort towards the improvement in the quality of basic education, the Japan
International Cooperation Agency (JICA) has implemented twelve Technical Cooperation Projects in the
primary and secondary science and mathematics education sector so far (as of the end of February 2004).
However, a systematic evaluation on the education projects has not been conducted yet, thus the review of
past and on-going education projects is strongly required. Within this context, JICA conducted the
Synthesis Evaluation Study on Technical Cooperation Projects in the Primary and Secondary Science and
Mathematics Education. The Evaluation Study addressed the following two objectives.
1) to review and classify the past and on-going twelve projects by JICA on primary and secondary
science and mathematics education.
2) to analyze project planning and modification processes as well as the components of cooperation,
to extract contributing and inhibiting factors of the projects, and to draw lessons learned for the
improvement of future project formulation and implementation processes.
(2) EVALUATION QUESTIONS
The evaluation questions of the Evaluation Study were as follows:
1) Summary and classification of Projects
Under what social and educational situation were the Projects implemented? What elements or
criteria could be used to classify the Projects?
2) Contributing and inhibiting factors
What were the contributing and inhibiting factors affecting the impacts of primary and secondary
science and mathematics education projects?
3) Lessons learned
What were the lessons learned extracted from these primary and secondary science and mathematics
(3) PROJECTS OF THE EVALUATION STUDY
Twelve JICA projects were selected for the Evaluation Study. Among them, five were completed and
seven were on–going projects. Out of the twelve projects, field studies for detailed evaluation were
conducted on two projects in Kenya and two in the Philippines. The following table shows the list of
projects evaluated under this Evaluation Study.
Table 1: Projects of the Evaluation Study
Region Country Project Title Abbreviation Project Period
Asia Philippines The Package Cooperation for the Development Package 1994/6-1999/5
of Elementary and Secondary Science and Cooperation
Strengthening of Continuing School Based SBTP 2002/4-2005/4
Training Program for Elementary and Secondary
Science and Mathematics Teachers
Indonesia Project for Science and Mathematics Teaching IMSTEP 1998/10-2003/9
for Primary and Secondary education
Cambodia Secondary School Teacher Training Project in STEPSAM 2000/8-2004/9
Science and Mathematics
Africa Egypt Development of Creativity Lessons for Primary - 1997/12-2000/11
Improvement of Science and Mathematics - 2003/4-2006/3
Education in Primary Schools in Egypt
Kenya Strengthening of Mathematics and Science in SMASSE I 1998/7-2003/6
Strengthening of Mathematics and Science in SMASSE II 2003/7-2008/6
South Mpumalanga Secondary Science Initiative MSSI I 1999/11-2003/3
Mpumalanga Secondary Science Initiative Phase MSSI II 2003/4-2006/4
Ghana Improvement of Educational Achievement in STM 2000/3-2005/2
Science, Technology and Mathematics in Basic
Latin Honduras Improvement of Teaching Method in PROMETAM 2003/4-2006/3
2. FRAMEWORK OF THE STUDY
(1) EVALUATION STUDY TEAM AND EVALUATION STUDY PROCESS
The Evaluation Study was conducted between July 2003 and February 2004. The Evaluation Study
was organized by the then Office of Evaluation and Post Project Monitoring, Planning and Evaluation
Department of JICA. The Evaluation Study Committee was established, consisting of JICA educational
task team representatives, the project operation division representatives, and individuals with relevant
expertise appointed externally. Two consultants were assigned to conduct the Evaluation Study based on
the strategies presented by the Evaluation Study Committee.
During this Evaluation Study, an in-country (Japan) study was first conducted to review and analyze
all twelve Projects. The methods adopted included document reviews, a questionnaire survey, and
interviews with relevant individuals. During this stage, the Approach taken by each Project was specified,
logic models were constructed, and the Projects were classified by Approaches.
Among the twelve Projects, four projects in Kenya and the Philippines (two projects in each country)
were selected for field studies. The Package Cooperation for the Development of Elementary and
Secondary Science and Mathematics Education in the Philippines was the first JICA Technical
Cooperation Project in basic education and the on-going Strengthening of Continuing School Based Training
Program for Elementary and Secondary Science and Mathematics Teachers (SBTP) is expanding its activities in
terms of geography and quality, utilizing the lessons learned from the Package Cooperation. The Kenyan
Strengthening of Mathematics and Science in Secondary Education (SMASSE I) was the first Technical
Cooperation Project in basic education in Africa and Strengthening of Mathematics and Science in Secondary
Education: SMASSE II (SMASSE II) is enhancing the outcomes achieved during Phase I in terms of
geography and quality. These four Projects, identified based on their significance and geographical
balance, were selected for field studies to draw lessons learned that can be useful for future science and
mathematics education projects. The activities conducted during the field studies included document
reviews, questionnaire surveys, interviews and site visits.
The results from the field studies on the above four projects and the remaining eight projects
investigated in Japan were analyzed and compiled in the final report.
(2) EVALUATION STUDY METHODOLOGY
Based on the three evaluation questions, the Projects were classified according to the Approaches
taken and were then analyzed using logic models.
1) Classification of Approaches
Under the Projects on primary and secondary science and mathematics education, a variety of
activities were conducted, including teacher training and the improvement and dissemination of textbooks
and materials to enhance the quality of education. Approach, in the context of this Evaluation Study,
refers to a group of activities or measures taken to achieve the Projects’ objectives. Figure 1 shows the
concept of Approach. In classifying the Approaches taken by the twelve Projects, a logic model (defined
in the next section) was constructed for each Project.
Figure 1: Conceptual Chart of Approach
Activity Activity Activity Activity
Activity Activity Activity Activity
Approach A Approach B
2) Analysis Using Logic Models
The logic model, in the context of this Evaluation Study, refers to a visual representation of the
cause-and-effect linkages present in each Project. Figure 2 shows the conceptual chart of the logic model.
During the Study, a logic model was constructed for each Project based on its Project Design Matrix
(PDM) or a Logical Framework in order to demonstrate and verify the cause-and-effect logic between
Project inputs and the final outcome.
Figure 2: Conceptual Chart of Logic Model
Intermediate Intermediate Important
Outcome 1 Outcome 2 Assumptions
Output 1 Output 2 Output 3 Output 4 Output 5
Activity 1 Activity 2 Activity 3 Activity 4 Activity 5
Input 1 Input 2 Input 3
3. SUMMARY OF PROJECTS (EVALUATION QUESTION 1)
(1) CLASSIFICATION OF PROJECTS BY CHARACTERISTICS
The first Project in the primary and secondary science and mathematics education sector was the
Package Cooperation launched in the Philippines in 1994. Since then, a considerable number of Projects
in the sector was implemented. Figure 3 shows the development and expansion of primary and secondary
science and mathematics education Projects from late 1990s to 2000. From a geographical point of view,
there were seven Projects in Africa, four in Asia, and one in Latin America. There has been a remarkable
increase in the number of Projects in Africa after 2000. There are two reasons for such an increase. First,
Japan defined the education sector as a priority area of assistance in the 1990s. Second, assistance to
African countries increased in general.
Figure 3: Development of Primary and Secondary Science and Mathematics Education Projects
Year 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008
Philippines Package Coorperation Follow-up
Cambodia Secondary School Teacher Training
Africa Egypt Development of Creativity
Improvement of Science and
Kenya SMASSE I
South MSSI I
Latin America PROMETAM
Remarkable differences were seen in the size of countries, population size, and social situations of
these eight countries. Nevertheless, the improvement of the quality of science and mathematics teachers
in primary and secondary education received priority in these countries. The classification of the twelve
Projects is shown as follows.
1) Duration of Projects
With regard to project duration, one Project had seven years, four Projects had five years, and seven
Projects had seven years. It was noted that out of the five Projects started in 2000 or later, four Projects
had the duration of three years, thus can be concluded that the number of three-year projects has been on
2) Primary and Secondary Education
There were three Projects focusing at the primary education level, five Projects at secondary
education level, and four at both primary and secondary education levels. Along with the expansion of
primary education as a result of assistance, there could be an increase in the secondary school enrollment
and, consequently, could lead to an increased demand for teachers. Therefore, a greater need for
assistance in secondary education could be expected.
3) Science and Mathematics
One Project focused on mathematics and the other eleven Projects simultaneously focused on science
and mathematics. The choice of science or mathematics, or both, was considered to be based on the
request made by the recipient county.
4) PRESET and INSET
More emphasis was placed on teacher training than material development. As to teacher training, ten
Projects conducted training for teachers in service (INSET) and two Projects conducted both INSET and
training for pre-service teachers (PRESET). It was observed that INSET was chosen more frequently than
PRESET because of its tendency to generate more immediate effects.
(2) CLASSIFICATION OF PROJECTS BY APPROACH AND CROSS-SECTIONAL EVALUATION
BASED ON APPROACH
To classify the Projects by Approach, a logic model was first constructed for each Project. The
activities presented in the logic model were then arranged based on the Approaches taken. Through this
analysis, eleven Approaches, or groups of activities, were specified. These included the establishment of
operation and management system, development of teaching materials, training for teachers in service,
and monitoring and evaluation.
The review of the twelve Projects revealed that all of them incorporated several Approaches in
combination, but always included teacher training as a core component. Given the importance of teacher
training as an Approach, the classification of the Projects was made based on two criteria: classification
by INSET or PRESET and classification by the training system. Here, two axes are used to demonstrate
the classification criteria: 1) beneficiaries of training and 2) teacher training system. The horizontal axis
classifies the beneficiaries of training by pre-service teachers and in-service teachers. The vertical axis
distinguishes between the cascade system and the direct teaching system. Through this analysis, the
twelve Projects in primary and secondary science and mathematics education were classified into the
following four types (see Figure 4).
Figure 4 Project Classification by Teacher Training Approach
Philippine Package Cooperation
Egypt Development of Creativity Lessons
Approach with Approach with for Primary Education
main focus on main focus on Kenya SMASSE I
teaching methods teacher
training Indonesia IMSTEP
South Africa MSSI I
Egypt Improvement of Science and
Mathematics Education in Primary Schools
South Africa MSSI
Approach with main Kenya SMASSE II
focus on teacher
Type 1 is the Approach of INSET by cascade system. The experts and counterparts of the Projects
conduct direct training up to teacher trainers. The Projects classified as Type 1 are Package Cooperation
in the Philippines, SMASSE Phase I & Phase II in Kenya, and MSSI I in South Africa.
Type 2 is also the Approach of INSET by cascade system. The difference from Type 1, however, is that
the experts and counterparts of the Projects conduct training directly to teachers at the school level
(cluster training or school training). The Projects classified as Type 2 are STM in Ghana, MSSI II in
South Africa, and SBTP in the Philippines.
Type 3 is the Approach of the development of teaching materials. The Projects classified as Type 3 are
Development of Creativity Lessons for Primary Education and Improvement of Science and Mathematics
Education in Primary Schools in Egypt and PROMETAM in Honduras.
Type 4 is the Approach combining INSET with PRESET conducted mainly at universities. The Projects
classified as Type 4 are IMSTEP in Indonesia and STEPSAM in Cambodia.
The twelve Projects were cross-examined in reference to the above classifications. The following points
were observed as a result of this.
1) Even when the Projects belonged to the same Type, the results of project evaluation on efficiency,
impact and sustainability were different.
2) The efficiency of the Projects was considered to be high in general; however, the evaluation criteria
of efficiency were not clear.
3) In education projects, it takes a long time for the impacts to be recognized; thus, making it difficult
to assess the projects’ impacts on students.
4) Further improvement is required for organizational, institutional, and financial sustainability.
5) As efficiency and impact cannot be assessed clearly, it is difficult to compare the efficiency, impact,
and sustainability among the Projects of different types.
4. CONTRIBUTING AND INHIBITING FACTORS OF THE PROJECTS
(EVALUATION QUESTION 2)
Based on the cross-sectional evaluation outlined above, the Evaluation Study extracted contributing
and inhibiting factors. The following are five elements that were considered as factors determining the
success of primary and secondary science and mathematics projects.
One of the important factors lies at the planning stage of the projects. Needs analysis, the
construction of logic, and the selection of inputs are factors especially significant in influencing the
efficiency of projects.
Means to expand outcome
The teacher training system employed under the primary and secondary science and mathematics
projects were classified into two groups: cascade and cluster systems (school training system). The
establishment of teacher training system is an important factor that influences the impact.
In recent years, many projects included collaboration with Japan Overseas Cooperation Volunteers,
other ODA schemes, local universities and other donors. The collaboration with related agencies in and
outside the projects is a factor that especially influences efficiency.
Governmental assistance and institutionalization has a great impact on projects through assuring
finance and authorization of training on weekdays.
Monitoring and evaluation
A timely modification of project planning through monitoring and evaluation greatly contributes to
the achievement of project purposes.
5. LESSONS LEARNED FROM CROSS-SECTIONAL ANALYSIS (EVALUATION
In the Evaluation Study, the Projects were analyzed based on the above five elements and lessons
learned were drawn as follows. In this Summary, the lessons learned of special priority are presented.
(1) LESSONS LEARNED ON PLANNING
In the planning stage, it is necessary to construct a logic model and formulate a logical framework
(PDM) after sufficient deliberation of cause-and- effect linkages.
The logic model analysis of all the Projects in the Evaluation Study found several examples that need
a reconsideration of the logical construction from the project purpose to the overall goal. While the
logical framework (PDM) employed in projects is generally an effective tool for project management, it
sometimes encounters difficulties in precisely describing complicated cause-and-effect linkages of the
projects. Constructing a logic model at the planning stage leads to a deliberate consideration of the
cause-and-effect linkages of projects and, as a result, to the formulation of a more refined logical frame
Planning of project inputs needs careful consideration with comparison to similar projects as the
quality and scale of inputs has a great influence on sustainability and impact.
In the Package Cooperation in the Philippines, in spite of considerably large-scale inputs, the
sustainability of training system began to disappear as soon as the Project was terminated. On the other
hand, SBTP that followed the Package Cooperation maintained a training system without major costs
while demonstrating sustainability and geographical expansion. One possible reason for this was that the
training system of SBTP was designed with the intention to reduce Japanese inputs and to establish a
training system that could be easily managed by local people. In other words, large-scale inputs may
result in an accelerated expansion of activities in the short-term, but there is a danger of inhibiting
long-term sustainability and impact. To calculate the adequate scale of inputs at the planning stage of
projects, it may help to identify similar projects in and out of implementing agencies and to compare the
necessary costs and outputs.
While formulation of projects utilizing existing resources is effective in terms of efficiency and
sustainability, it is necessary to deliberately consider if the utilization is in line with the objective of
It is effective to formulate a project plan with considerations to the existing resources in the country
as well as to the outcomes and experiences of past projects. However, a clear overall project plan must
first be in place when a priority is given to the utilization of existing resources. Without an appropriate
project plan, the project objectives may be distorted through a mere utilization of resources and
implementation of inputs. An example from the Package Cooperation in the Philippines is relevant in this
regard. In this Project, too much emphasis was placed on the utilization of the Science Teacher Training
Center constructed through Grant Aid and the real objective of the Project was not clearly defined. In the
utilization of resources, a careful consideration should be made so that the inputs are consistent with
(2) LESSONS LEARNED ON EXPANSION OF OUTCOME
In teacher training by cascade system, it is effective to minimize the number of cascade layers and
to conceptualize what is delivered in the training in simple keywords.
Teacher training by cascade system has a tendency for the contents to be delivered to diminish in
proportion to the number of cascade layer added. In the Package Cooperation in the Philippines, training
courses were conducted in three stages, namely, national, regional, and division levels. It was observed
that the training period and the effects of the training were diminished from the national level to the
division level. In the training system of SMASSE in Kenya, the three stages established in the initial
project plan were reduced to two and this modification enhanced the effectiveness of the training. In the
cascade system, the delivery of fundamental concepts, rather than of complicated issues, avoids the
diminishing of the contents, and thus contributes to its effectiveness. In SMASSE in Kenya, the keyword
of "ASEI/PDSI1" was contrived and this made it easy to spread the concept intended in the Project. On
the other hand, if the keyword itself is presented without an underlying context, the message conveyed
may be misunderstood. In the Package Cooperation in the Philippines, the keyword of PWA2 was
adopted, but the keyword was misunderstood as being just conducting experiments in class. Unfortunately
in this case, the misunderstood concept was disseminated through training. In simplifying a concept into a
keyword, it is necessary to organize the concept and to carefully prepare for dissemination, for example,
by producing manuals for training.
The expansion system employing cluster (direct) system and school system is suitable to spreading
the outcome in a geographically limited area. In introducing cluster training, it is necessary to
obtain understanding and support by concerned stakeholders in the area.
Training through cluster and school training systems has an advantage in delivering the effects of the
training not only to teachers but also to schools as well as to the entire school catchment area (cluster). In
STM in Ghana, in response to the high turnover rate of teachers, the support to school training was
strengthened. In SBTP in the Philippines, a training system where schools in the same cluster hold
training in turn contributed to establishing a network among teachers in the cluster who teach the same
subject. The expansion system described above can be especially effective at a primary education level,
because primary schools have closer relationships with local areas. In addition, obtaining more
understanding by principals and school inspectors promotes the participation of teachers. In cluster and
school training systems based at schools and local communities, holding a workshop for principals and
school inspectors is important for gaining understanding and support at school and community levels.
The above training system depends on the conditions of education administration system, level of
education (primary/secondary), and geography. The training system should be designed with
considerations for these conditions.
It is important to consider the following three conditions: education administration, the level of
education, and geography. In terms of education administration, the cascade system is adequate in a
situation where administration capacity of both central and local levels is high, while the cluster system is
adequate where decentralization has been established. At a higher secondary education level, schools are
1 ASEI (Activity, Student, Experiment, Improvement) expresses important attitude of teaches toward
science and mathematics classes. PDSI means four stages (Plan, Do, See, Improve) to improve
2 Practical Work Approach: Teaching method incorporating experiments
located in a considerable distance and the cascade system is desirable, as it requires less occasions to get
together. At the primary education level, on the other hand, education is conducted in close relationship
with local communities and therefore, cluster training is recommended, as those concerned can get
together more often. From a geographical point of view, in areas where transportation systems are not
well developed, it is difficult to continuously conduct cluster training. Thus, a large-scale cascade training
held during vacation time is suitable.
(3) LESSONS LEARNED ON COLLABORATION
Collaboration with local universities or academic institutions is effective in terms of the quality
control of training, sustainability, and incentives for teachers. In promoting collaboration, it is
necessary to clarify the organizational relationships surrounding each institution.
Collaboration with local universities helps to share local knowledge and experiences, to control the
quality of training, to achieve sustainability, and to promote participation in training. Under SBTP in the
Philippines, a new evaluation method was introduced as a result of the Project’s collaboration with a
university. Under MSSI in South Africa, a university provides a training course on teacher qualification at
a discounted rate and this enhances the motivation to participate in training. In pursuing collaboration
with universities, it is important to clarify responsibilities as well as organizational relationships
surrounding concerned institutions in order to avoid the diffusion of ownership.
Collaboration with Japan Overseas Cooperation Volunteers (JOCV) program can be a great
contributing factor. A precondition for collaborating with JOCV program is to formulate a full
agreement between experts and volunteers on the focus and activities of the projects.
Among the Projects of this Evaluation Study, JOCV program was frequently chosen as a partner for
collaboration. Collaboration with the JOCV program took two forms: organized collaboration and flexible
collaboration. Under organized collaboration, the JOCV program was officially designated as a
component of project inputs and a certain level of outputs was expected out of this arrangement. Under
flexible collaboration, the JOCV program was positioned external to the project but was requested as
needed by the project to collaborate through conducting activities such as monitoring. Either of the two
collaboration modalities was chosen according to the purpose of each project. However, JOCV posts in
science and mathematics education were generally difficult to fill due to a shortage of eligible candidates
who met the requirements. To counter this constraint, senior JOCVs and short-tem emergency JOCVs
were assigned under SBTP in the Philippines to secure a required number of experienced volunteers for
achieving the Project’s outputs. In collaborating with the JOCV program, it is important to confirm the
intention of volunteers and to fulfill two objectives at the same time: civil participation aimed by the
JOCV program and the achievement of outputs sought by projects. It is especially important to confirm
the intention of JOCVs from the recruitment stage and not to impede activities desired by JOCVs
Collaboration with other donors can be effective if the objective of the collaboration is clearly
defined. In addition, donors with past experiences in similar projects can be an important source
for information exchange.
In this Evaluation Study, two Projects, STM in Ghana and PROMETAM in Honduras were analyzed
for lessons learned on collaboration. In particular, the process for establishing collaboration and its
impacts were identified. These two Projects approached the issue of collaboration in a similar manner.
When these Projects encountered any issues to be tackled in forming collaboration with other institutions,
they raised and discussed such issues among other donor agencies. The Evaluation Study, through a
questionnaire survey, found that donor collaboration did not take place under other Projects due to a lack
of recognition for the need or problems to be solved in establishing collaboration. Under SMASSE in
Kenya and MSSI in South Africa, information exchange with other donors took place during the project
planning stage and this helped to avoid the duplication of cooperation as well as in receiving useful
advice. It was observed that information exchange with other donors is especially essential at the planning
stage of new projects.
(4) LESSONS LEARNED ON INSTITUTIONALIZATION
To gain governmental assistance and to achieve institutionalization, it is important to establish a
training system that can be easily managed by a recipient country and to establish an institutional
mechanism suitable for the administrative system of the country.
Institutionalization of a teacher training system contributes not only to the achievement of impacts
and sustainability but also to promote the participation of teachers in the training. Necessary factors for
successful institutionalization include strong requests to those concerned in administration as well as the
introduction of a training system that can be easily accepted within the country of the project. A training
system with a manageable size of running cost and that utilizes an operation fund collected from
beneficiaries does not give too much operational burden to institutes of the partner country, thus paves the
way to institutionalization. Another factor leading to institutionalization concerns the structure of an
administrative system. When decision–making power is simultaneously assigned to several government
agencies and/or when the responsibility of each agency is strictly defined, it may be more effective to
establish the project at the local level first, rather than trying to obtain agreement on institutionalization
from the very beginning.
Governmental assistance can be obtained more smoothly for a training system built upon an
existing system, rather than for a newly established system.
Under SBTP in the Philippines, a school training system that was conducted by the Philippine side
was strengthened and expanded by Japanese inputs and was established as a cluster system. At present,
SBTP is the only training system authorized to be held on weekdays and supported by the government.
This could be attributed to the fact that the school training system was already prevalent and recognized
by authorities to a certain degree. As this example shows, cooperation and/or assistance to existing
training system facilitates the establishment of a system in a short term and, as a result, sustainability
In countries where effective collaboration among donors is in progress, coordination among donors
If there were no other donor implementing projects in the same sector in the country of the project,
the possibility of institutionalization becomes higher in comparison. In Kenya, donor coordination in the
education sector was not very developed and there were no other donors implementing projects in the
area of secondary science and mathematics education. Consequently, the importance of SMASSE in
Kenya became considerably high in relation to other donors and this led to the institutionalization of the
Project. In countries where donor coordination is well in advance, enhancing Japanese presence in the
framework of donor coordination can promote institutionalization. In the case of Honduras, Japan had
been recognized among donors as being competitive in mathematics education, and this contributed to the
official adoption of materials developed by the Project. Therefore, it can be said that the enhancement of
donor coordination is a contributing factor for institutionalization in countries where donor coordination
in the education sector is already in progress.
(5) LESSONS LEARNED ON MONITORING AND EVALUATION
Establishing an independent monitoring and evaluation group in a project management system can
clearly define the responsibility of evaluation tasks.
SMASSE in Kenya established a taskforce conducting monitoring and evaluation activities and it
assigned staff members mainly responsible for monitoring and evaluation tasks. Establishing an
evaluation group is effective in clarifying where the responsibility for evaluation lies and, hence, for
regular and continuous evaluation activities. The support extended by Japanese experts will further
strengthen the evaluation and management system. When establishing an evaluation and monitoring
taskforce, it is necessary to continuously improve the knowledge on evaluation and management capacity
of the staff. One possible measure is to dispatch a short-term expert specialized in evaluation, conduct
an evaluation seminar several times a year, and construct a framework of evaluation in cooperation with
In the evaluation of primary and secondary science and mathematics education projects, an
attempt for adopting a method to objectively evaluate the teaching capacity of teachers and the
improvement of classes was launched. It is desirable to accumulate evaluation results on the
capacity of students and to establish an evaluation method based on such results in the future.
The evaluation method adopted for primary and secondary science and mathematics education
projects mainly focused on interviews and questionnaire surveys targeting teachers who have participated
in training. Recent impact evaluations include comparisons of effects seen on participants before and after
training as well as effects seen on training participants and non-participants. In addition to this,
quantitative analyses applying academic theories were also conducted. The evaluation of students,
however, has not been conducted in some Projects of this Evaluation Study. The improvement of student
capacity depends on local community and individual characteristics to a great extent, thus, it is difficult to
establish appropriate indicators to measure the effects. In the mid-term evaluation of STM in Ghana,
interviews were conducted not only with students but with parents as well as an attempt to perform a
multi-aspect evaluation. Although the evaluation results of STM may not be sufficient to find direct
cause-and-effect linkages within the Project, the accumulation of data will contribute to the development
and improvement of a method to evaluate the capacity of students.
Monitoring results bring about important information useful to the improvement of projects.
Mid-term evaluation, if conducted appropriately, contributes considerably to the improvement of
However carefully preliminary study may be conducted in planning stage, it is impossible to grasp all the
necessary information before the start of the project. Therefore, almost all the projects need modification
after they start. The quality of modification depends on regular monitoring activities. Under IMSTEP in
Indonesia, a pilot activity was introduced in the middle of the project period and this successfully
enhanced the Project’s effectiveness at the school level. This initiative was evaluated as a tangible output
of monitoring activities. A mid-term evaluation is a good opportunity to find potential inhibiting factors
that may be overlooked under daily monitoring activities. Under STM in Ghana, the mid-term evaluation
revealed that the high overturn rate of teachers could be an inhibiting factor. Based on this evaluation
result, the project plan was modified and support for school training was strengthened to generate impacts
on entire schools as well as on individual teachers. Although mid-term evaluations tend to be conducted
in a more simplified manner than preliminary and terminal evaluations, it is desirable to enhance the role
of mid-term evaluation as it greatly contributes to the efficiency and impact of the projects.