USING FEEDBACK FROM PUBLIC EXAMINATIONS AND TEACHER ASSESSMENT TO IMPROVE CLASSROOM TEACHING By Gilbert M. Mwanzia Patrick Miano Kenya National Examinations Council Kenya National Examinations Council Nairobi, Kenya Nairobi, Kenya Abstract Summative public examinations are reputed to have very little contribution to the improvement of teaching and learning process. Teacher’s assessment on the other hand is an ongoing process in teaching and learning situations and is used mainly for diagnostic purposes. Teacher’s assessment provides feedback on challenges that should be addressed to ensure effective provision and acquisition of knowledge and skills through the school system. The Kenya National Examinations Council offers public examinations that measure candidate’s attainment. In its vision to assist schools in offering quality education, it provides feedback to schools in form of backwash documents both at end of primary and end of secondary examinations. The paper discusses the role played by these backwash documents, the structure and effectiveness. The paper also discusses ways in which school examinations both formal and informal, and external examinations play a complimentary role to improve the classroom teaching. 1.0 INTRODUCTION The quality of Education system is a very crucial determinant of the economic development and social stability of the nation. According to Udo Bude (1995) “The renewed interest in schools is based, among other reasons, on the results of a number of studies that seem to indicate that, under certain conditions of modernization and social improvement, schools may play a useful role if they provide education of sufficient quality”. He also points out that “School is an accepted vehicle for societal transformation as long as it succeeds in providing the three Rs (reading, writing and arithmetic) and the basic skills and knowledge for participating actively in society”. In essence Udo Bude perceives quality education as that which provides competences that are relevant to the needs of society. Bunyi (2001) expresses similar views. Findings in her research revealed that in the perception of the key stakeholders (pupils, parents, teachers, students teachers, educational planners and managers, educational researchers and members of the civil society), quality education has two components: 1. What is learned, i.e. the knowledge, skills, values, and attitudes acquired through schooling and 2. How well what is learned is learned i.e. the levels of competency attained. Consequently, for assessment to play a role in fostering quality education, it must pay attention to the goals of education in terms of what is taught and learned and the levels at which the knowledge and skills acquired by the learner are assessed. Assessment therefore serves as a monitoring and evaluation tool in curriculum implementation. If properly used, assessment, whether external or institutional based can serve as a tool for enhancing the quality of teaching and learning. What then is assessment? 2.0 ASSESSMENT According to Taylor (1984) Assessment refers to the gathering of relevant information to help individual (teacher) make decisions regarding appropriate goals and objectives, teaching strategies and programme placement. Assessment in education is essentially a measurement process. Attainment testing attempts to measure what the student knows or can do and is usually related to the syllabus of a course, which the student has followed. Predictive testing, on the other hand attempts to assess what the student might learn or learn to do successfully given suitable tuition. Assessment of attainment may include the use of a number of different assessment methods. Methods available include written examinations, practical and oral examinations, projects and assignments during the course. Each of these approaches to assessment provides information, which the education providers and the learners themselves can use to decide the interventions that can be employed to improve educational attainment. 2.1. PURPOSES OF ASSESSMENT There are many reasons of assessing attainment. The purpose of assessment helps in deciding the type of assessment to be used. However, the purposes of the assessment can be listed according to their relevance for the student, the teacher or institution, the employer and the public. Christine Ward (1980) has given the following as some of the purposes served by assessment. 2.1.1. For the Student An incentive to study the course. A means of indicating areas on which he needs further study. A recognition of getting his knowledge or skills and of the effort he has put into his course. A means of entry to a higher course or a later stage of the same course. 2.1.2 For the Teacher/Institution Feed back Part of teaching process, indicating areas on which further tuition is needed. To teach more efficiently the teacher needs feedback in finding out if what has been taught has been learned, whether a particular approach to a particular topic was successful or whether to use a different method. The teacher needs to know whether or not the pupils have understood the topic properly so that misconceptions can be corrected. Monitoring Progress Monitoring progress of individual pupils or classes is important. Progress of parallel sets of classes under different teachers may need to be compared as well as checks on the progress of individuals or pupils in a class for the teacher to know who may need remedial teaching. When pupils have to be allocated to different streams, their levels of attainment in particular subjects are factors to be considered. Attainment records are also useful to parents who may want to know the progress being made from year to year and to the new teachers who may want to be acquainted with their pupils (School council publications 1975). Evaluation of Materials When a teacher may have to compare new teaching materials (text books and apparatus) with the old. He needs to have basis on which to make judgment on whether the school should or should not acquire new teaching materials. 2.1.3 For the employer and the Public Records of school attainment are important to those outside the school. There is need for keeping parents informed about school progress. Employers want a guide in form of certificates or leaving certificates from pupils looking for jobs. Entry to higher levels of education may be dependent on satisfactory completion of an earlier stage. It is responsibility of teacher to make sure that this information is available to assist pupils in their future careers. To the general public it is a means of safeguarding public safety and health by restricting employment in some areas (e.g. aircraft maintenance, electrical installation). Assessment also ensures adequate standards of work on part of those who serve public (e.g. car servicing) and evaluation of courses financed by the taxpayer. Considering assessment as a necessary part of the teaching/learning process, it therefore requires assessment of parts of syllabus be undertaken during the course while there is still time for remedial teaching. 3.0 Constrains in the Teacher Assessment of learner achievement The system of assessment at all levels in primary and secondary schools consists of basically classroom tests constructed by class teachers. The tests may be given any time - weekly, monthly, in the middle of the term or at the end of the term. The purpose is to prepare reports on student progress upon which decisions can be made on whether to pass a pupil to next class or not. The other purpose is to inform parents of their children’s performance. Many primary Schools also administer continuous assessment tests (CAT). Schools with low enrolment, coupled with a wealth of teaching and learning resources, often apply CAT fairly effectively as teacher only have a small number of learners to give attention. At the same time they have requisite resources with which to carry out necessary interventions. Such schools are usually privately owned and are managed professionally. In contrast the high enrolment and scarcity of facilities in many public schools make it difficult for teachers to effectively carry out continuous assessment. In Kenya, provision of free primary education has made the situation worse. Free primary education has resulted in high enrolment, leading to a tremendous increase in class sizes in most schools. The average class size is currently estimated at 60:1, which has made it difficult for teachers to observe learners on a one to one basis. As a consequence of these developments teachers’ reporting of learner progress is less than wholesome as it is dominantly based on performance in classroom tests. Needless to say, objective reporting needed by providers of education to guide interventions in their efforts has been a casualty of the unprecedented growth in enrollment. Even where teachers use written tests, many of the tests they develop are wanting in quality. The tests are usually wanting in originality of style, clarity of language and abilities tested. Some teachers do not even bother to develop their own tests. Instead they simply lift questions from past national examinations or from commercial publications. Rather than help improve the learning process, these practices seem to encourage rote learning, which both the teacher and the pupils believe would improve performance in national examinations. Thus, a combination of large class size, teacher’s inability to develop suitable assessment instruments and endemic shortage of suitable and sufficient educational resources explains why teachers are unable to effectively assess their pupils and collect information that would help them improve teaching. Apart from lack of educational resources, many teachers’ inability to develop suitable assessment instrument is a result of the shortcomings of the teacher training programmes which give little emphasis on acquisition of assessment skills. There are many teachers in our schools who cannot apply the principles of assessment for diagnostic purposes because they were not given sufficient training. A growing phenomenon in the Kenya education sector today is the stiff competition for good performance at all levels of education. The competition is not only between individual learners and schools but also between various geographical zones. The competition has led to a mushrooming of zonal examining panels formed by schools in a given zone or districts. By pooling resources together schools in a zone finance development of examination instruments which individual schools administer to their learners. The tests target several levels, but especially those that are near the top of the primary and secondary education cycles. Since most of the panel members are drawn from teachers who mark national examinations, the tests they develop bear all the characteristics of standardised tests. When the feedback from these tests is used to inform teachers’ approaches to teaching the results are always positive. Also, arising from the appointment of the most competent teachers to the panels, other teachers in the zone learn from their experience. 4.0 Role of KNEC in enhancing quality Education. The Kenya National Examinations Council was established in 1980 by an Act of Parliament to conduct School and Post-school examination and award certificates to successful candidates. Examinations and Certification are major indicators of quality of Education. One of the strategies employed by KNEC to improve the quality of education in Kenya is through effective assessment. This is in line with the Mission Statement of KNEC, which is, 'to objectively test and evaluate the curriculum and to enhance and safeguard globally acceptable certification standards". This strategy is accomplished within the broad framework provided by the Council's Vision which is, "Efficient Testing for Quality Education". Efficient testing involves maintaining relevance, fairness, validity, reliability and equity in all examinations. KNEC plays a strategic role in national development since examinations and certification are central to education and training as a means of evaluating levels of achievement for further education, training and employment. 5.0 Shortcomings of Feedback from external (public) Examinations. Public examinations refer to any assessment that is set, moderated, administered and marked by any agent other than the teacher. These examinations are usually administered at the end of the course and are meant for selection purposes. They do not provide the teacher with immediate feedback on the progress of the learner during the study. However, as discussed later in this paper their results may be used to provide feedback for future use. The assessment of the implementation of the curriculum in Kenya is too dominated by terminal examinations principally, the Kenya Certificate of Primary Education (KCPE) and the Kenya Certificate of Secondary Education (KCSE). These examinations are administered nationally to test a wide range of subjects as stipulated in the curriculum. However, both give insufficient attention to formative monitoring. As a result, schools ignore aspects of learning that are not tested but yet stipulated in the curriculum. Both the KCPE and KCSE examinations are mainly used for selection and certification and this makes the system biased towards further education and training. The country does not have an internal system of monitoring learning achievements of competencies at other levels of the education cycle. The absence of a system for the national monitoring of achievement of learning outcomes, therefore, denies the country data and information that could be used for improvement of education. 6.0 Using School Based Assessment (SBA) to Improve Instruction and Learning The 1988 Presidential Working Party on Education and Manpower Training for the next decade and beyond was emphatic on the need to incorporate School Based Assessment in national certification. The new curriculum at the time (first examined at end of the primary cycle in 1985 and 1989 at the secondary cycle) had made provision for the existing assessment procedures to incorporate SBA especially for vocational and practical subjects offered in the secondary cycle. To date this has been fairly successfully by the KNEC despite the challenges associated with it. As a tool for generating information to improve the learning/teaching processes, school based assessment has several advantages, but also challenges. SBA offers the classroom teacher an opportunity to obtain constant flow of information about the student’s progress or lack of it, and perhaps the reasons why. It offers the teacher the opportunity for remedial teaching and serves as a means to systematic implementation and evaluation of curriculum. When this important aspect of assessment is lacking the quality of teaching and learning will be affected and hence the quality of education is seriously compromised. There are several assumptions that have to be made when considering SBA for certification. Some of these include ensuring that: The teachers are qualified in assessment methods; The teachers are honest; Teachers have enough time to construct and administer assessment instruments and keep proper students scores/records; The instruments constructed and scores awarded are valid and reliable; The class sizes are manageable. It is important to note that SBA can enhance the quality of learning if used appropriately. SBA individualizes the curriculum and focuses on assessing the application of knowledge in a real world situation. The role of assessment should not be seen as being primarily that of a judge to determine the future of the learner but that of being an integral part of the teaching and learning process. The purpose of SBA should be seen as that of improving quality of teaching and it should therefore play a complimentary role to external assessment as it can be used to assess areas that external tests cannot assess. Such areas include attitude, interest and other human attributes that are difficult to assess using public examinations. Knowledge about students achievement obtained through SBA in any education system could be used to make many important decisions about changes in the curricula or allocation of resources that could have far reaching positive implications on quality of education though we may not have a great deal of evidence that these changes take place. When considering what role assessment might play in improving the quality of education, it is important to distinguish between SBA and external (public) examinations. In both cases assessment is concerned with the performance of individual students. However, the differences arise in the way individual student’s data are aggregated and in the use that is made of the data. For the purpose of obtaining data on learner achievement for use in national certification at the end of the secondary cycle, SBA is applied to evaluate project work carried out by learners. During the conduct of the project, teachers are required to guide learners to identify and develop suitable methods of carrying it out. The project scores in most cases vary from subject to subject and are used in the final grading of candidates at end of the secondary cycle. The subjects that have a project component are Home Science, Art and Design, Agriculture, Woodwork, Metalwork, Building and Construction, Power Mechanics, Electricity, Drawing and Design, Aviation Technology and Computer Studies. Teachers assess their candidates at various stages of the projects. External assessors, who are teachers from other schools, assess the projects. Teachers and the assessors do a thorough coordination of the scores in order to standardize the marking. 7.0 Challenges faced by KNEC when dealing with SBA a. The cost of assessing projects is exorbitant, yet the contribution of the project score to the total score for the paper is relatively insignificant. b. Some of the teachers and assessors inflate the scores. In a few cases the scores submitted have at times been proved to have been faked. c. Lack of uniform facilities in schools undermines the reliability and validity of assessment results obtained from SBA since candidates may not be in a position to produce work of comparable standards. d. Inaccurate data transcription and recording of SBA information by all parties involved in SBA process (eg. wrong coding of subjects, interchanging marks, omission of candidates’ numbers, incomplete scores). e. Lack of adequate qualified personnel for specialized subjects like Computer Studies to assess projects. f. Interference by School Administration on SBA processes. 8.0 FEEDBACK FROM PUBLIC EXAMINATIONS 8.1 Feedback from the Marking exercise The Kenya Certificate of Primary Education (KCPE) examination is mainly scored (marked) by machines except for English composition and insha in Kiswahili, which are manually scored. Each year examiners, who are mainly primary school teachers, converge at designated marking centers to take part in this exercise. They spend at least a day discussing the merits and demerits of some carefully selected essays before marking starts. Much attention is given to the subtle points of difference, which make one essay worth a few marks more or a few marks less, than the essay adjacent to it on either side in the standard scale. Because the frame of reference they are using is a set of real essays by the candidates, rather than a set of generalised points in a formal marking scheme, examiners very quickly begin to relate the discussions to their own work as teachers of English/Kiswahili, and hence to formulate ideas as to how they could improve standards of prose writing in their own schools. Somerset, (1981), notes that, since the examiners are drawn from all over the country this backwash effect may play a significant role in improving teaching of the subject. Similarly the marking of secondary examinations may also provide great opportunity for the examiners; majority of who are Secondary school teachers, to re-evaluate their teaching methods in their subject areas. The marking exercise takes longer time than that of the KCPE and the coordination involves a more thorough discussion of specific content areas. The effect of the knowledge and skills acquired in this exercise on teacher's work in the classroom is difficult to quantify but it certainly has an added value to the teaching and learning. The participation becomes more than a marking exercise; it is an intensive period of in-service as well. Subject specialists in KNEC are of opinion that the quality of answers has improved over the years but whether this is as a result of the experience gained by the teachers during marking needs to be researched on. 8.2. Feedback from Ranking of Schools For many years ranking of schools and candidates has been a controversial issue not only in Kenya but also in many other countries, including those in the developed world. The purpose of ranking schools is to generate debate that would, hopefully, lead to improvement in education provision. Ranking of Schools is expected to: a. Motivate schools that are highly ranked so that they can maintain their good work. b. Challenge the less well performing schools into improved performance c. Encourage healthy or positive competition between schools and learners. d. Prompt school inspectors and supervisors into endeavoring to make more frequent visits to schools under their care. e. Sensitize school communities, the parents and teachers associations to take more interests in their schools and ensure provision of needed resources. In an endeavor to be ranked higher the teacher will strife to improve his teaching approaches. Whereas ranking of candidates may have similar motivational impact as the ranking of schools, it is of greater personal value to those who are highly ranked. It bestows esteem and opens doors of opportunity in the search for further education or placement. 8.3 Feedback Reports. To maximize the positive wash back effect of its examinations, the council has adopted an effective reporting approach of performance of candidates. At the time of release of results a comprehensive report is issued on performance trends in all the subjects. This is soon followed by preparations of comprehensive Newsletters analyzing performance in every subject with suggestions of how the weaknesses displayed can be rectified. The targeted consumers of these reports are teachers, school managers, curriculum developers, subject inspectors, parents and other interested stakeholders. The main objectives of the feedback reports are to: a. Give a detailed analysis of candidate’s performance in each of the KCPE and KCSE examination papers. b. Provide the classroom teachers with information about pupils' weaknesses in the course content. c. Provide suggestions on better teaching and learning methods that can enhance performance. d. Give teachers advice on how they can re-orientate their teaching strategies to enhance pupils learning and performance. 8.3.1. KCPE Newsletter The newsletter contains reports on performance of candidates in each subject and is published after the release of end of primary cycle results. It highlights the items that were found difficult by the candidates and also advances possible reasons for poor performance. Teachers and learners are advised on the interventions that are needed to improve performance in the identified areas. Although the intention of this publication is to provide information which can be used to improve the teaching/learning practices, available evidence seems to indicate that the publication does not reach more than a fraction of the targeted audience. Part of the reason for this is that, unlike in the past, when the publication used to be distributed free to schools, they are now required to meet part of the production cost. 184.108.40.206. Main features of the Newsletter: Information about candidature. Candidates raw mean performance by gender. Performance per item in terms of percentage of candidates scoring correctly in each question. Discussion of items with facility index of 30% or less. In languages selected samples of compositions that scored the whole range of marks, from the lowest 03/40 say to 38/40 are included together with comments on errors made and possible reasons for the score. 220.127.116.11. Analysis of Candidates Performance In determining the performance of candidates item analysis is used. Item Analysis involves determining the facility index (f-index) and discrimination index (d-index) of each question in a paper for the chosen sample population of candidates. A good question is considered to be one that has a facility index of between 30% and 80%. Any question therefore with a facility index of below 30% is considered to have proved difficult to the candidates and one above 80% is considered to have been easy. It is in this way that the item analysis programme picks out questions with low facility indices for discussion in the KCPE Newsletter. In the discussions of the questions that candidates performed poorly, a response is given for every question showing the percentage of candidates from the sample population choosing the various responses to the question. The following is an example of a question in Science as reported in a KCPE Newsletter; Which one of the following sets of characteristics are descriptions of a wind- pollinated flower? A Fewer large pollen grains, petals sometimes absent. B Fewer large pollen grains, large loosely attached anthers. C Small in size, large loosely attached anthers, sticky stigma. D Small in size, large loosely attached anthers, petals sometimes absent. Response Pattern Option A B C D* % Choosing Option 9.10 24.77 36.69 28.62 Mean Mark in Other Questions 22.94 23.59 23.12 28.25 Important to note was the assortment of characteristic of insect pollinated flowers. Candidates had to sort out those characteristics that belong to wind pollinated flowers only. Sticky stigma is characteristics of insect pollinated flowers hence option C, which was chosen by majority of candidates could not be correct. Those who chose B did not also know that fewer pollen grains is a characteristic of insect pollinated flowers. Lack of knowledge about characteristics of the two types of flowers led to guess work as is evident with the choices. An asterisk (*) on an option denotes the correct response to the question. Also under the response pattern, information on the mean mark of candidates in other questions is given. This is the average score on the rest of the items for each group of candidates choosing an option and it is important as it shows the way each group of candidates choosing a specific option scored in the other questions in the paper. As a means of getting the feedback on the relevance and effectiveness of the feedback reports teachers are requested to comment and/or make suggestions that can be considered for inclusion in future reports. 8.3.2 KCSE Feedback Reports The main features of the KCSE reports are: Overall candidates’ performance. Questions candidates found difficult. Discussion of the weaknesses in the candidates’ responses. Expected responses. Advice to teachers. The report also highlights the areas of the syllabus not adequately covered as evidenced by the poor performance by candidates in these areas and gives recommendations on how such areas could be managed or taught better. 18.104.22.168. Identification of Questions for Discussion At KCSE level the papers are manually marked, unlike in the KCPE where scoring is mainly done by use of machine. Consequently data on candidate’s performance is not as easily available at KCSE as in the KCPE. Thus the Council relies mainly on the chief examiners reports and sampling the candidate’s scripts. 22.214.171.124. Chief Examiners Reports During marking the examiners provide crucial clues on questions in which candidates perform poorly. Examiners are exposed to a large sample of the responses to the questions during the marking exercise and therefore are able to provide the council with information on weaknesses and misconceptions resulting from poor teaching approaches and inadequate syllabus coverage. Examiners are also experienced teachers and are able to detect candidates’ weaknesses and content areas requiring attention. 126.96.36.199. Sampling Candidates Scripts The Council samples scripts for analysis. Questions or parts of questions that are scored below average (50%) are sorted out for scrutiny. Subject specialists and research officers scrutinize the responses of the items individually to determine the nature of the weakness in the responses. 8.4 Feedback of Post-schools Examinations The department of Research and Data processing usually compiles reports on performance statistics at the release of the post school examinations results. These reports rely heavily on the Chief Examiners recommendations to highlight areas of weakness as observed during the marking exercise. The council is faced with the following constrains in this area: The number of examination papers is too large compared to the number of the subject officers. Officers handle up to 100 or more papers in a year, which makes it difficult to get time to research and write the reports. Subject officers supervise the development of papers that are not in their area of specialization and therefore have little or no expert advice to give. Some courses are examined two or three times in a year, which gives the overloaded officers no room to do the reports. Kenya National Examinations Council then hands over Chief Examiners and recommendations to the Directorate of Quality Assurance and Standards (QAS) in the Ministry of Education Science and Technology. It is hoped that based on these recommendations appropriate advice is given to the teachers/lecturers to improve on their teaching approaches. 8.5 Constraints Faced by KNEC In Preparation of Feedback Reports The size and method of presentation of the reports has been bulky and costly. It is important to consider the production of concise, but informative reports that can quickly reach the users when there is a fresh recollection of the examinations. KNEC has managed to produce the reports on yearly basis to reach the schools in at the latest by June in the year after the examination. Establishing a KNEC web-site could also considerably reduce the cost of disseminating such reports. In the meantime the Ministry of Education through the Directorate of Quality Assurance Standards has come up with a Policy whereby all schools will be supplied with these reports free of charge from year 2005. This will go a long way in enabling the feedback documents to reach the intended classroom teacher. 8.0. Conclusion This paper has highlighted the various ways in which feedback from the teachers’ assessment and the external examinations are being used to improve classroom teaching. In particular the paper has outlined the insufficient attention given to formative monitoring of learning achievement competencies As long as the teacher training programs do not prepare teachers to face classroom assessment challenges that will enable them to assess and interpret learners’ competencies, improvement in teaching and learning cannot be achieved. There is need for establishment of a National Assessment System that will monitor achievements of competencies at all levels of education cycle. The absence of such a system denies the country the data and information that could be used in improving the quality of education. REFERENCES 1. Udo Bude (1995), Strategies for using Information to Improve Learning Conditions and Instructional Practices at the School Level DSE 2 D. S. Frith, H G Macintosh (1986), A Teacher’s Guide to Assessment. 3. Evans/Methuen (1975), Educational Schools Council Examinations Bulletin 3 – Assessment and Testing in the Secondary School. 4. David Satrerly , Assessment in Schools. 5. G. W. Bunyi (2001), School Based Assessment: An Unattainable. Good in Kenya’s Education System? 6. Ronald L. Taylor, Assessment of Exceptional Student. 7. Christine Ward (1980), Designing a Scheme of Assessment. 8. (MOEST September 2003), Education Sector Strategic Plan 2003-2007, 9. David M. Weerhe (2001), Utilizing School Based Assessment for Improving Instruction and Learning: An Overview of constraints and Challenges in Operationalizing SBA. 10. P. Wasanga & G. G. Ingolo (2001), SBA – KNEC Experience at KCSE Level. 19th – AEAA Annual Conference, Nairobi . 11. Prof. J. C. Kiptoon (2001). Permanent Secretary, Ministry of Education, Science and Technology. Closing speech at the 19th AEAA, Annual Conference, Nairobi. 12. H.C.A. Somerset (1981). Examination Reform in KNEC 13. Kenya Education Strategic Sector Plan ( 2003-2007) 14. Obadiah M. Wamai (1991), Scratching the surface (MEd. Thesis).