"reader - DOC"
The Reader Quiz Module for Extensive Reading Thomas Robb Kyoto Sangyo University Many practitioners of Extensive Reading (herein referred to as “ER”) are convinced to the effectiveness of the ER approach, yet despite these accolades, 1) physical obstacles, such as the need to establish and maintain a library of readers, and 2) student management issues, such as the need of an efficient mechanism for holding students responsible for actually doing the reading, have hampered the widespread adoption of ER in the world of EFL. In fact, both of these problems can only be effectively addressed at a curriculum-wide or school-wide level. Yet it is often the individual teacher, who practices ER with his/her students while other classes at the same level in the same school do not. The administrative overhead, mentioned above, discourages all but the hardiest enthusiasts from implementing ER with their classes. (The educational system in Hong Kong has been one “success story” where the approach has been implemented on a curriculum-wide level.) The objective of the Moodle Reader Quiz module is to remove obstacle #2, by offering teachers or entire school systems a practical way to test whether students have actually done the reading that they claim to have done. This is achieved by serving each student with a freshly generated quiz with questions drawn from a question bank for each book, questions which should be easy for the students to answer if they have understood the basic plot of their book. In fact, at Kyoto Sangyo University, where the system as already been implemented, students take the quizzes with books open. A time limit of 15 minutes is too short to allow students to simply scan to find answers if they haven’t read the book beforehand. Inspiration for the Reader Module The English Department, Faculty of Foreign Languages at Kyoto Sangyo University (“KSU”) has had an ER program since the mid-1980s, not centered around “graded readers” but rather “American Youth Literature” such as the works of Judy Blume, Beverley Cleary, the American Library Associations Newbury Award books, etc. A product of a U.S. based company, Renaissance Learning, (http://www.renlearn.com) offers a product called the “Accelerated Reader Program” (henceforth, “AR”) which has been widely adopted in elementary and middle schools in the U.S. and Canada. Schools may purchase 10-item multiple choice quizzes to complement their library holdings, with quizzes for over 30,000 titles currently available. The AR system is based on points that the students receive for each book read. Each book has an assigned point value based on the number of running words and the calculated “readability level” of the book. Students, however, do not receive the full complement of points unless they score 100% on the quiz. Otherwise, assuming that they receive a score of 60% or above, their points are calculated by multiplying their percent correct by the total point value of the book. Thus a student who scored 80% on a 3-point book would receive 2.4 points towards his/her target score. Figure 1 – An Accelerated Reader Quiz Item Figure 2 - An Accelerated Reader Quiz Report The Accelerated Reader program served us well in the beginning, but the more we used it, the more we realized its defects when used in an ELT setting. Since graded readers are not included in the quizzes available from the publisher, they had to be created locally. The AR program, however, only permits 100 "teacher-made quizzes" and we had reached this limit at Kyoto Sangyo. The quizzes assume native language proficiency and often include words in the question items that the students do not know, even though they might know the answer to the question from their reading of the book. There are a fixed number of questions for each book that do not vary from student to student when taking the test. It is therefore possible for students to cheat and tell the answers to other students. Due to the strong potential for cheating, we decided that the quizzes had to be taken under controlled conditions, (in our case, in the latter half of the lunch break Monday through Friday). This placed an additional burden on the teaching staff. The report program is cumbersome to use and we had to rely on the university Computer Center staff for back-up and support. There was no efficient mechanism for teachers to share "teacher-made quizzes" between schools. With the advent of Moodle on campus, another possibility presented itself – the use of the Moodle quiz module. Direct use of the quiz module, however, would have meant placing all quizzes in a single course, or perhaps in multiple courses depending on the level of the student. Such an approach, however, would not permit the fine level of control that we deemed necessary to make and effective system. Problems, such as managing the promotion of students from one level to the next, made it infeasible to use the quiz module “out of the box.” Thus the advent of the “Reader Module.” The Reader Module The Reader Module provides an interface to the students that permits them to take only quizzes at whatever their current level has been determined to be. Rather than being confronted with a long display of quizes, it allows them to select the quiz that they wish to take from a pulldown menu bu publisher, series and level. The essential differences between the AR system, as implemented at KSU and the Reader Module is summarized in Figure 3. Moodle Reader Module Accelerated Reader 1) Students can only select from books at their 1) Students can select any book that has current Reading Level, or one below it. an AR quiz. 2) Students are promoted to the next level after 2) No promotion system. gaining 6 points at their current level. 3) Books are basically all worth 1 point, but some are 3) Books have a point value that is based worth more or less purely based on their relative on both length and difficulty. length compared to graded readers at that level. 4) Students may take the tests anywhere, even at 4) Student can take the quizzes only under home. supervised conditions. 5) Quizzes are randomly generated from a large 5) AR quizzes consist of 10 unvarying question bank. There are several question types. multiple-choice questions. 6) Quizzes are timed – Maximum 15 minutes. 6) Students can take up to 30 minutes to complete a quiz. 7) In order prevent a last-minute rush, students may 7) Students can take multiple quizzes on take a quiz only every third the same day. Figure 3 – Comparison of the Reader Module and Accelerated Reader Appearance of the Reader Module The reader module appears to the student as shown in Figure 4. It consists of the following elements. 1. A "stamp collection" of book covers for which the quiz was successfully passed. 2. A listing of all books for which quizzes have been taken, which shows which were passed or failed, how many points were awarded (usually 1.0 per book), and the cumulative point total. 3. A report of the student's current level 4. A message indicating how many more books at this level need to be passed before promotion to the next level and how many quizzes one level down that the student can still take. Finally, there is a highlighted sentence that says one of three things: (A) You can take a quiz now. (B) You can take a quiz tomorrow. (C) You must wait x days before you can take another quiz. 5. A message area. Messages can be sent to all students or to any specific group as defined by the Moodle "group" feature. There is no provision for individual messages because Moodle's built-in function is more efficient for contacting individual students since they will see the message when they log in and/or it will be sent to them as e-mail. 6. Quiz book selection pulldown menus with buttons to either take the quiz or return to the main page. Figure 4 – The student screen in the Reader Module Determination of reading levels There is no agreement among the various publishers on how to name (or number) book levels, so a book that is at "Level 1" for one publisher can be considerably easier or more difficult than a book labeled at the same level by another publisher. For this reason, all books in the Reader program need to be assigned to a separate reading level designation ("RL") so that students can be assigned to read books that are at a suitable level of difficulty for them. This is required because more able students would tend to read books which are far to easy for them if that would satisfy the reading requirement. In the previous Accelerated Reader system, we had the opposite problem, since the more difficult books had a higher point value, students would attempt to read books which were actually too difficult for them. This meant that they had to spend much more time using their dictionaries and that they often could not comprehend the book well enough to pass the comprehension quizzes. Figure 5 shows how we equated the series based on our own experience, the publishers claims on how the texts correspond to various proficiency tests such as the TOEFL and IELTS, as well as the levels in the Common European Framework. The "Other" category is for books that are not "graded readers" but rather standard "youth literature" available for native speaker children. Figure 5 – Reading Level Correspondences (May 2008) Question Types The questions themselves, with one exception in the standard out-of-the box Moodle format. Any given book will normally consist of 10 items distributed among at least three types. 1. True/False 2. Multiple Choice 3. Matching questions 4. Who said this? and/or Who was this said to? 5. Ordering The “ordering” question type is new. Questions of this type are created by listing the major events in the book in chronological order. Each quiz then displays a randomized selelection of 50-60% of the total number of events, which the student has to drag into the correct order with the mouse. Figure 6 – The Ordering Question Type For graded readers that are collections of short stories, these two additional types are employed. 1. In which story does someone say this? 2. In which story does this occur? The administrative area Teachers see a set of tabs at the top of the main reader screen which affords them access to the administrative area which includes the items shown in Figure 7. These are self-explanatory, so I will merely mention that each selection, where appropriate allows the teacher to view any group, to resort by clicking on column headers and to download the data in as an Excel .xls file. Figure 7 – Administrative Menu Results from the first two months There are two reading deadlines in the Spring term at KSU, one at the end of May, and the other at the end of the term, in mid-July. The graphs below record the number of quizzes taken in 2007, with the Accelerated Reader software, and in 2008 with the Moodle Reader Module. Since we had a considerably larger intake of students in 2008 due to the establishment of an International Relations major within the Faculty of Foreign Languages, the numbers have been adjusted so that the figures are comparable. Figure 8 – Student quiz accesses, April-May 2007 using the AR Program Figure 9 – Student quiz accesses, April-May 2008 using the Reader Module Figures 8 and 9 show the number of books for which quizzes were taken on each day in April and May of 2007 and 2008 respectively. A quick glance reveals that students read their books and took quizzes at a much more even pace, and that there were by far fewer failures in 2008. Of course, the fact that students, in 2008, were permitted to take their quizzes anywhere, including at home, helped to make the distribution more even, but this fact does not account for the lack of the “last-minute rush” observed in 2007. Since the quizzes themselves were different, no clear claim can be made that the system itself has improved their comprehension, although the fact that their were few failures, even for those who took quizzes just before the deadline would seem to indicate that the reading was within their level or comprehension. One other logical possibility would be that the tests in 2008 are easier, but the subjective judgement of this writer does not think that this is the case. Figure 10, with summary statistics clearly shows that students were more successful with the quizzes. It also shows that the students read more books each – 3.65 vs 3.16 in the previous year. (In addition to these, there was one initial book that all students in the class read for homework and which was then discussed in class as a means of familiarizing them with the extensive reading procedure. This book has been excluded from the statistics for both years.) Student Quizzes Quizzes Total Adjusted Pass/ Quizzes Year Number Passed Failed Total Fail Per ratio student 2007 117 301 69 370 370 4.3 3.19 2008 204 687 58 745 427.2 11.8 3.69 Figure 10 – Summary Statistics, April-May Procedure for Setting Up the Reader Module 1. Instructors install the "Reader Module" in their Moodle course. 2. They import, from a central repository, quizzes for the books that they have in their own ER collection. They may also create their own quizzes for any books for which quizzes are not available. 3. They determine which reading level the books should be set at and the point value for each book in case they are significantly longer or shorter than the “standard” graded reader at that level. This only need be done if the instructor wishes to depart from recommended levels in the repository. Finally, the initial reading level for each student needs to be determined. 4. They set parameters in the adminstrative panel such as the target point goal the time limit where quizzes may be accessed (via IP address ranges) the passing percentage, the frequency with which quizzes can be taken 5. They determine a policy for how points gained, or how total points above and below the set target point value are to be integrated into the students’ final class evaluation. Procedure for Students 1. Students may select any book at their current reading level ("RL") or from those at the RL below their current level. 2. Students borrow books, read them and when they are ready to take a quiz, they log into the Moodle course. They select the publisher, level and title and take the quiz within the determined time limit. (15 minutes recommended). Students may refer to the book when taking the quiz. 3. If students pass the quiz, they are awarded one point, none if they fail. 4. Books at any level that are considerably longer or shorter than the average graded reader at that level may have a higher or lower point value. 5. When students have passed a specific number of quizzes at their current level, they are promoted to the next level. They can still read and take quizzes on books at their previous level up to a set number. (Three at Sandai). Publishers and the Reader Module 1. Graded reader publishers will be offered the opportunity to password-protect the quizzes for their publications. These passwords can then be offered to teachers on request, through a teacher’s club web page or however desired. This will allow the publishers to monitor usage and ascertain that quizzes for their publications are being used only at institutions with legitimate copies of their readers. 2. Publishers will be asked to provide the quizzes for current (or new) publications that they wish to be included in the system, but for which quizzes are not yet available. A template and guidelines for quiz preparation will be provided. Publishers will also be asked to provide a graphic of the cover of each text as a 100px jpeg image (at 72 dpi). Conclusion While it is still too early to state definitively that the Reader Module encourages more extensive reading than the past system, it does have certain advantages that make it easier to implement, and which should encourage schools to implement ER as part of their own curricula: 1) the module is free, as is Moodle, in which it is implemented 2) while one staff member must oversee the program, the quiz program requires little intervention on the part of the classroom teacher, apart from periodic progress checks and encouraging students to read more. 3) the work of creating quizzes is a cooperative enterprise among the users, in cooperation with the graded reader publishers, which will allow all instructors a wider range of high-quality quizzes than could possibly be created on an individual basis. While the Reader Module does not solve the problem of management of the book graded reader library, it goes a long way towards allowing extensive reading to be administered on a curriculum wide level. Finally, this is an on-going project. Extensive reading enthusiasts are welcome to try out the system and particularly, to help develop quizzes for those graded readers that are not yet in the system. Please contact the author if you are willing to help. URL: http://moodle2.kyoto-su.ac.jp/moodle/course/view.php/id=116 Username: readerlook Password: readerlook