Distance education by 8Hp08vI3


									      Supporting Engagement in Asynchronous Education
                 Scott LeeTiernan                                                   Jonathan Grudin
              University of Washington                                             Microsoft Research
              Department of Psychology                                             One Microsoft Way
                 Seattle, WA 98195                                                Redmond, WA 98052
               slt@u.washington.edu                                              jgrudin@microsoft.com

ABSTRACT                                                         comments left earlier and can add replies, comments, or
A key challenge for software that supports asynchronous          questions as they watch or read. Success with these
distance education is to engage and guide students who are       powerful features was limited by procrastination: Students
not interacting in real-time. We describe a first study of two   who postpone viewing the material have little time to
approaches to adding interactive exercises to the viewing of     review and reflect on prior comments, and little time or
videotaped lectures. We found individual differences, but a      incentive to add last-minute comments that others are
surprising tendency to prefer more intrusive exercises. We       unlikely to see or respond to. In addition, the lack of
conclude with possible next steps.                               interaction may reduce engagement with the material.
Keywords                                                         In traditional courses, exercise and quizzes help combat
Distance education, interactive multimedia                       procrastination and encourage engagement. We extended
                                                                 MRAS to provide self-assessment questions timed to
INTRODUCTION                                                     appear at specific points in a video lecture. Although not
Videotaped lectures and presentations are already used in        included in this study, students could be given links to the
education, including distance education. As technological        points in a lecture where questions were answered, and
barriers gradually diminish, their use will increase. The        instructors could be notified of student progress.
“talking head” lecture may not prove to be the best form of
instruction online, but the multitudes of experienced            We were interested in knowing how viewers would react to
lecturers, the growing ease of recording their performances,     this feature, and whether they would prefer the appearance
and the fact that people like to see a speaker insures that it   of a question to stop the video, forcing a response, or
will be a widespread resource.                                   appear in a separate window without halting the lecture.
For distributed classes, material available on demand offers     DESIGN AND METHOD
numerous benefits to students. They can watch them when          Using MRAS, 12 participants viewed an 18-minute lecture
they have time and can review portions as needed.                on economic issues surrounding the Microsoft antitrust
Disadvantages include reduced opportunities to discuss           trial. Each annotated the video with public and private
material and to get timely answers to questions. Other           notes, with the goal of developing the two best arguments
problems include procrastination and reduced motivation to       that Microsoft is a monopoly and the two best arguments
engage with the material.                                        that Microsoft is not a monopoly. Six times during the
                                                                 lecture, a topically relevant question was presented. For half
A much-cited Stanford study at [4] showed that studying
                                                                 of the participants, the lecture was paused until they
videotaped lectures is less effective than attending a lecture
                                                                 responded correctly to the question in a multiple-choice
or watching it live on television. Most effective is when a
                                                                 format (the forced response condition). Participants in the
group of students meet to watch and discuss a lecture video.
                                                                 non-forced response condition were exposed to the same six
Two studies have shown that students in different locations
                                                                 questions at the identical points in the lecture, but the
get the same benefit with an audio connection and software
                                                                 questions were presented in the public annotation area as
allowing simultaneous viewing [3, 5].
                                                                 thought questions requiring no response and not stopping
We focused on students working independently—the                 the video stream.
completely asynchronous case. [1, 2] examined a system
                                                                 Following the lecture, each participant completed a ques-
called MRAS that allows students to take notes, ask
                                                                 tionnaire and a comprehension test for the lecture material.
questions, and engage in asynchronous discussions while
watching videos or reading articles on-line. Viewers see         RESULTS
                                                                 The primary measures concerned the effectiveness of each
                                                                 engagement device in aiding comprehension and fostering
 LEAVE BLANK THE LAST 2.5 cm (1”) OF THE LEFT                    participation as well as subjective experience of each
     COLUMN ON THE FIRST PAGE FOR THE                            engagement condition.
Participation                                                      subjective reports. Taking the five measures together,
Participation was measured by the number of annotations            participants in the forced response condition reported the
made, including private notes and contributions to the             lecture and the exercise overall more stimulating than did
public annotation forum. Participants in the two conditions        participants in the non-forced response condition.
were similar in terms of the total number of annotations
                                                                   The greater number of public annotations made by
made (Mforced = 7.83, Mnon-forced = 8.5). However,
                                                                   participants in the non-forced condition was unexpected.
participants in the non-forced response condition
                                                                   One possible explanation is that viewing thought questions
contributed an average of 2.5 annotations to the public
                                                                   already in the public annotation forum created a greater
annotation forum, versus an average of .67 annotations
                                                                   sense of public contribution that carried over into
contributed by participants in the forced response condition
                                                                   participants’ behavior. A second possibility is that
(t(10) = 2.02, p = .07, 2-tailed).
                                                                   participants in the forced response condition made fewer
Comprehension                                                      public annotations because the forced response questions
All participants scored similarly on the lecture                   focused their attention on their own learning experience.
comprehension test, with participants in both engagement           These two possibilities warrant further investigation, as they
conditions averaging 5.67 out of 10 correct responses.             suggest different approaches to meet different pedagogical
Subjective Experience                                              goals.
Five subjective experience measures were collected using           CONCLUSION AND FUTURE DIRECTIONS
7-point Likert scales (Figure 1). An overall trend for a           Synchronizing questions with presentation of material
better subjective experience in the forced-response                seems a promising step to enhancing distance learning.
condition emerged. In terms of perception of the lecture,          Further studies could explore the indications that pausing a
participants in the forced-response condition rated it as          presentation for questions is the right default, although
more interesting (Mforced = 5.5, Mnon-forced = 4.5), less boring   students could have an option to respond as a video rolls.
(Mforced = 2, Mnon-forced = 2.5), and more enjoyable (Mforced =
                                                                   It seems likely that less frequent interruptions will work
5.5, Mnon-forced = 5). With two-tailed significance tests these
                                                                   better. Much education content may well shrink from hour-
effects are not individually statistically significant.
                                                                   long lectures to modules of 10 or 15 minutes, with exercises
                                                                   at the end of each. There is obvious merit in experimenting
                                                                   with open-ended questions rather than multiple-choice, with
    6                                                              links back to the relevant material.
                                                                   Embedded exercises or quizzes could be purely for self-
    4                                            Forced            assessment, but they could also be used to counter
    3                                            Non-Forced        procrastination. At the least intrusive, anonymous
                                                                   summaries of class progress could be forwarded to
                                                                   instructors. At the other extreme, student responses could
                                                                   be forwarded automatically to instructors, which in the case
    0                                                              of open-ended questions could allow constructive feedback,
                                                                   especially for responses not submitted at the last minute.








                                                                   This is early in the exploration of pedagogic benefits of






                                                                   these technologies that are gradually becoming accessible.


Figure 1: Summary of subjective experience                         1. Bargeron, D., Grudin, J., Gupta, A., Sanocki, E., Li, F.
                                                                      & LeeTiernan, S. (2002). Asynchronous collaboration
Regarding the exercise, participants in the forced response           around multimedia applied to on-demand education.
condition felt that they learned more than those in the non-          Journal of MIS, 18, 4, 117-145.
forced response condition (Mforced = 5.5, Mnon-forced = 4; t(10)
= 1.94, p = .08, 2-tailed). Participants in the forced-            2. Brush, A.J.B., Bargeron, D., Gupta, A., & Grudin, J.
response condition also reported that they were forced to             (2002). Notification for shared annotation of digital
think more critically about the material (Mforced = 6.5, Mnon-        documents. Proc. CHI 2002, 89-96.
forced = 5.5), although this is only a mild statistical trend.     3. Cadiz, J.J., Balachandran, A., Sanocki, E., Gupta, A. &
                                                                      Grudin, J. (2000). Distance learning through distributed
                                                                      collaborative video viewing. Proc. CSCW 2000, 135-
Forcing participants to respond to in-line questions requires
more interaction with the lecture than simply presenting
thought questions for consideration while viewing the              4. Gibbons, J. F., Kincheloe, W. R. & Down, K. S.
lecture. This increase in interaction seems to generate a             (1977). Tutored videotape instruction: a new use of
more active, engaging experience, as reflected in the
   electronics media in education. Science, 195, 1139-
5. Sipusic, M., Pannoni, R., Smith, R., Dutra, J., Gibbons,
   J., and Sutherland, W. (1999). Virtual collaborative
   learning: A comparison between face-to-face tutored
   video instruction and distributed tutored video
   instruction (DTVI). Sun Microsystems Laboratories,
   Inc. TR-99-72.

To top