Vol. 38, No. 9 647
Medical Student Education
Web-based Versus Face-to-Face Learning
of Diabetes Management: The Results
of a Comparative Trial of Educational Methods
John M. Wiecha, MD, MPH; V.K. Chetty, PhD; Timothy Pollard, MD; Peter F. Shaw, PhD
Background and Objectives: Relatively little is known about the effectiveness of Web-based learning
(WBL) in medical education and how it compares to conventional methods. This study examined
the inﬂuence of an interactive, online curriculum in a third-year medical school family medicine
clerkship on students’ ability to create a management plan for a patient newly diagnosed with type
2 diabetes. We also evaluated how the online curriculum compared to a conventionally taught cur-
riculum. Methods: The online course included three integrated activities: (1) self-study modules,
(2) a patient case study, and (3) a moderated discussion board for posting and discussing patient
care plans. The WBL curriculum was compared to small-group case-based sessions with a faculty
facilitator. Students completed a test case before and after the clerkship. Results: Among standard-
of-care diabetic management interventions not ordered on the pretest, 38% were subsequently cor-
rectly ordered by WBL students on the posttest, versus 33% by students in the comparison group. For
four out of ﬁve subgroups assessed on the case write-ups, the gain from before to after the clerkship
favored the WBL group. Conclusions: Improvement among students learning online exceeded that
of students learning face to face. This suggests superiority of the online method, a ﬁnding consistent
with other recently published, well-controlled studies.
(Fam Med 2006;38(9):647-52.)
Courses in the clinical years of medical school often interactions with peers or full-time medical school
place students in preceptor arrangements in off-campus faculty.
clinical sites with community-based faculty. Because of Leaders in medical education have challenged medi-
the many beneﬁts of these educational settings, medi- cal schools to “exploit the potential of distance learning
cal education experts have advocated acceleration of technology to deliver educational programs in which
community-based teaching.1 instruction and evaluation are of a consistent and high
Despite the beneﬁts of community-based teaching, standard across multiple settings in the community.”3
students and faculty in such settings are often physi- The Association of American Medical Colleges’ Medi-
cally remote from the academic medical center2 and cal School Objectives Project has recommended the use
have restricted access to conventionally structured of interactive Web-based courses to supplement and
curricular materials delivered at the medical school complement conventional curricula.4
campus. Students in community-based clerkships may Properly designed Web-based learning (WBL)
be placed alone and, relative to clerks on rotations at the programs can serve to efﬁciently deliver educational
academic health center, often have fewer educational programs to students, residents, or practicing physicians
who may be far from each other and from full-time
medical school faculty. WBL can also help to standard-
ize students’ learning, ensuring that all students are
From the Department of Family Medicine (Drs Wiecha and Chetty) and the exposed to essential curricular elements.
Ofﬁce of Medical Education (Dr Shaw), Boston University; and the Greater Given its recent introduction into medical educat-
Lawrence Community Health Center, Lawrence, Mass (Dr Pollard). ing, relatively little is known about the effectiveness of
648 October 2006 Family Medicine
WBL and how this method compares to conventional To encourage peer learning, all students in the dis-
face-to-face educational methods. This report describes cussion groups were required to read and comment on
the design, teaching, and evaluation of a Web-based at least one other student’s posting each week. With
curriculum for third-year medical students on a family each passing week, the online diabetes case reﬂected
medicine clerkship rotation. passage of time and progression of disease with new
The content area selected for our WBL activity was management challenges.
type 2 diabetes mellitus (T2DM), a disease of epidemic In addition to studying T2DM online, clerkship
proportion in the United States, and the second most- students also completed a Web-based evidence-based
common reason for visits to family physicians in the medicine (EBM) curriculum and an online medi-
United States.5 The age-adjusted prevalence of diabetes cal humanism journal activity. Blackboard software
has increased by 54% among US adults during the (Blackboard 2000, Blackboard, Inc, Washington, DC)
period 1994–2002.6-9 We examined the extent to which was used to organize course materials and activities
implementation of an interactive, online curriculum in a for all three curricula, collectively called the Online
family medicine clerkship improved students’ ability to Clerkship (OC). Students were required to access the
create a management plan for a patient newly diagnosed OC at least twice weekly and were provided protected
with T2DM and how the online curriculum compared time to do so.
to a conventionally taught method.
The Comparison Face-to-Face Curriculum
Methods The WBL curriculum was compared to our standard
Design and Content of the Web-based Curriculum clerkship didactic educational activity, which was
Our approach to WBL course design has been small-group case-based sessions with a faculty facilita-
described previously.10-12 It is guided by educational tor. These sessions occurred on 4 days when students
psychology principles including adult learning, reﬂec- were not seeing patients. They were provided with
tive learning, and collaborative learning.13-15 Our study outpatient medical charts for a simulated family16 and
was approved by the Boston University Medical Center discussed management plans for each visit of a member
institutional review board. of the simulated family. This curriculum addressed a
The content of the WBL curriculum was adapted variety of common outpatient medical conditions (such
from an existing face-to-face (F2F) case-based cur- as T2DM, low back pain, asthma, pregnancy, urinary
riculum, originally created according to a simulated tract infection, alcoholism, and others) over simulated
family model,16 designed to promote competence in passage of time.
the diagnosis, initial evaluation, and management of a Students who were in comparison F2F groups were
patient newly diagnosed with T2DM. The online WBL taught the principles of management of T2DM using
course included three integrated educational activities: this seminar approach and had no online assignments.
(1) self-study modules, (2) a patient case study, and (3) a Like students in the WBL groups, the assignment for
moderated discussion board for posting and discussing students in the F2F groups was to discuss and develop
patient care plans. a management plan for a patient newly diagnosed with
Students ﬁrst studied an online module composed of T2DM. Students had access to printed diabetes care
12 html pages with text and interactive features. The guidelines comparable in content to material accessed
module addressed the diagnosis and initial management by online students in their html-based module. Each
of T2DM diabetes based on current guidelines.17 weekly diabetes case encounter, both WBL and F2F
Next, students applied these concepts to a case study version, was designed to require equivalent duration
presented in three forms: a streaming video of a case (about 1 hour and 15 minutes) of student engagement.
patient describing her symptoms, a simulated electronic Small groups were facilitated by four department
medical record, and a brief written case summary. The faculty who, to promote standardization of curriculum
case patient was a middle-aged Hispanic woman with presentation and teaching process, underwent formal
symptoms of DM and risk factors for T2DM including training in the curriculum in faculty development ses-
obesity, family history of T2DM, and hypertension. sions. They used a written guide for faculty modera-
After reviewing these materials, learners then tors.
described their assessment and management plan
in a posting to an online asynchronous discussion The Study Sample and Evaluation Methods
board. The discussion board activity represented a Data were gathered from third-year medical stu-
reﬂective learning step in the curricular cycle. These dents participating in the Boston University School
discussions were moderated by a trained faculty mem- of Medicine family medicine clerkship from January
ber who provided feedback, asked for clariﬁcation, and 2001 through June 2002. Students in even-numbered
probed the learner for deeper reﬂection and understand- clerkship blocks (2,4,6,8) were assigned to the WBL
ing using established principles of moderating online group. Students in odd blocks (1,3,5,7) were assigned
educational discussion boards.18 to the F2F curriculum.
Medical Student Education Vol. 38, No. 9 649
Clerks completed surveys that recorded baseline higher posttest score and a greater gain in overall
demographic data and their experience with the WBL score from pretest to posttest than did students in the
curriculum. To evaluate the effect of the DM curricula, non-online group.
at the start and end of the clerkship on the ﬁnal testing Subgroup analysis shows that for four out of ﬁve
day, students wrote an assessment and management categories assessed on the case write-up, the difference
plan based on a paper-based case of a middle-aged male in gains favored the WBL group, although the differ-
patient who presented to the ofﬁce with lab values suf- ence in gain for only one subgroup (“Plan”) reached
ﬁcient to diagnose T2DM. The validity of such a written statistical signiﬁcance. Among subjects who did not
case-based approach to clinical skills assessment has order an American Diabetes Association (ADA)-rec-
been established.19 ommended intervention at baseline, subjects in the
The student management plans were completely WBL group were more likely than students in the F2F
blinded so that research assistants reviewing them could groups to correctly order the intervention on the post-
not determine students’ name, date, clerkship block, test (Table 2).
before or after status, or intervention group status. Each Students reported roughly equivalent attitudes to-
management plan was rated by using a checklist based ward enjoyment of the two curriculum formats and
on then-current clinical guidelines from the American indicated similar impressions of effectiveness (Table
Diabetes Association for the content of the initial visit 3). Most students felt the WBL curriculum was easy
for a newly diagnosed patient with T2DM.17 to use, and more than 50% reported that the online
The case was constructed such that drug therapy was discussion groups were valuable (Table 3).
unequivocally indicated. The general domains rated in
the 32-item checklist included appropriate additional Discussion
history and review of systems sought, appropriate This study demonstrated that improvements in
physical exam planned, assessment (diagnosis of T2DM student performance on a case study among students
stated), plan (correct labs ordered and appropriate drug learning online exceeded those of students learning in
therapy recommended), and referrals and counseling face-to-face groups. This suggests at least equivalence,
documented. The same case was administered before if not superiority, of the online method, a ﬁnding con-
and after the clerkship. Students were aware that scores sistent with other recently published, well-controlled
on this assessment were not included in their ﬁnal clerk- studies evaluating effectiveness of online learning in
ship grade determination. medical education.20-22 There were no signiﬁcant differ-
ences in student attitudes with respect to effectiveness
Data Analysis or enjoyment of the two learning methods, suggesting
In addition to comparing overall preclerkship and that both may be equally acceptable from a learner
postclerkship scores, a subset analysis was done of those satisfaction standpoint.
interventions not ordered on the pretest to measure In the online group, students were able to learn
what proportion were subsequently correctly ordered from engagement in a curriculum that included no F2F
on the posttest. instruction with on-campus faculty. In both experi-
The effectiveness of the intervention was tested by mental and control groups, the overall posttest results
using the Mantel-Haenzel procedure. This statistic was were only modestly superior to baseline performance.
computed using the mantelhaen.test function in S-Plus- Three possible biases may account for this ﬁnding and
version 7 (Insightful Corporation, Seattle). illustrate the complexity of carrying out research in
real educational settings. First, the pretests and post-
Results tests were not used in determining the ﬁnal grades of
The ﬁnal evaluation case was completed by 159 students, which may have contributed to suboptimal
(88.3%) clerkship students, with a similar response rate student effort, whereas the other tests administered on
in both study groups. There were no signiﬁcant differ- the last clerkship day were used to evaluate and grade
ences between students in the online versus comparison student performance. Second, the content in the case
groups in terms of age, gender, race/ethnicity, Medical study was not included in the students’ course learning
College Admissions Test (MCAT) scores, preclinical objectives, so students would probably be less likely to
grades, or previous Internet use patterns. Students study the material using methods other than the F2F
reported spending an average of 4.3 hours weekly on or online groups. This was an attempt to isolate the
all three activities in the online clerkship. Students ac- learning associated with these methods only. Third, we
cessed the project Web site from home (61%), preceptor suspect that students may have exerted less effort on
ofﬁce (24%), or medical school (10%) campus, and the the posttest versus the pretest. Speciﬁcally, the posttest
remainder did so from other locations. environment, following a day of other testing, may have
Results from the evaluation cases are shown in Table reduced student effort and biased scores downward,
1. Students in the WBL group showed a statistically reducing the apparent effectiveness of both curricula.
650 October 2006 Family Medicine
Performance Outcomes of Students Enrolled in the Online Diabetes Curriculum
Conventional Diabetes Curriculum Online Diabetes Curriculum
Mean Percent Correct** Mean Percent Correct**
Gain Gain Difference P Value
Evaluation Case (After- (After- in for Difference
Component Before After Before) Before After Before) Gain*** in Gain
History (12 items) 34.6 30.6 -4.02 34.9 37.4 2.5 6.6 .069
Essential exam (four items) 66.6 72.9 7.32 60.8 72.9 12.1 4.8 .29
Assessment (two items) 61.7 71.4 9.76 62.5 68.2 5.7 -4.0 .60
Plan (seven items) 30.4 37.1 6.71 26.0 44.1 18.1 11.4 .025
Referrals/education 44.7 43.5 -1.19 43.3 46.7 3.5 4.7 .32
All items 40.8 42.1 1.30 39.2 46.9 7.2 6.4 .008
* Sample sizes vary across tables since some clerkship blocks were not administered all instruments.
** Average of the percentage correct among items in each category.
*** Gain in online group minus the gain in conventional curriculum group.
between the pretests and posttests
Table 2 might have occurred. Nonetheless, such
fatigue would have been experienced
Change in Test-case Management Behavior of Students Enrolled presumably equally between the WBL
in Web-based Versus Conventional Diabetes Curriculum and control group students and does
not invalidate the observed favorable
Conventional Diabetes Curriculum Online Diabetes Curriculum results of the WBL group.
Total # of Interventions Total # of Interventions The subgroup analysis of gain,
Interventions Not Correctly Ordered Interventions Not Correctly Ordered among those interventions not ordered
Ordered on at Posttest Ordered on on Posttest at baseline, was included to demon-
Pretest* n (%)** Pretest* n (%)** P Value
strate the relative inﬂuence of the two
teaching methods, while isolating to
1,549 512 (33.1) 1,336 507 (38.0) .018
the extent possible the influence of
* The total number of interventions, summed across all students in each group, that should have
differences in the pretest and posttest
been ordered under ADA guidelines for the case patient but were not. environments. This subgroup analysis
showed that students in both groups
** A measure of how many (and %) of interventions not ordered on the pretest were subsequently improved substantially, and those who
correctly ordered on the posttest.
learned online were signiﬁcantly more
likely than controls to order interven-
tions on the posttest that they had ne-
The pretest was administered in the morning of the glected to order at baseline.
ﬁrst clerkship day while students were fresh, whereas Of note, the most signiﬁcant difference between
the only opportunity for administration of the posttest performance of WBL and F2F groups was in the abil-
was at the end of a full day of testing. The pretest was ity to suggest an appropriate treatment plan, including
the only survey given on day 1, whereas the posttest lab screening and pharmacotherapy. The substantial
was the last of four tests. superiority of the online method of learning in these
Time of day and sequence of examinations have key domains might be attributable to the requirement
been demonstrated to cause statistically signiﬁcant in the online curriculum that students write, and post
variation in student performance on examinations. It 23 for feedback, a management plan for the case patient,
seems possible that given these factors, testing fatigue whereas in the F2F group, oral discussion is used to
Medical Student Education Vol. 38, No. 9 651
Student Attitudes Toward Web-based Learning and Face-to-Face Diabetes Curricula
Conventional Diabetes Curriculum Online Diabetes Curriculum
Attitudes Toward the Diabetes Agree Neutral Disagree Agree Neutral Disagree
Curriculum n (%) n (%) n (%) n (%) n (%) n (%) P Value
I enjoyed working on the diabetes 55 (53.4) 35 (34.0) 13 (12.6) 50 (47.6) 29 (27.6) 26 (24.8) .08
The diabetes curriculum was effective 74 (71.8) 15 (14.6) 14 (13.6) 73 (69.5) 16 (15.2) 16 (15.2) .93
in teaching management of type 2
Attitudes Toward the Online Curriculum
— — — 76 (58.0) 31 (23.7) 24 (18.3) —
The online course was easy to use.
— — — 72 (55.8) 33 (25.6) 24 (18.6) —
I valued interacting with other students
online during the threaded discussion
— — — 76 (58.0) 34 (26.0) 21 (16.0) —
I valued interacting with faculty online
during the threaded discussion groups.
develop consensus about a management plan, which seminar. This study has two important implications:
is written on a blackboard by one student. In the F2F students can be taught effectively using asynchronous
groups, it is possible for students to participate and communication on the Web, and Web-based curricula
engage in the discussion and planning at more variable improve the geographic availability of teaching and
levels than is possible online. In other words, students pave the way for further decentralization of medical
can be relatively disengaged during F2F sessions, school instruction.
whereas this is less possible among Web learners given The ﬁndings on the effectiveness of this WBL educa-
the design of the online sessions. Further, the process tion approach to EBM training also add to a growing
of writing and submitting for peer and faculty review and generally favorable body of literature on online
is likely to encourage deeper reﬂection about the case learning.21 Although “evaluation of Web-based learning
and supporting material. (WBL) is in its infancy,”27 the comparative research on
Learning styles differ from student to student.24,25 distance-learning technologies is massive, dating back
These differences incorporate student preferences as to the early 20th century.28-30 A recent review noted that
well as ability to learn, integrate, and utilize data from overall WBL appears to be equivalent in knowledge
different sources. It would be surprising, therefore, if gains to other educational methods.27 We have subse-
any educational initiative found the best way to teach quently used the methods developed in this research
every student. A rich environment with a variety of in a number of other projects, including international
teaching and learning techniques and options for stu- continuing medical education (CME) training and
dents to choose among those techniques is likely to patient education.11,31
be a more proﬁtable approach, and WBL is proving Leaders in medical education have called for medical
to be an important new option. Future research needs schools to rely more heavily on high-quality distance
to deﬁne how to map WBL design to cognitive and learning.3 In response, we have developed a new WBL
learning styles.26 method of teaching key components of diabetes care.
With these additional favorable data on the effective-
Conclusions ness of WBL, we anticipate that medical educators will
This randomized controlled trial of a speciﬁc teach- increasingly take advantage of emerging distance learn-
ing and learning method, asynchronous Web-based ing technologies to improve the quality of undergradu-
instruction, demonstrated the ability of students to ate, graduate, and postgraduate medical education.
learn diabetes management skills as well as in an F2F
652 October 2006 Family Medicine
Acknowledgments: This study was supported by the Robert Wood Johnson 14. Novack DH, Epstein RM, Paulsen RH. Toward creating physician-heal-
Generalist Physician Faculty Scholars program and by HRSA Training in ers: fostering medical students’ self-awareness, personal growth, and
Primary Care Medicine and Dentistry Grant #5 D56HP03355-02-00, both well-being. Acad Med 1999;74(5):516-20.
to Dr Wiecha. 15. Davis BG. Tools for teaching. San Francisco: Jossey-Bass Publishers,
Corresponding Author: Address correspondence to Dr Wiecha, Boston 16. Pugnaire MP, Leong SL, Quirk ME, Mazor K, Gray JM. The standard-
University, Department of Family Medicine, Dowling 5S, Boston Medical ized family: an innovation in primary care education at the University
Center, 1 BMC Place, Boston MA 02118. 617-414-4465. Fax: 617-414- of Massachusetts. Acad Med 1999; 74(1 Suppl):S90-S97.
3345. firstname.lastname@example.org. 17. American Diabetes Association. Standards of medical care in diabetes.
Diabetes Care 2005;28(4):990.
REFERENCES 18. Collison G, Elbaum B, Haavind S, Tinker R. Facilitating online learning:
effective strategies for moderators. Madison, Wis: Atwood, 2000.
1. Schroeder SA. Expanding the site of clinical education: moving beyond 19. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison
the hospital walls. J Gen Intern Med 1988;3(2 Suppl):S5-S14. of vignettes, standardized patients, and chart abstraction: a prospec-
2. O’Neil EH, and the Pew Health Professions Commission. Recreating tive validation study of three methods for measuring quality. JAMA
health professional practice for a new century. San Francisco: Pew 2000;283(13):1715-22.
Health Professions Commission, 1998. 20. Fordis M, King JE, Ballantyne CM, et al. Comparison of the instructional
3. Council on Graduate Medical Education. Thirteenth report of the Coun- efﬁcacy of Internet-based CME with live interactive CME workshops:
cil on Graduate Medical Education (COGME): physician education for a randomized controlled trial. JAMA 2005;294(9):1043-51.
a changing health care environment. Rockville, Md: COGME, March 21. Cook DA, Dupras DM, Thompson WG, Pankratz VS. Web-based learn-
1999. ing in residents’ continuity clinics: a randomized, controlled trial. Acad
4. Contemporary issues in medicine—medical informatics and population Med 2005;80(1):90-7.
health: report II of the Medical School Objectives Project. Acad Med 22. Lipman AJ, Sade RM, Glotzbach AL, Lancaster CJ, Marshall MF.
1999;74(2):130-41. The incremental value of Internet-based instruction as an adjunct to
5. National Center for Health Statistics. National Ambulatory Medical Care classroom instruction: a prospective randomized study. Acad Med
Survey, 2004. Washington, DC: US Department of Health and Human 2001;76:1060-4.
Services, Centers for Disease Control and Prevention, National Center 23. Battles JB, Carpenter JL, McIntire DD, Wagner JM. Analyzing and
for Health Statistics, 2006. adjusting for variables in a large-scale standardized patient examination.
6. Harris MI, Hadden WC, Knowler WC, Bennett PH. Prevalence of Acad Med 1994;69(5):370-6.
diabetes and impaired glucose tolerance and plasma glucose levels in 24. Mehta MP, Sinha P, Kanwar K, Inman A, Albanese M, Fahl W. Evalua-
US population aged 20-74 years. Diabetes 1987;36(4):523-34. tion of Internet-based oncologic teaching for medical students. J Cancer
7. Harris MI. Undiagnosed NIDDM: clinical and public health issues. Educ 1998;13(4):197-202.
Diabetes Care 1993;16(4):642-52. 25. Lynch TG, Woelﬂ NN, Steele DJ, Hanssen CS. Learning style inﬂuences
8. Harris MI, Flegal KM, Cowie CC, et al. Prevalence of diabetes, im- student examination performance. Am J Surg 1998;176(1):62-6.
paired fasting glucose, and impaired glucose tolerance in US adults. The 26. Cook DA. Learning and cognitive styles in Web-based learning: theory,
Third National Health and Nutrition Examination Survey, 1988–1994. evidence, and application. Acad Med 2005;80(3):266-78.
Diabetes Care 1998;21(4): 518-24. 27. Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound
9. King H, Rewers M. Global estimates for prevalence of diabetes mel- educational method or hype? A review of the evaluation literature. Acad
litus and impaired glucose tolerance in adults. WHO Ad Hoc Diabetes Med 2002;77(10 Suppl):S86-S93.
Reporting Group. Diabetes Care 1993;16(1):157-77. 28. Imel S. Trends and issues alerts: distance education. Eric Clearinghouse
10. Wiecha JM, Gramling R, Joachim P, Vanderschmidt H. Collaborative on Adult, Career, and Vocational Education, Ohio State University,
e-learning using streaming video and asynchronous discussion boards 1996.
to teach the cognitive foundation of medical interviewing: a case study. 29. Imel S. Distance education: myths and realities. Eric Clearinghouse
Journal of Medical Internet Research 2003;5(2):e13. on Adult, Career, and Vocational Education, Ohio State University,
11. Wiecha JM, Barrie N. Collaborative online learning: a new approach 1998.
to distance CME. Acad Med 2002;77(9):928-9. 30. Sherry L. Issues in distance learning. International Journal of Educa-
12. Wiecha JM, Vanderschmidt H, Schilling K. HEAL: an instructional tional Telecommunications 1996;1(4):337-65.
design model applied to an online clerkship in family medicine. Acad 31. Wiecha JM, Pollard T. The interdisciplinary eHealth team: chronic care
Med 2002;77(9):925-6. for the future. Journal of Medical Internet Research 2004;6(3):e22.
13. Stagnaro-Green A. Applying adult learning principles to medical educa-
tion in the United States. Med Teach 2004;26(1):79-85.