VIEWS: 16 PAGES: 49 POSTED ON: 8/14/2011
Outcomes Assessment in the Distance-Learning Environment We are: Citrus College, Glendora, California (LA County) Cedar Valley College, Lancaster, TX (Dallas) • Marcy Morris: Learning Center, English (CC) • Theresa Villeneuve: Communications, Fine Arts • Tom Eiland: English (CC) • Lisa Nightingale: Instructional Support & Dist. Ed, Psychology (CVC) • Eric Nightingale: Psychology, History (CVC) Citrus College, Glendora, Cedar Valley College California (LA County) (Dallas CCCD) • 11,000 Students • 4800 students • 100+ DE sections this • 160+ DE sections this semester semester • As many as 4000+ • As many as 4000+ students per semester students per semester The Questions… • Do differences exist in outcomes assessment for the distance education / online environment vs. the on-ground, traditional environment? • Should they be different for the DE environment? • Citrus’ DE program & the DCCCD have been evaluating/reevaluating outcomes for 11 years. DE Curriculum Outlines— The History… The first big questions… • 1996—discussion about whether DE courses should have a separate course outline • contact hours—connection with students? More History… • 1998—incorporated the Distance Education course outline into the conventional course outline. • Independent/Directed Study added as method of instruction. • Instructor syllabus included which specifies how contact occurs. The History, continued… • March 10, 1998—California Board of Governors approved recommendations to Title 5 Regulations on Distance Education. • Each DE course to be separately reviewed and approved. • Emphasis on regular effective contact And Yet… Even More History… • 2004: Distance Learning Course Outline Addendum • Benefit of providing course in DE format • Methods of contact • Course objectives / outcomes / assignments / assessments • Methods of evaluation for online mode. Accreditation Standards...a tilt…? • An effective institution maintains an ongoing, self-reflective dialogue about its quality and improvement (Introduction to the Accreditation Standards 2002) But, That Was Not Enough... • Distance Learning Manual • published August 2006 by Accrediting Commission for Community and Junior Colleges (ACCJC) / Western Association of Schools and Colleges (WASC) ACCJC / WASC Distance Learning Manual • Institution: Academic standards for all courses and programs should be the same as for all other educational experiences delivered by the institution. • Students: Students should expect that distance learning programs will permit completion of learning outcomes and objectives in the same manner as those delivered in traditional programs. • which validated our DE curriculum “journey” Interesting Acknowledgement: methodology used in DE classes have been incorporated into classrooms on campus • Communication: • Email • Web • Discussion / message / announcement Board • Chat • Other: • Websites/FAQ’s • Course and Study Materials • Turnitin.com • Sample material The Charge • To identify elements of quality online courses rather than elements of good instruction. • To identify class management standards and responsibilities for teaching online. The Action Review of existing course standards documents/models Quality Matters: Inter-Institutional Quality Assurance in Online Learning A Grant Initiative of MarylandOnline Sponsored by the U.S. Dept. Education Fund for the Improvement of Postsecondary Education (FIPSE) FIPSE Interested Because … • Quality assurance of courses is important • Voluntary inter-institutional assurance has never been done before • This can serve as a national model Quality Matters! For Our Purposes, Quality Is… Higher 100 • More than average; Standard more than “good enough” • An attempt to capture what’s expected in an 50 effective online course • Based on research and widely accepted standards 0 “They” say there is . . . . “No Significant Difference” • In student outcomes between alternate modes of educational delivery . . . but • The No Significant Difference Phenomenon (Thomas L. Russell 2001, IDECC) • includes 355 research reports, summaries, and papers that document no significant differences (NSD) http://nosignificantdifference.wcet.info there is a Significant Difference • When comparing face-to-face education with Internet-Based Distance Education, there is a significant difference in favor of the Internet-Based Distance Education . . . . In regards to student outcomes Significant Difference - Better Results with Technology 2006 - Sahin, C. S.Overcoming the "No Significant Difference" Phenomenon in Distance Education by Internet Andolu University, Turkey http://aof.edu.tr/iodl2006"... Example: Freshman Composition 101: • 1996 : Created freshman reading and composition course for distance learning • Faculty demands: • Maintain course standards • Administration requirements: • Cannot change course outline • Only add Independent Study Traditional Primary Learning Objectives • Analyze Fiction Objectively • Present Research Paper • Demonstrate Use of Literary Terms in Deductive Essay • Utilize MLA Formatting Distance Education Objectives • Maintaining the Flexibility of Distance Education • Providing Students with Quality Educational Experience • Keeping Distance Ed Classes Legitimate in the Eyes of Academia Traditional Assessment Tools • Timed, Proctored, Deductive Essay • Analytical Research Paper • In-class Lecture • In-class Discussion • Topical Presentation Primary Hurdle • Primary concern among English faculty when presented with possibility of distance education Freshman Comp online course was synchronous class contact. • Theory was that in order to understand the application of literary terms in the analysis of fiction, one must be able to practice through class conversation Other factors . . . . • Ensuring rigidity of testing • Time • Security • Content • Format • Maintaining quality and quantity of written research essays as required by established college standards and IGETC agreement (Intersegmental General Education Transfer Curriculum) Solutions… • Classroom time / Synchronous Contact • Replaced with weekly chat room meetings • Synchronous conversation with students • Chat room transcript archived on web site for review and retrieval • All students required to attend or to read archive and respond Solutions…continued • Testing Protocol: • Establishment of Proctored Testing Center • Testing center controls: • Allowed materials • Allotted time • Identification of test-taker • Adapted Testing needs More Solutions • Term Papers: • Strict adherence to established guidelines • Content • Length • Format • Peer editing asynchronously through message board • Turnitin.com • SafeAssign Solutions… (almost done) • Asynchronous Contact • Message board • E-mail • Web page • Archived chat room • Web links Results • Comparing ENGL 101 and ENGL 103 Freshman Composition Courses • Retention is higher in on-campus courses • Traditional Avg: 22.5/30 • Distance Learning Avg: 15.3/30 Results…continued • Grade range reflects a standard curve in traditional courses (frown) • Traditional Delivery Averages: • A: 18% • B: 36% • C: 24% • D: 22% More Results… • Distance education grade range is wider (smile) • Distance Education Grade Averages: • A: 38% • B: 23% • C: 8% • D: 31% FACTORS (as determined by observation, surveys and interviews) • Population • Age (older is better) • Experience (older is still better) • Distractions (less is better) • Motivation (more is better) • Delivery Method • Technology (BB, attachments, chat room) • Influence of real, live person What It Means . . . . • Delivery methods create some different outcomes BUT… • Assessments tools and criteria are the same • Testing center more uniform than individual classroom practices (now being used for traditional courses as well) • Technical Support would help many students succeeded in distance education • The distance education version of freshman composition at Citrus College has as much, if not more, student learning outcome assessment and verification than the on-campus version of the same course. Art 199 Motion Picture Appreciation Distance Ed vs. Traditional Deliveries Content Of The Course • Social/technical factors of film production • Genres • Aesthetics • Narrative and non-narrative construction • Film form and history • Ability to make sophisticated small talk at cocktail parties. Concerns • Logistical concerns re: films • Rigor • Quality and Quantity of education to maintain IGETC agreement • I am the only faculty member who teaches this course. Methods of Instruction (from course outline) • Lecture • Lecture/Discussion • Independent Study • Out of class film viewing • Directed study Assessment • Discussion • Large Paper • Quizzes • Taken online • Tests • Taken in testing center under controlled conditions until this semester What it means • No one flies under the radar • No one can sleep through a film and pass my class • Testing is controlled • Criteria are unchanged • At least the same learning outcomes as a traditional course. Overall Midterm Comparison, Fall 2006 16 14 12 10 8 Distance Ed 6 Traditional 4 2 0 As Bs Cs Ds Fs Final Comparison, Fall 2006 14 12 10 8 Distance Ed 6 Traditional 4 2 0 As Bs Cs Ds Fs CVC Online Success & Retention Online Course Success and Retention 100% 80% 60% Overall Success % 40% Overall Retention 20% 0% Spring 2005 Fall 2005 Semester Spring 2005 Fall 2005 Overall Success 58% 56% Overall Retention 81% 79% Enrollments 1939 2066 CVC Spring 2006 Distance Learning and On Campus Grade Distributions Course Instr. Method College Success Retention ACCT-2301 Prin Accounting I On Campus 47% 70% ACCT-2301-3427 INET Distance Learning 50% 75% BIOL-1408 Biology Non-Sci Maj I On Campus 58% 79% BIOL-1408-3460 INET Distance Learning 17% 50% BIOL-1408-3461 INET Distance Learning 53% 89% BIOL-1408-3462 INET Distance Learning 58% 75% BIOL-1408-3463 INET Distance Learning 49% 78% BIOL-1408-3464 INET Distance Learning 80% 100% BIOL-1409 Biology Non-Sci Maj II On Campus 70% 84% BIOL-1409-3460 INET Distance Learning 71% 100% BIOL-1409-3461 INET Distance Learning 56% 78% BIOL-1409-3462 INET Distance Learning 54% 85% BMGT-1303 Prin of Management On Campus 60% 81% BMGT-1303-3426 INET Distance Learning 100% 100% BMGT-1303-3427 INET Distance Learning 42% 82% BUSI-1301 Intro to Business On Campus 54% 84% BUSI-1301-3426 INET Distance Learning 100% 100% BUSI-1301-3427 INET Distance Learning 50% 78% COSC-1300 Computer Literacy On Campus 64% 82% COSC-1300-3426 INET Distance Learning 67% 93% COSC-1300-3428 INET Distance Learning 60% 84% COSC-1401 Microcomp. Conc./Apps. On Campus 77% 89% COSC-1401-3426 INET Distance Learning 82% 88% COSC-1401-3427 INET Distance Learning 44% 80% COSC-1401-3428 INET Distance Learning 39% 78% COSC-1401-3429 INET Distance Learning 100% 100% CVC Spring 2006 Distance Learning and On Campus Grade Distributions ECON-2301 Prin Economics On Campus 63% 80% ECON-2301-3426 INET Distance Learning 33% 33% ECON-2301-3427 INET Distance Learning 49% 72% ECON-2302 Prin Economics On Campus 66% 78% ECON-2302-3426 INET Distance Learning 61% 61% ENGL-1301 Composition I On Campus 58% 78% ENGL-1301-3420 INET Distance Learning 45% 55% ENGL-1301-3421 INET Distance Learning 50% 91% HIST-1301 History U S On Campus 55% 80% HIST-1301-3420 INET Distance Learning 63% 73% HIST-1301-3421 INET Distance Learning 57% 57% HIST-1302 History U S On Campus 70% 85% HIST-1302-3420 INET Distance Learning 59% 84% NUTR-1322 Prin of Nutrition On Campus 71% 81% NUTR-1322-3426 INET Distance Learning 36% 62% PSYC-2301 Intro Psychology On Campus 62% 83% PSYC-2301-3420 INET Distance Learning 56% 80% SOCI-1301 Intro Sociology On Campus 68% 84% SOCI-1301-3420 INET Distance Learning 81% 95% SOCI-1301-3421 INET Distance Learning 63% 92% SPCH-1311 Intro Speech Comm On Campus 71% 85% SPCH-1311-3420 INET Distance Learning 73% 87% What Makes DE Work? . . . INFRASTRUCURE! A viable Distance Learning program requires changes to the infrastructure of campus to sustain learning outcomes mandated by: • College District • State Regulations • Accrediting Commission • Federal Government—Title V Infrastructure Changes Testing Center • Grew out of need to offer timed, proctored testing for DE students to establish controls for test taking to insure quality • Meets regular effective contact requirements to authenticate outcomes • Now serves DE as well as the larger campus community--academic makeup testing, assessment testing, adapted testing accommodation, job applicant testing, computer skills competency testing. Infrastructure— Supporting Services of SLO’s in DE • Distance Education Department • Administrative, Clerical, and Technology Support Staff • Training for Students and New DE Instructors • Orientation of New Students and New Faculty • Online Services including registration and tutoring • DE Student Support staff (between technical & counseling) The End? • DE was implemented long before and is keeping pace with if not exceeding SLO requirements for traditional courses • Distance Education has clearly prompted the college to reevaluate outcomes, has affected how we teach in the traditional classroom, and has effected a change in infrastructure to meet needs in regards to outcomes.
Pages to are hidden for
"SLO's . "Please download to view full document