Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

SLO's . by pengxiuhui

VIEWS: 16 PAGES: 49

									Outcomes Assessment in the
Distance-Learning Environment
 We are:
Citrus College, Glendora, California (LA County)
Cedar Valley College, Lancaster, TX (Dallas)

• Marcy Morris: Learning Center, English (CC)
• Theresa Villeneuve: Communications, Fine Arts
• Tom Eiland: English (CC)
• Lisa Nightingale: Instructional Support & Dist. Ed,
      Psychology (CVC)
• Eric Nightingale: Psychology, History (CVC)
Citrus College, Glendora,    Cedar Valley College
   California (LA County)      (Dallas CCCD)
   • 11,000 Students            • 4800 students
   • 100+ DE sections this      • 160+ DE sections this
     semester                     semester
   • As many as 4000+           • As many as 4000+
     students per semester        students per semester
The Questions…
• Do differences exist in outcomes assessment
  for the distance education / online
  environment vs. the on-ground, traditional
  environment?

• Should they be different for the DE
  environment?

• Citrus’ DE program & the DCCCD have been
  evaluating/reevaluating outcomes for 11
  years.
DE Curriculum Outlines—
The History…
The first big questions…

• 1996—discussion about whether DE
  courses should have a separate course
  outline

• contact hours—connection with
  students?
More History…
• 1998—incorporated the Distance Education
  course outline into the conventional course
  outline.

• Independent/Directed Study added as
  method of instruction.

• Instructor syllabus included which specifies
  how contact occurs.
The History, continued…

• March 10, 1998—California Board of
  Governors approved recommendations
  to Title 5 Regulations on Distance
  Education.
• Each DE course to be separately
  reviewed and approved.
• Emphasis on
    regular effective contact
And Yet… Even More History…

• 2004: Distance Learning Course
  Outline Addendum
• Benefit of providing course in DE format
• Methods of contact
• Course objectives / outcomes /
  assignments / assessments
• Methods of evaluation for online mode.
Accreditation Standards...a tilt…?

• An effective institution maintains an
  ongoing, self-reflective dialogue about
  its quality and improvement (Introduction to the
  Accreditation Standards 2002)
But, That Was Not Enough...

• Distance Learning Manual

• published August 2006 by Accrediting
  Commission for Community and Junior
  Colleges (ACCJC) / Western
  Association of Schools and Colleges
  (WASC)
ACCJC / WASC
Distance Learning Manual
• Institution: Academic standards for all
  courses and programs should be the same as
  for all other educational experiences
  delivered by the institution.
• Students: Students should expect that
  distance learning programs will permit
  completion of learning outcomes and
  objectives in the same manner as those
  delivered in traditional programs.

• which validated our DE curriculum “journey”
Interesting Acknowledgement:
  methodology used in DE classes have been
incorporated into classrooms on campus
• Communication:
   •   Email
   •   Web
   •   Discussion / message / announcement Board
   •   Chat
• Other:
   •   Websites/FAQ’s
   •   Course and Study Materials
   •   Turnitin.com
   •   Sample material
The Charge

• To identify elements of quality online
  courses rather than elements of good
  instruction.

• To identify class management
  standards and responsibilities for
  teaching online.
The Action

 Review of existing course standards
  documents/models
Quality Matters:
 Inter-Institutional Quality Assurance
in Online Learning
   A Grant Initiative of MarylandOnline



   Sponsored by the U.S. Dept. Education Fund
   for the Improvement of Postsecondary
   Education (FIPSE)
FIPSE Interested Because …
   • Quality assurance of courses is important
   • Voluntary inter-institutional assurance has
     never been done before
   • This can serve as a national model



          Quality Matters!
For Our Purposes, Quality Is…
                              Higher   100
• More than average;        Standard
  more than “good
  enough”
• An attempt to capture
  what’s expected in an                50
  effective online course
• Based on research and
  widely accepted
  standards
                                       0
“They” say there is . . . .
“No Significant Difference”
• In student outcomes between alternate
  modes of educational delivery . . . but

• The No Significant Difference Phenomenon
  (Thomas L. Russell 2001, IDECC)

• includes 355 research reports, summaries,
  and papers that document no significant
  differences (NSD) http://nosignificantdifference.wcet.info
there is a Significant Difference

• When comparing face-to-face education with
  Internet-Based Distance Education, there is a
  significant difference in favor of the
  Internet-Based Distance Education

     . . . . In regards to student outcomes
Significant Difference - Better Results with Technology 2006 - Sahin, C.
   S.Overcoming the "No Significant Difference" Phenomenon in Distance
   Education by Internet Andolu University, Turkey http://aof.edu.tr/iodl2006"...
Example:
Freshman Composition 101:

• 1996 : Created freshman reading and
  composition course for distance
  learning
• Faculty demands:
  • Maintain course standards
• Administration requirements:
  • Cannot change course outline
  • Only add Independent Study
    Traditional Primary Learning
             Objectives


• Analyze Fiction Objectively
• Present Research Paper
• Demonstrate Use of Literary Terms in
    Deductive Essay
• Utilize MLA Formatting
          Distance Education
              Objectives

• Maintaining the Flexibility of Distance
  Education
• Providing Students with Quality
  Educational Experience
• Keeping Distance Ed Classes
  Legitimate in the Eyes of Academia
Traditional Assessment Tools

•   Timed, Proctored, Deductive Essay
•   Analytical Research Paper
•   In-class Lecture
•   In-class Discussion
•   Topical Presentation
              Primary Hurdle

• Primary concern among English faculty when
  presented with possibility of distance
  education Freshman Comp online course
  was synchronous class contact.
• Theory was that in order to understand the
  application of literary terms in the analysis of
  fiction, one must be able to practice through
  class conversation
Other factors . . . .
• Ensuring rigidity of testing
   • Time
   • Security
   • Content
   • Format
• Maintaining quality and quantity of written
  research essays as required by established
  college standards and IGETC agreement
  (Intersegmental General Education Transfer Curriculum)
Solutions…

• Classroom time / Synchronous Contact
  • Replaced with weekly chat room meetings
  • Synchronous conversation with students
  • Chat room transcript archived on web site
    for review and retrieval
  • All students required to attend or to read
    archive and respond
Solutions…continued

• Testing Protocol:
  • Establishment of Proctored Testing Center
  • Testing center controls:
     •   Allowed materials
     •   Allotted time
     •   Identification of test-taker
     •   Adapted Testing needs
More Solutions

• Term Papers:
  • Strict adherence to established guidelines
    • Content
    • Length
    • Format
  • Peer editing asynchronously through
    message board
    • Turnitin.com
    • SafeAssign
Solutions…        (almost done)




• Asynchronous Contact
  •   Message board
  •   E-mail
  •   Web page
  •   Archived chat room
  •   Web links
Results

• Comparing ENGL 101 and ENGL 103
  Freshman Composition Courses
• Retention is higher in on-campus
  courses
  • Traditional Avg: 22.5/30
  • Distance Learning Avg: 15.3/30
Results…continued

• Grade range reflects a standard curve
  in traditional courses (frown)
• Traditional Delivery Averages:
  •   A: 18%
  •   B: 36%
  •   C: 24%
  •   D: 22%
More Results…

• Distance education grade range is wider
     (smile)
• Distance Education Grade Averages:
  •   A: 38%
  •   B: 23%
  •   C: 8%
  •   D: 31%
                   FACTORS
(as determined by observation, surveys and interviews)


• Population
   •   Age (older is better)
   •   Experience (older is still better)
   •   Distractions (less is better)
   •   Motivation (more is better)
• Delivery Method
   • Technology (BB, attachments, chat room)
   • Influence of real, live person
What It Means . . . .

• Delivery methods create some different outcomes
  BUT…
• Assessments tools and criteria are the same
• Testing center more uniform than individual
  classroom practices (now being used for traditional
  courses as well)
• Technical Support would help many students
  succeeded in distance education
• The distance education version of freshman
  composition at Citrus College has as much, if not
  more, student learning outcome assessment and
  verification than the on-campus version of the same
  course.
Art 199


Motion Picture Appreciation
Distance Ed vs. Traditional Deliveries
Content Of The Course
• Social/technical factors of film
  production
• Genres
• Aesthetics
• Narrative and non-narrative
  construction
• Film form and history
• Ability to make sophisticated small talk
  at cocktail parties.
Concerns

• Logistical concerns re: films
• Rigor
• Quality and Quantity of education to
  maintain IGETC agreement

• I am the only faculty member who
  teaches this course.
Methods of Instruction
(from course outline)

•   Lecture
•   Lecture/Discussion
•   Independent Study
•   Out of class film viewing
•   Directed study
Assessment

• Discussion
• Large Paper
• Quizzes
  • Taken online
• Tests
  • Taken in testing center under controlled
    conditions until this semester
What it means
• No one flies under the radar
• No one can sleep through a film and
  pass my class
• Testing is controlled
• Criteria are unchanged
• At least the same learning outcomes
  as a traditional course.
Overall Midterm Comparison, Fall
2006


16
14
12
10
8                             Distance Ed
6                             Traditional
4
2
0
     As   Bs   Cs   Ds   Fs
Final Comparison, Fall 2006

14
12
10
8
                              Distance Ed
6
                              Traditional
4
2
0
     As   Bs   Cs   Ds   Fs
CVC Online Success & Retention
                    Online Course Success and Retention

    100%
    80%
    60%                                                           Overall Success
%




    40%                                                           Overall Retention

    20%
     0%
               Spring 2005                        Fall 2005
                               Semester


                                    Spring 2005       Fall 2005


           Overall Success                   58%           56%

           Overall Retention                 81%           79%

           Enrollments                      1939           2066
             CVC Spring 2006 Distance Learning and On Campus Grade Distributions
                        Course             Instr. Method       College         Success   Retention
ACCT-2301 Prin Accounting I                                  On Campus          47%        70%
ACCT-2301-3427                                INET         Distance Learning    50%        75%
BIOL-1408 Biology Non-Sci Maj I                              On Campus          58%        79%
BIOL-1408-3460                                INET         Distance Learning    17%        50%
BIOL-1408-3461                                INET         Distance Learning    53%        89%
BIOL-1408-3462                                INET         Distance Learning    58%        75%
BIOL-1408-3463                                INET         Distance Learning    49%        78%
BIOL-1408-3464                                INET         Distance Learning    80%       100%
BIOL-1409 Biology Non-Sci Maj II                             On Campus          70%        84%
BIOL-1409-3460                                INET         Distance Learning    71%       100%
BIOL-1409-3461                                INET         Distance Learning    56%        78%
BIOL-1409-3462                                INET         Distance Learning    54%        85%
BMGT-1303 Prin of Management                                 On Campus          60%        81%
BMGT-1303-3426                                INET         Distance Learning   100%       100%
BMGT-1303-3427                                INET         Distance Learning    42%        82%
BUSI-1301 Intro to Business                                  On Campus          54%        84%
BUSI-1301-3426                                INET         Distance Learning   100%       100%
BUSI-1301-3427                                INET         Distance Learning    50%        78%
COSC-1300 Computer Literacy                                  On Campus          64%        82%
COSC-1300-3426                                INET         Distance Learning    67%        93%
COSC-1300-3428                                INET         Distance Learning    60%        84%
COSC-1401 Microcomp. Conc./Apps.                             On Campus          77%        89%
COSC-1401-3426                                INET         Distance Learning    82%        88%
COSC-1401-3427                                INET         Distance Learning    44%        80%
COSC-1401-3428                                INET         Distance Learning    39%        78%
COSC-1401-3429                                INET         Distance Learning   100%       100%
                 CVC Spring 2006 Distance Learning and On Campus Grade Distributions

ECON-2301 Prin Economics                                      On Campus         63%    80%
ECON-2301-3426                                INET          Distance Learning   33%    33%
ECON-2301-3427                                INET          Distance Learning   49%    72%
ECON-2302 Prin Economics                                      On Campus         66%    78%
ECON-2302-3426                                INET          Distance Learning   61%    61%
ENGL-1301 Composition I                                       On Campus         58%    78%
ENGL-1301-3420                                INET          Distance Learning   45%    55%
ENGL-1301-3421                                INET          Distance Learning   50%    91%
HIST-1301 History U S                                         On Campus         55%    80%
HIST-1301-3420                                INET          Distance Learning   63%    73%
HIST-1301-3421                                INET          Distance Learning   57%    57%
HIST-1302 History U S                                         On Campus         70%    85%
HIST-1302-3420                                INET          Distance Learning   59%    84%
NUTR-1322 Prin of Nutrition                                   On Campus         71%    81%
NUTR-1322-3426                                INET          Distance Learning   36%    62%
PSYC-2301 Intro Psychology                                    On Campus         62%    83%
PSYC-2301-3420                                INET          Distance Learning   56%    80%
SOCI-1301 Intro Sociology                                     On Campus         68%    84%
SOCI-1301-3420                                INET          Distance Learning   81%    95%
SOCI-1301-3421                                INET          Distance Learning   63%    92%
SPCH-1311 Intro Speech Comm                                   On Campus         71%    85%
SPCH-1311-3420                                INET          Distance Learning   73%    87%
What Makes DE Work? . . .
INFRASTRUCURE!
A viable Distance Learning program
  requires changes to the infrastructure of
  campus to sustain learning outcomes
  mandated by:
• College District
• State Regulations
• Accrediting Commission
• Federal Government—Title V
Infrastructure Changes
Testing Center
• Grew out of need to offer timed, proctored
  testing for DE students to establish controls
  for test taking to insure quality

• Meets regular effective contact requirements
  to authenticate outcomes

• Now serves DE as well as the larger campus
  community--academic makeup testing,
  assessment testing, adapted testing
  accommodation, job applicant testing,
  computer skills competency testing.
Infrastructure—
Supporting Services of SLO’s in DE
• Distance Education Department
• Administrative, Clerical, and Technology
  Support Staff
• Training for Students and New DE Instructors
• Orientation of New Students and New Faculty
• Online Services including registration and
  tutoring
• DE Student Support staff (between technical &
  counseling)
The End?

• DE was implemented long before and is
  keeping pace with if not exceeding SLO
  requirements for traditional courses

• Distance Education has clearly prompted the
  college to reevaluate outcomes, has affected
  how we teach in the traditional classroom,
  and has effected a change in infrastructure to
  meet needs in regards to outcomes.

								
To top