Digital Chemistry at UC Berkeley Challenges

Document Sample
Digital Chemistry at UC Berkeley Challenges Powered By Docstoc
					19.10.00   CSHE   1
Digital Chemistry at UC Berkeley

                    Digital Chemistry 1A:
           An Economic and Pedagogical Analysis
             of Technology Enhancements in a
            Large Lecture Course at UC Berkeley

                   Principal Investigator:
                      Diane Harley, Ph.D.,
             Berkeley Multimedia Research Center
             Center for Studies in Higher Education
               University of California, Berkeley
19.10.00                      CSHE                    2
Digital Chemistry at UC Berkeley

Two Year Project
Partially funded by the Andrew W. Mellon Foundation
Research Partners: Implementation and Evaluation

 Center for Studies in Higher Education (CSHE)

 Berkeley Multimedia Research Center (BMRC)

 College of Chemistry (UCB)

19.10.00                   CSHE                       3
 Digital Chemistry at UC Berkeley

Today’s talk:
 Describe pressures we are facing in California in general,
  and at UCB in particular.
 Give you an overview of one experiment at UCB that has
  potential to address some of those pressures
 Discuss the evaluation research we are conducting to
  determine effectiveness and cost of technology
 Highlight some of the challenges we face in implementation,
  evaluation, and scaling
 Demonstrate the technology enhancements (if we have
  time and the technology works!)
 19.10.00                   CSHE                         4
 Digital Chemistry at UC Berkeley

California: Three-tiered Public Higher Education System

 California Community Colleges (CCC) “Open Door Public
 California State University System (CSU) “Polytechnics”

 University of California (UC) 10 campuses
  Elite, Public, Research
  Missions:Teaching, Research, and Service
  Highly selective, Grants Ph.D.
 (Plus whole system of Private Colleges and Universities)
19.10.00                     CSHE                            5
  Digital Chemistry at UC Berkeley

 More than two million new full-time students expected
  to enroll in US public and private colleges and universities
  by 2010, referred to as Tidal Wave II (CPEC, 2000).

 The University of California (UC) ten-campus system faces
  an increased enrollment of almost 63,000 full-time
  students, (a 43 percent increase).

 UC Berkeley is being asked how it could absorb an
  additional 4,000 students by 2010. (An annual growth
  rate of 1.1 percent over the next 10 years (UC News and
  Communications, 2000).
19.10.00                     CSHE                          6
  Digital Chemistry at UC Berkeley

Options for accommodating influx of new UC students

 How can we serve more students, more effectively,
  and economically, i.e., without increasing teaching and
  support staff in large lecture courses?
  Suggestions include:

   Offering classes during the summer,
   Expanding regular enrollments during fall and spring
    semesters, and
   Using technology to expand on- and off-campus
    learning opportunities.
19.10.00                   CSHE                            7
 Digital Chemistry at UC Berkeley

Background: UC’s Use of Technology Enhanced Education
 CSHE is tracking developments in on-line distributed education
  taking place at UC Berkeley, and throughout the University
  of California (UC) ten-campus system.

 UC efforts have been characterized as bottom-up efforts on
  the core campuses, and by more aggressive experimental and
  scalable work in the UC Extension units.
 The last year has seen a shift in institutional and faculty
  attitudes about ICTs at UC. A number of large-scale
  experiments are now taking place throughout the core UC
  campus system.
  19.10.00                    CSHE                              8
  Digital Chemistry at UC Berkeley

  Large scale experiments include those in:
   College of Engineering
   College of Chemistry
   Haas School of Business

  They reflect UC’s technological response to the
    urgent pressures of :
   Increasing access
   Maintaining quality
   Containing cost
19.10.00                  CSHE                      9
    Digital Chemistry at UC Berkeley

These experiments offer case studies to analyze:
   The cost effectiveness of technology enhancements
   Patterns of faculty decision making and work,
 The role of core campus faculty and staff in continuing
 The relationship between private sector courseware
  developers and the university,
 Course ownership and intellectual property, and
 ICTs potential for altering and reshaping existing academic
  environments and of creating new ones.
19.10.00                      CSHE                          10
 Digital Chemistry at UC Berkeley

 The Chem1A evaluation project will describe the
  challenges of deploying, and evaluating the effectiveness
  and scalability, of a large-scale instructional technology
  experiment taking place at UC Berkeley.

 Approximately 2,100 students take Chemistry 1A each

 Over 100 teaching and support staff are required to
  teach the course.

 Salaries are the greatest expense in teaching the course.
  19.10.00                    CSHE                             11
  Digital Chemistry at UC Berkeley

The College of Chemistry would like to specifically:

 Reduce the number of faculty teaching during the fall
  and spring semesters in order to have faculty available to
  teach in the summer session.

 Reduce the number of Graduate Student Instructors
  (GSIs) teaching the course by one-third, to free up GSIs
  for the summer session.

 Reduce the student and GSI time spent in lab by 25%,
  thus increasing the use of lab rooms from two sections per
  day to three sections per day.
19.10.00                   CSHE                         12
  Digital Chemistry at UC Berkeley

To save costs and staff time, the College of Chemistry
has a technology strategy that includes:

      Conversion of the lecture chalk board content to
       text-based PowerPoint slides
      Broadcast of video lectures, with synchronized and
       indexed slides, over the Internet for on-demand replay
           BMRC currently webcasts 13 lecture courses
      Deployment of on-line quizzes and pre-laboratory
19.10.00                       CSHE                        13
19.10.00   CSHE   14
19.10.00   CSHE   15
19.10.00   CSHE   16
19.10.00   CSHE   17
19.10.00   CSHE   18
19.10.00   CSHE   19
19.10.00   CSHE   20
19.10.00   CSHE   21
19.10.00   CSHE   22
   Digital Chemistry at UC Berkeley


 Addressing specific questions related to the use of on-line
  lecture and laboratory material, and their potential to free
  up teaching staff time and/or serve more students off-site.

 Primary goals: determine if the utilization of on-line
  teaching materials results in significant restructuring of staff
  time in laboratories, and if laboratory facilities can be used
  by more students.

 19.10.00                     CSHE                            23
  Digital Chemistry at UC Berkeley

   Research Questions: Laboratories
 Will on-line prelab work and quizzes reduce the time GSIs
  spend on grading and administering quizzes?
 Will availability of on-line resources reduce faculty and GSI
  preparation time in subsequent semesters?
 Can laboratory sections be reduced from four to three hours
  if students complete prelaboratory exercises/quizzes on-line?
 Will completion of on-line prelab work improve the quality of
  student conceptual understanding of chemistry?
 Will the convenience of access to on-line materials improve
  student perceptions of the overall quality of the course?

19.10.00                      CSHE                          24
  Digital Chemistry at UC Berkeley

Research Questions: Lectures

 Will the availability of on-line lecture resources result in a
  different use of faculty and GSI time during lecture periods
  (e.g., different ratios of Q&A, demonstrations, discussions)?

 Will access to on-line lectures reduce attendance at lectures,
  thus allowing more students to be enrolled in the course?

 Does the ability to review lectures on-demand, and by topic,
  result in improved student understanding of the material and
  perceptions of the overall quality of the course?
19.10.00                      CSHE                           25
Digital Chemistry at UC Berkeley

 Comparisons between treatment groups (access to
  technology enhancements) and control groups (no access
  to technology enhancements).
 Data collected on cost effectiveness and
  pedagogical effectiveness (e.g., quantitative and qualitative
   data on faculty and staff teaching and preparation activities, student
   performance, and student access of, and attitudes about, on-line
   laboratory and lecture material).

 Secondary goal: conduct a non-experimental
  evaluation on the integration and use of indexed on-
  line lectures.
19.10.00                          CSHE                                 26
  Digital Chemistry at UC Berkeley

The economic evaluation involves four activities:

      Specification of the archetypal approach to teaching
       in the two contexts (traditional UC classroom,
       technology enhanced UC classroom)
      Compilation of data on the costs of teaching in each
       of the two contexts

      Identification of relevant measures of output and the
       compilation of related data

      Analysis of costs
19.10.00                     CSHE                             27
    Digital Chemistry at UC Berkeley

The pedagogical evaluation will measure:
 Student learning outcomes as measured by performance on
  course and lab quizzes and exams, laboratory reports, and a
  “carry-forward” experiment.

 Student attitudes regarding the course as determined by
  analysis of pre- and post- survey and focus group data.

 Course completion and retention data as measured by data
  collected on course completion and attrition.

   Additional data: faculty attitudes regarding development and
    adoption, institutional decision making, and adoption of the
    course materials at other institutions, including high schools.
19.10.00                       CSHE                           28
Digital Chemistry at UC Berkeley

Challenges: Coordination
   An experiment of this scope and complexity
   requires significant technical resources and cross-
   campus collaborations.

 Mellon funds are primarily for evaluation activities at
 Berkeley Multimedia Research Center (BMRC) and the
  College of Chemistry are providing technical and
  disciplinary expertise and execution.
 Additional technical support provided by other campus
  support units (SIMS, OMS, and ITP).
 19.10.00                     CSHE                          29
Digital Chemistry at UC Berkeley

Challenges: Implementation
The course is a technological chimera:

 BMRC-- Video Streaming and Lecture Browser
  open source, part of an on-going research project,
  highly skilled technical and scholarly staff involved

 Web-CT--Course Management and Interactive Quizzes
  commercial Learning Management System (LMS)
  “beta” version with lots of bugs
  supported by central campus servers and tech staff

19.10.00                     CSHE                         30
Digital Chemistry at UC Berkeley

Challenges: Evaluation

 Controlled Experiment
  Student consents
  Problems in quiz delivery
  Working within the established structures of the course

 Activity Based Costing Method
  Details of how staff spend time
  Determining costs for diffuse technical infrastructure
19.10.00                    CSHE                            31
 Digital Chemistry at UC Berkeley

Challenges: Scaling
 Will faculty in subsequent semesters of Chemistry 1A at UC
  Berkeley adopt these materials? Faculty at other UC

 Will faculty and teachers at universities and high schools
  access the information to enhance their Chemistry offerings?
  (e.g., to train students and Advanced Placement chemistry
 Will Digital Chemistry1A be distributed by a private publisher,
  or other entity, thus generating revenue for the faculty
  member and the university?
  19.10.00                    CSHE                           32
Digital Chemistry at UC Berkeley

              Digital Chemistry Website

  Berkeley Internet Broadcast System(BIBS)

                  Diane Harley, Ph.D.

19.10.00                    CSHE                          33
19.10.00   CSHE   34

Shared By: