workload by panniuniu

VIEWS: 43 PAGES: 4

									                                                                                            Page 1 of 4

Item 1 (the “Workload” Item) On the New Faculty Course Questionnaire (FCQ)

Perry Sailor
University of Colorado-Boulder
Planning, Budget, and Analysis
Jan. 2009, revised July 2009


Introduction/Summary
The new FCQ form was introduced in the fall of 2006
(http://www.colorado.edu/pba/fcq/form/formBD.pdf). The previous form’s question about
workload (http://www.colorado.edu/pba/fcq/form/paperform.pdf) asked the respondent to
rate the course’s workload relative to the credit given for the course, on a 1-9 scale, with
1 labeled “too light,” 9 labeled “too heavy,” and 5, the midpoint, labeled “OK.”

The new form’s workload item does not ask the respondent to judge the appropriateness
of the workload, but rather simply to estimate the number of hours per week spent on the
course. The actual wording is “Estimate the number of hours per week you have spent
on this course for all course-related work including attending classes, labs, recitations,
readings, reviewing notes, writing papers, etc.” (Emphasis in the actual item as written.)
There are six response options: “0-3,” “4-6,” “7-9,” “10-12,” “13-15,” and “16+.”

This seems straightforward enough, but we decided to look at some data to see if the
newly worded item is “behaving” as expected. Based on our analyses (reported in detail
below), here are some guidelines for interpreting results on the reconstituted item:

Results: Guidelines for interpretation of Workload item results
We believe the implications of our studies for interpreting results are as follows:
    Responses probably should not be interpreted literally, only comparatively. We
       suspect results underestimate actual time spent.
    Courses should be compared only to other courses carrying the same credit load
       – 3-credit courses with 3-credit courses, 5 with 5, etc.
    With respect to lab/recitation sections, it is difficult to say anything other than that
       results should be interpreted very cautiously. Lab/recitation sections probably
       should not be compared to lecture/seminar sections. We suspect that workload
       ratings of lectures that have labs and/or recitations underestimate the total
       workload of the course, and we’re almost certain that ratings taken from the
       lab/recitation sections do so.
    Because summer sessions vary in length, workload results from summer
       sessions should only be compared with other summer sessions of roughly the
       same length. Similarly, Maymester terms should be compared only to other
       Maymester terms, and full fall or spring terms with other full terms.

Despite the above, we do not recommend any changes to the item at this time. We’re
not sure what could be done to improve it. The wording could be changed to specify
only time spent outside class, which would eliminate the problem of some students
apparently forgetting to include time spent in class in their estimations. Similarly,
instructing students to consider only time spent connected to the lecture portion alone of
a lecture/lab or lecture/recitation course would appear to remove the problem of some
students failing to include all time spent on the course. But the larger problem appears



PBA, PS, D:\Docstoc\Working\pdf\6f610030-a65a-4cd4-b412-49c6b39e9c0f.doc_10/26/2011_10:01:33 AM
                                                                                                        Page 2 of 4

to be some students not following instructions, which changing the specifics of those
instructions would do nothing to solve.


Details of analyses
Our basic approach was to ask, “If students are answering the item as instructed, what
response patterns should we see?” For example, one would expect that course
workloads, in terms of hours per week, would be heavier in the relatively brief summer
terms compared to the much longer fall term, and heaviest of all in the very short (18
days in 2007) Maymester term. This was indeed the case:


Summer and fall 2007, mean section averages (lecture and seminar sections only).

                       Days in Session Mean Workload
                  (incl. weekends/holidays) Response
Summer
Session A                         33                         3.9*
Session B                         32                         4.0
Session C                         54                         3.9
Session D                         68                         4.8
Maymester                         19                         5.1

Fall
(Full term)                  116                            2.6
*NOTE: These and all averages in this report are average responses on the 1-6 scale, not average hours/week.     On this
scale 1 = 0-3 hours, 2 = 4-6 hours, 3 = 7-9 hours, 4 = 10-12 hours, 5 = 13-15 hours, and 6 = 16 or more hours.


The correlation between the number of days in the term for the summer sessions, and
the mean workload response, was -.13. This is small but statistically significant.

Likewise, there was a clear relationship between the average rating on the workload
item and the number of credit hours assigned to the course.


Fall 2007, lecture and seminar sections only (no labs or recitations).

                         Mean Workload
Credit hours               Response
      1                      1.9
      2                      2.3
      3                      2.5
      4                      3.1
      5                      3.4
      6                      4.0




The correlation between credit hours and the mean workload response was .43, which is
both sizable and statistically significant. So just as we would expect, students report


PBA, PS, D:\Docstoc\Working\pdf\6f610030-a65a-4cd4-b412-49c6b39e9c0f.doc_10/26/2011_10:01:33 AM
                                                                                            Page 3 of 4

spending more hours per week on courses of shorter duration, and on courses with
more credit hours attached. The clear implication of this is that workload ratings should
only be compared to workload ratings of courses carrying the same credit load, and that
ratings of summer courses should only be compared within summer session of roughly
the same length, not those of much longer or shorter duration and not to full fall or spring
terms.

Next, we looked at the issue of interpretation of the workload item in lecture sections vs.
lab and recitation sections.

Here we have reason to believe many students are not interpreting the item correctly.
They don’t seem to be following the instruction to include all lab and recitation work in
their ratings. Logically, if a student fills out a form for, say, PSYC 1001, in the lecture
section, and then also fills out another form for the recitation section, if the student
follows the instructions as written, the workload rating should be the same in both
sections, since the item says to include all course-related work.

We tested this hypothesis by building a dataset from fall 2007 FCQ data that included all
undergraduate courses that had at least one lecture section and at least one recitation
section. For each lecture section, we calculated the average workload rating. We then
calculated the average workload rating for the recitation sections paired with that lecture
section. For example, there was one lecture section of ANTH 1100 in fall 2007, and 5
recitation sections. The average workload rating in the lecture section was 1.70; the
average across the 5 recitations was 1.40. We did this for every undergraduate course
that had a lecture section and at least one recitation section. And we did the same for
courses with lab sections. (We could not match on the level of individual students, of
course, since the forms do not carry identifying information.)

Again, if students were actually considering all coursework, including lectures,
recitations, labs, etc., in their workload ratings, then the average ratings of lecture and
paired recitation/lab sections should have been about the same. In fact, we found that
the lecture sections had higher average workload ratings than their paired recitation
sections 93% of the time, higher than their paired lab sections 71% of the time, and that
there was a sizable gap in average ratings.


Section           Average
Type              Rating
Lecture           2.55
Recitation        2.25

Lecture           3.16
Lab               2.98


Results were nearly identical whether section mean or individual form was the unit of
analysis.

Further, even when looking at lecture sections alone, we have reason to believe some
students are misinterpreting the item. For example, 14% of forms from 3-credit courses,
and 6% from courses worth more than 3 credits, were marked “0-3 hours per week.”


PBA, PS, D:\Docstoc\Working\pdf\6f610030-a65a-4cd4-b412-49c6b39e9c0f.doc_10/26/2011_10:01:33 AM
                                                                                            Page 4 of 4

Since these courses require 3 (or more) hours per week of class time alone, marking
that response means either a student is spending no time outside class doing
coursework, or is not attending class regularly. While either of these alternative
explanations is possible, it’s also possible that at least a significant minority of students
is not accurately estimating time use, or not answering according to instructions to
consider all time spent on the course.




PBA, PS, D:\Docstoc\Working\pdf\6f610030-a65a-4cd4-b412-49c6b39e9c0f.doc_10/26/2011_10:01:33 AM

								
To top