My UW-Madison Online Survey Report - DOC

Document Sample
My UW-Madison Online Survey Report - DOC Powered By Docstoc
					   Online Survey Report
           Spring 2004




Prepared by Jane K. Terpstra, Ed.S.




    Academic Technology Solutions
   Division of Information Technology
    University of Wisconsin-Madison

 Report Completion Date: July 2, 2004
Table of Contents


Background and Methodology                     3
Usage                                          4
My Front Page Tab                              4
Resources Tab                                  5
Academic Tab                                   6
Library Tab                                    7
Campus Life Tab                                8
Financial Tab                                  8
Student Record Tab                             9
Work Record Tab                              10
Customizations and Settings                  10
Usefulness of Possible New Features          11
Overall Usefulness Ratings                   12
Survey Instruments                    Appendix A
Survey Results                        Appendix B




                                                   2
Background and Methodology

Samples of 1,600 students, facstaff, instructors, and advisors were drawn from
the University of Wisconsin-Madison‟s ISIS and LDAP campus databases. In
April 2004 an initial e-mail message invited participation in the online survey.
Three follow-up reminder messages were sent in the weeks that followed. Of
those contacted, 218 students, 178 instructors, 333 facstaff, and 182 advisors
completed the survey, yielding the following response rates: 14% for students,
11% for instructors, 21% for facstaff, and 11% for advisors.

Students completing the survey were fairly evenly distributed among class
standings. Eighteen percent (18%) were freshmen, 16% sophomores, 17%
juniors, 21% seniors, 23% graduate students. Four percent (4% ) reported they
were special students.

Participants tended to serve in multiple roles within the University. Students
reported additional roles as staff (11%), instructor (3%), and advisor (1%).
Instructors included roles as student (51%), faculty (35%), staff (15%), and
advisor (8%). Facstaff indicated roles as staff (79%), faculty (15%), student
(6%), instructor (5%), and advisor (1%). Advisors reported additional roles as
faculty (72%), instructor (16%), staff (15%), student (2%).

Roughly one-third (30%) of survey participants described themselves as power
users of the Internet, with two-thirds (66%) as intermediate users, and very few
(4%) as novices. A majority of survey participants (79%) use My UW-Madison as
their only portal. Others (21%) report also using primarily Yahoo as their
additional portal. Others mentioned were www.wisc.edu, www.Madison.com,
ISIS, Google, America Online, and Hotmail.




                                                                                   3
Usage

My UW-Madison pattern of usage and frequency coincides with the amount of
personalized and updated information available to users. Almost all students
surveyed (99+%) reported using the portal, not surprising since enrollment is
completed via the portal. Yet, of these students, nearly half (46%) reported using
My UW-Madison on a daily basis, with 78% reporting usage at least weekly.
Instructors and advisors reported high usage (92% each). However, their usage
is more intermittent (weekly for 33% of instructors and 48% of advisors, monthly
for 41% of instructors and 30% of advisors). Only a quarter (25%) of instructors
and advisors report using My UW-Madison on a daily basis. Facstaff reported
lower (56%) and intermittent (42% monthly, 39% weekly) usage.


My Front Page Tab

The Front Page Tab provides user access to information and communication
tools. Campus e-mail, WiscMail, was rated as very useful by more than one-third
of survey participants (37%) and as useful by an additional 27%. Students find
WiscMail most beneficial (53% very useful, 28% useful). Advisors and facstaff
rated WiscMail the least useful, with 32% of advisors and 30% of facstaff
reporting not using WiscMail.

Campus calendaring, WiscCal, is underutilized, with a majority of users (70%)
reporting they do not use WiscCal. Of those who used WiscCal, it was deemed
very useful by only 8% of survey participants and as useful by an additional 11%.

Campus communications received mixed ratings. While about half of survey
participants rated Today‟s Events as useful (19% very useful, 29% useful), 42%
of participants reported not reading these. This half and half pattern held true for
Campus Scenes (20% very useful, 27% useful, 42% don‟t use), and Athletics
Headlines (25% very useful, 31% useful, 35% don‟t use). UW-Madison
Headlines fared less well (10% very useful, 18% useful, and 63% don‟t read),
and Announcements were mostly (71%) unread.

Information access also received mixed ratings. Some were viewed as
moderately helpful: Campus Search/Directories (24% very useful, 26% useful),
Weather (15% very useful, 16% useful), and Bookmarks (10% very useful, 17%
useful). Others were rarely used: Stock Watch (85% don‟t use),
Dictionary/Thesaurus (84% don‟t use), Calculator (78% don‟t use), and Maps
(74% don‟t use).

Survey participants entered the following comments or My Front Page:
       •   reported very specific features that they use regularly and view other
           features as unnecessary clutter.



                                                                                    4
      •   reported not discovering the More button leading them to add, delete,
          and/or re-order features.
      •   reported many features are available and used through other means
          (www.wisc.edu, other e-mail and calendar systems)
      •   requested news be updated more often
      •   requested no additional logins to gain access to WiscMail and WiscCal
      •   viewed WiscMail as slow to load, clumsy, not user-friendly, too many
          outages, and missing essential features (text formatting, spell check,
          autofill, better address book, virus detector for attachments, address
          filter, lower case names)
      •   requested ability to compose message in WiscMail when using Safari
      •   reported problems with WiscMail for Mac users
      •   report WiscCal is not widely adopted on campus, nor is it reliable or
          easily exportable

Resources Tab

Features of the Resources Tab were primarily unused. Among the features
available, the online DoIT Tech Store received the highest overall ratings of very
useful for 14% of survey participants and useful for 24%. The online Help Desk
was rated overall as very useful by 16% of these users and as useful for 16%.
DoIT Consolidation Billing was viewed as very useful by 9% of users and as
useful by 14%. However, relatively high percentages of users did not use these
resources (70% for DoIT Consolidation Billing, 64% for Online Help Desk, and
52% for Online DoIT Tech Store). The following features were reportedly not
used by a majority of survey participants (91% for Purchasing System Link, 91%
for Document Query, 90% for Wiscard, 86% for Resource Bookmarks, 85% for
WISDM Financial Accounting, 80% for WISC Online Software Catalog, 80% for
Data Digest, and 72% for Electronic Shelf).

Survey participants using Resource Tab features provided the following as
comments:
      •   reported typically accessing these resources from www.wisc.edu, thus
          no advantage to accessing these services through the portal.
      •   described as a „random collection of links‟
      •   requested the search features of the Online Tech Store and Electronic
          Shelf be improved
      •   requested access to the Electronic Shelf through the portal from off-
          campus.



                                                                                     5
      •   reported problems ordering online from the DoIT Tech Store and
          Electronic Shelf that wasted a lot of their time to straighten out
      •   reported too many clicks in the Purchasing System
      •   requested Consolidated Billing have a separate invoice for each funding
          string


Academic Tab

Survey participants reported that they generally did not use academic information
that is also available on www.wisc.edu: 74% for the Graduate School Catalog,
73% for ASM Course Evaluations, 70% for the Undergraduate Departments &
Programs, 68% for Academic Bookmarks, and 60% for the Academic Calendar.
Exceptions were the Undergraduate Catalog, rated very useful by 26% of survey
users and as useful by 23%; the Timetable, rated very useful by 25% of surveyed
users and as useful by 21%; and General Education Requirements, rated very
useful by 26% of users and as useful by 15%.

Of the features available only to students, the Course List was rated most
beneficial (44% very useful, 33% useful), followed by Academic Advisor(s) (12%
very useful, 22% useful). Other features were reported as not used (86% for
Global Gateway, 84% for Educational Orienteering, 83% for GUTS, 79% for
Advising Toolkit, and 73% for the Writing Center).

Among features available to advisors, the List of Advisees offering access to
advisee records proved the most beneficial feature (34% very useful, 30%
useful), along with Advisee Lookup (24% very useful, 24% useful). Features that
advisors used least within the portal were Course Grid (73%) and Student
Services for Advisees (53%).

Instructor survey participants rated Class Rosters as most beneficial (34% very
useful, 32% useful), followed by Course Services (15% very useful, 29% useful).
The Library Reserves by Department feature was generally unused by instructors
(74%).

Comments entered for the Academic Tab included:
      •   Students
           o reported not being aware that so many features were available
           o requested the Academic Calendar be in a standard calendar grid
              display
           o reported problems with no or incorrect advisors listed
      •   Instructors
           o requested more integration of online course resources (class lists,
               reserves) with Learn @UW


                                                                                   6
          o reported difficulty loading (slow) and exporting class rosters to
            Excel
          o requested improvements in library reserves to allow reserves by
            course section and to be less cumbersome to access
          o reported that ASM Course Evaluations were incomplete
      •   Advisors
          o reported using www.wisc.edu to access many of the features
          o liked searching for advisees by name
          o requested broader access to Advisee Lookup for academic staff
          o reported many errors in their List of Advisees
          o requested advisee e-mail lists
          o reported ASM Evaluations are useless
          o requested DARS access for their advisees
          o requested names on course grids and the ability to display multiple
             grids for advisees
          o requested a listing of general ed requirement codes in the student
             academic record (courses classified as E, H, L, Z, S)

      •   Facstaff
          o reported too much information already available on www.wisc.edu
          o reported the timetable difficult to navigate and requested it be
             printable
          o requested links to more course descriptions
          o liked their students having access to e-Reserves from the
             Academic Tab
          o requested improvements in the Academic Calendar format and
             content


Library Tab

Survey participants were fairly evenly split between finding the Library Tab
features beneficial and not using them. While the Library Resources feature was
rated as very useful by 26% of survey respondents and as useful by 23%, an
average of 45% reported not using this feature. Campus Library Information was
viewed as very useful by 23% of participants and as useful by 25%, yet 47%
reported not using this resource. Similarly, Library Services were rated as very
useful by 26% of users and as useful by 22%; however, 47% of the survey
participants reported not using them. Library Bookmarks were also generally not
used (74%).

Course e-Reserves were rated as very useful by 33% of students and 26% of
instructors responding to the survey and were rated as useful by 28% of students
and 20% of instructors.




                                                                                7
Survey participants provided the following comments:
      •   reported no advantage of accessing Library Web sites through the
          portal as they are merely links to the same information available more
          directly – just redundant links
      •   requested library hours and maps be more readily available
      •   reported e-reserves are great but require too many steps to reach
          materials
      •   requested a longer timeout limit for materials accessed through Library
          Express
      •   requested more personalized access to journals and library resources
          – by field of study
      •   requested faster access to search databases


Campus Life Tab

Users responding to this survey benefited from some features within the Campus
Life Tab but minimally used others. The Union Events feature was viewed of
benefit by about one-third of respondents (11% very useful, 20% useful). The
Campus Map was rated very useful by an average of 12% of respondents and as
useful by 19%. Ten percent (10%) of students rated Off-campus Housing
Services as very useful, with 9% rating this as useful. Overall, a majority of
respondents did not use the following features: ESCAPE Events Calendar
(100%), McBurney Disability Resources (90%), Bookmarks (88%), Student
Computing Guide (86%), Lost and Found (84%), Daily Cardinal Online (83%),
and Student Organization Office Directory (78%).

Survey participants offered these comments:
      •   requested Union Events be updated earlier in the day
      •   reported this a „pointless tab‟, just lists of things
      •   requested a more reliable and detailed Campus Map
      •   requested a link to the Campus Information and Visitor Center


Financial Tab

Student respondents rated as most beneficial the Account Summary (34% very
useful, 30% useful) and Financial Aid (23% very useful, 18% useful). The wa
Features moderately beneficial to these students were Office of Student
Financial Services (17% very useful, 16% useful) and Wiscard (19% very useful,
12% useful. However, more than half (55% and 60%, respectively) did not use


                                                                                   8
these features. A majority of student participants reported they did not use
Graduate Student Fellowships (87%), Residence Hall Billing (75%), or Financial
Bookmarks (73%).


Students submitting surveys contributed the following comments:
      •   requested the ability to transfer money from campus cash to housing
          account online
      •   liked the account summary
      •   requested online tuition payment


Student Record Tab

Students responding to this survey rated many of the Student Record Tab
features as beneficial. These included List Courses/Grades (61% very useful,
31% useful), Course Grid (59% very useful, 32% useful), Grade Report (56%
very useful, 28% useful), Request Student Record (56% very useful, 28%
useful), and Enrollment (54% very useful, 30% useful).

Of moderate benefit were the Student Record (48% very useful, 34% useful),
Preferred E-mail Address (42% very useful, 33% useful), DARS Report Request
(39% very useful, 23% useful), Enrollment Verification (39% very useful, 27%
useful), Grade Report Request (38% very useful, 21% useful), Contact
Information (37% very useful, 29% useful), Restrict Third Party Mail (37% very
useful, 29% useful), Fines and Holds (29% very useful, 27% useful).

More than half of the students reported not using PIN Change (57%) or Student
Record Bookmarks (79%).

Student survey respondents provided these related comments:
      •   requested Course Grid that would print on one page
      •   requested faster access to DARS Reports (without requests)
      •   requested access to final exam grades as well as final course grades
      •   reported problems getting Preferred E-mail Address to work
      •   requested Grade Reports that would print on one page
      •   reported that GPAs for Law School students are misleading and
          requested that the Law School scale be incorporated for their GPAs
      •   reported that their Course List contained grades faster than the Grade
          Reports



                                                                                   9
      •   reported difficulty accessing and understanding Enrollment
      •   requested ability to print an unofficial transcript



Work Record Tab

Work Record is a relatively new tab, and its features have gone largely
undiscovered. An average of 59% of all respondents reported not using the
Earnings Statement. Of those who did, 18% rated this feature very useful and
18% as useful. Eighty-seven percent (87%) reported not using the Professional
Development feature, and 83% did not use Work Record Bookmarks.

Of the students responding, 91% reported not using the NRAware Nonresident
Tax Preparation. Two percent (2%) of the 9% of students who used this feature
rated it as very useful and 2% as useful.

Comments entered by survey respondents are as follows:
      •   reported being unaware these resources were available and requested
          advertising to let employees know
      •   reported the Earnings Statement an “awesome idea”
      •   requested an e-mail notice when a new pay stub is available
      •   reported that requesting a particular pay stub resulted in a new
          browser window that covered the entire screen
      •   requested access to old earning statements
      •   reported no understanding of the “Professional Development thing”
          and requested more explanation up front


Customization and Settings

Unfortunately, more than half of survey respondents reported that they had not
discovered or used the More for Add/Delete (51%) feature or the Rearrange
Content (59%) feature.

In their comments, respondents …
      •   reported not knowing that these features existed
      •   reported not realizing there were more features that could be added
      •   reported not realizing that they could rearrange or delete features they
          did not use often or ever.



                                                                                 10
Usefulness of Possible New Features

Several possible new features were proposed to survey participants. Of these,
respondents rated the Searchable Online Course Guide as most beneficial (73%
very useful by students, 16% useful; 55% very useful overall, 21% useful overall).
This feature was followed by Instant DARS Report (57% very useful by students,
17% useful; 55% very useful overall, 13% useful), Website for Every Timetable
Course (48% very useful by students, 35% useful; 41% very useful overall, 30%
useful), and DARS What-if Reports (47% very useful by students, 26% useful;
44% very useful overall, 19% useful). In addition, 59% of student respondents
rated Textbook Exchange as very useful, with 29% rating this feature as useful.

Of moderate interest to respondents were Online Ordering/Payment/Delivery of
Textbooks and Copied Packets (39% very useful, 22% useful), Pay Tuition by
Credit Card (39% very useful, 21% useful), Personalized Library Resources
(28% very useful, 25% useful), GPA Calculator (25% very useful, 19% useful)
and Address Book (20% very useful, 21% useful).

The majority of respondents reported they would not use Online Discussion
(79%), Ride Board (73%), Personalization of Look (71%), Memo (67%),
Classified Ads (65%), or Available Distance Education Courses (58%).

Participant comments regarding possible new features follow:
      •   reported many of the future features would be very helpful
      •   requested links to Class Connections and Badger Herald
      •   requested improved Timetable and adding Textbook Exchange
      •   requested “weeding out” unnecessary features first
      •   reported a Website for every course seemed out of the purview of DoIT
      •   reported that distance learning would be “fantastic” for teacher
          credentialing
      •   reported too much duplication between the portal and www.wisc.edu
          and requested better integration
      •   requested improving the current portal before adding more features
          (single login, clumsy e-mail)
      •   viewed WiscMail as “hopelessly archaic and primitive”
      •   requested adding Pine e-mail (faster, can view e-mail with
          attachments, images, etc. without concern about downloading viruses)




                                                                               11
      •   reported problems getting to all features for all their roles, having to
          choose between access to Enrollment as a student and Class Rosters
          as instructor for classes taught
      •   requested paying SEG fees online
      •   viewed Classified Ads as “spam”
      •   requested improvements in WiscCal to better support scheduling
          advising appointments
      •   requested a way to make one tab with all features desired
      •   requested synchronization with PDAs


Overall Usefulness Ratings

Despite wide variance in the usefulness ratings of features, reported problems,
and requested changes among the 2004 survey participants, the My UW-
Madison portal is widely accepted and would be recommended to others by 93%
of students, 88% of instructors and advisors, and 87% of facstaff. My UW-
Madison received high overall usefulness ratings (29% very useful, 56% useful)
and was rated as poor by only 1.5% of all survey respondents.

Specific survey results, by user group, follow in Appendix A.




                                                                                12
       Appendix A

     My UW-Madison
Online Survey Instruments
       Spring 2004




                            13
            Appendix B

          My UW-Madison
Online Survey Results By User Group
             Spring 2004




                                      14