Use of web-based resources in Engineering_ IT _ Architecture

Document Sample
Use of web-based resources in Engineering_ IT _ Architecture Powered By Docstoc
					Use of web-based resources in Engineering, IT &
Architecture undergraduate teaching, Semester 2 2007
Executive summary

This report presents findings of the Semester 2, 2007 review of web resource use in undergraduate
teaching in the Faculties of Engineering and Information Technologies and of Architecture, Design and
Planning. The review was conducted to assist in identifying:
     1.   The extent and role of web resource use in undergraduate units currently taught by
          Engineering, IT and Architecture faculties and schools.
     2.   Staff support needs related to use of web-based resources in teaching.
     3.   Priorities for improving support for academic staff in use of web-based resources.
The review was based on a stocktaking survey of course related websites in use during 2007, combined
with an interview survey of relevant staff. The sites covered include those inside and outside the central
LMS (formerly known as WebCT now Blackboard). The staff survey gathered input from staff
involved in teaching of undergraduate units and administration of supporting web resources.
The review has been conducted in a context where the capabilities and decision-making related to
support for use of ICTs in teaching are split across a number of different agencies and portfolios at
central and faculty level. With neither the central LMS staff (Sydney eLearning) nor faculty-based
services able to cover the varied ICT needs of Engineering, IT and Architecture courses, ICT support in
these teaching areas relies heavily on DIY approaches and the personal capabilities and initiative of
individual staff members. As specialists in the design and building of large-scale systems and
environments, engineers, information technologists and architects are well placed to appreciate the
implications (and ironies) of the fragmentation, opacity and lack of consistent standards in their own
working infrastructure.
The various agents with actual or potential role in support for use of ICTs in teaching include at central
level Sydney eLearning, central IT and web services, the Institute for Teaching and Learning and the
Library. At faculty and school level, there are various IT support units, plus Teaching and Learning
committees, eLearning Faculty Representatives, and a sub- dean for ICT. Formal responsibility for
management and quality assurance of information resources used in teaching currently lies with the
faculty and the dean, as does responsibility for management and quality assurance of teaching globally,
under existing Academic Board Resolutions (The University of Sydney, 2001). However, where
teaching practice enters the digital domain, the resources necessary for effective management and
quality assurance do not lie with the faculty to the same extent as with central ICT service providers.
Current service arrangements assume that the faculty will have the means of effectively managing and
supervising use of ICTs within courses and units of study, while at the same time resources of real ICT
management and quality control are concentrated in central 'silos' for most practical purposes. How this
contradiction at the heart of university ICT governance ultimately plays out is not for this report to
resolve. The recommendations aim to assist in the development of ICT roles and management
structures at faculty level by defining the most pressing management tasks to be assumed overall,
without going deeply into who precisely may or may not assume these tasks, or how they might do so.
The recommendations are aimed at the general 'what' that needs to be accomplished, rather than
detailed 'how'. The critical requirement at this point is not for detailed plans of execution but some sort
of matching up of responsibilities and capabilities on the specific tasks to be done. Details of how each
task is to proceed are best left to a stage where there are people to undertake them.
The key ICT support tasks are of two sorts. The first three are concerned with ICT standards in
teaching and learning: standards of resource availability, standards of content quality, and standards of
support for academic staff users. The next four recommendations are concerned with development of
essential support services: basic access support, assessment information management, ICT support
coordination and ICT asset management and new resource development.
Recommendation 1. Access to web-resources should be maintained on a consistent basis for all units of
study through common list pages for sites used in each faculty/school and through systematic checking
of sites available against Units of Study in session every semester. Remedial assistance for cases where
sites are not available would be part of the checking process.
Recommendation 2. Content of Unit of Study websites should be maintained on a consistent basis
through systematic checking each semester to ensure that the web resources for each unit of study
include at minimum a well organised, clearly labelled set of basic course documents including unit
outline, assessment details and list of topic materials consistent with the scope of unit. Remedial
assistance would again be provided as part of the checking process.
Recommendation 3. Minimum standards of support for academic staff in the use of ICT across the six
faculty/schools should be negotiated with the two main central providers, the university’s ICT service
and Sydney eLearning. Agreed standards should be specified in Service Level Agreement with support
providers, as is normal IT practice in regulating service standards. Agreed standards for staff support
should be at a level commensurate with the minimum standards website use expected under the
previous two recommendations.
Recommendation 4. A pro-active facilitation program needs to be undertaken directed towards
academic staff having difficulty in making use of the university's web-resources as well as those who
have no access at all. The ‘Teaching Websites Made Easy’ program should aim to streamline access
procedures for teaching websites and extend participation in the use of web resources by actively
seeking out staff who experience difficulties in accessing the resources that the university and the
faculty make available.
Recommendation 5. Focus for technical improvement in central university web systems for teaching
and learning should be first of all on achieving integrated management of assessment information. This
is currently split between student administration systems, the central LMS and local school and faculty
systems. Having student IDs on the ‘Gradebook’ screen of the central LMS would be a good start, but
what is really required is a fully integrated system dealing with assessment information management as
a whole, including pedagogical alignment of assessment tasks, processing of student marks, incident
handling for plagiarism and special consideration and long-term tracking of student progress.
Recommendation 6. A full-time ICT in T&L coordinator position should be created on a two year
project grant basis with the task of developing longer term systems and capabilities in the management
of ICT use in teaching & learning.
Recommendation 7. Support for development of innovative materials and approaches in teaching
should be targeted in the first instance towards systematic evaluation of innovative materials or
approaches that already exist from a perspective of wider application and of cases where innovative
approaches or materials need to be applied.
Details of each recommendation are set out in the final section of this report.




Use of web-based resources in EITA undergraduate teaching                                               2 of 32
Introduction

Background
The survey of Semester 2, 2007 is the first attempt at mapping actual website provision in Engineering,
IT and Architecture courses, and also the first attempt at identifying priority requirements for support
of academic staff using web resources there. Current statistical reporting on web-based resource use in
the university covers use of LMS-based websites only with no recognition for the extensive role of
non-LMS sites that has always been a distinctive feature of web use in both faculties. The eLearning
Audit of 2004 in the former College of Sciences and Technology (The University of Sydney, 2005)
provides figures for web use generally, but from an academic staff development perspective only,
giving proportions of staff making use of websites and various website features in their teaching, but
no numbers for actual websites themselves, or for the different web platforms, LMS and non-LMS. The
2004 figures are also difficult to translate into current terms for comparison purposes, with breakdown
based on a faculty structure that no longer exists.
Student experience and perceptions of web resources use are part of the background against which the
survey was conducted, though outside the main scope. Recent surveys of student responses to ICTs
here and internationally emphasise the increasing complexity of the ways in which students relate to
technology and the need for approaches that cater to diverse student backgrounds rather than expecting
them to follow a single stereotyped mould of the 'net generation' type (Kennedy, Judd, Churchward,
Gray, & Krause, 2008) (Caruso & Kvavik, 2005). What common expectations there are pull in
conflicting directions. The expectations that students freshly graduated from high school bring in
relation to overall teaching are firmly traditional, as academic staff well know. This traditionalism
carries over into the role that technology is expected to play: as an adjunct to traditional face to face
learning events in lectures and tutorials, not as any sort of comparable replacement for traditional forms
(Scott, 2006)(JISC, 2007). Expectations regarding technology access are on another level, however,
with a more contemporary edge. An overwhelming majority of commencing Australia first year
students, over 80%, now see computers, course portals, web-based information seeking tools and
mobile messaging as wanted learning resources, according the most recent survey on the subject
(Kennedy et al, 2008). Web-based instant messaging and audio downloads were also identified as tools
wanted for university by a majority of students, while mobile web services, blogs and personal
websites were wanted by substantial minorities. The common preference seems to be for a learning
experience that is both traditional and digital in an open-ended variety of ways, not dominated by
technology but where technology is nonetheless a pervasive presence. How far various student
expectations should be addressed (or possibly challenged) is primarily a decision for those with
teaching responsibilities and not the business of this report to predetermine. The focus here is rather on
resources available to academic staff in making these decisions. What sort of resources can be expected
by academic staff in addressing the increasingly complex expectations of students?
The main limitation of the survey is its focus on web-based resources rather than ITCs in teaching
more generally. This was in part simply a continuation of the predominant web focus of ICT strategy
across the university, partly reflection of the fact that the main resources of interest, the non-LMS
course sites happened to be web resources and partly a result of the practical difficulty of locating
resources which cannot be located with a few clicks on a browser. The primary focus on web use does
not prevent the survey from incorporating information regarding non-web ICTs, particularly when this
information was often useful in placing web resource use in broader teaching context. Further
reflections on these issues are to be found in the body of the report.

The stocktaking survey
The first part of the review was a stocktaking survey of websites supporting undergraduate units of
study. The sites are referred to for convenience as 'UoS websites' and defined by the following
characteristics.
     1.    There are multiple linked pages with content supporting particular units of study, grouped
           around a specific homepage. To provide as realistic a picture of website use, the census
           needed to avoid including ‘dummy sites’ and only count those where there is substantive
           support for a specific unit of study. Cases where sets of pdf documents are linked to a



Use of web-based resources in EITA undergraduate teaching                                               3 of 32
           homepage are recognised as sites for the purpose of the census but not cases where there is
           little more than a link to a unit outline or another website. Pages on local faculty/school web
           pages that simply provided link page service were excluded from the count. 'Sites' on the
           central LMS that filled a similar role were excluded unless there was other unit-related
           content at the same time in order to maintain a consistent approach across both LMS and
           non-LMS environments.
     2.    There is a site title explicitly citing the unit code or name of a particular unit of study.
     3.    The site is set up in a way that indicates intention to support a particular unit or units in the
           year and semester for which the unit or units were timetabled. Sites that provide a more
           general or distant form of support, such as lab sites, are not included. Old sites from previous
           years are also excluded where there has clearly been no updating.
     4.    The unit supported by the site forms a regular and specific content element in an
           undergraduate program. To provide a realistic picture of website coverage it was necessary to
           avoid overstating the number of units that could expect to be covered, just the same as for the
           actual number of sites. Generic electives and special project type units are not included as
           these units do not run on the same regular basis as other units, and it would be unreasonable
           to expect web coverage on the same basis. In the rare cases where sites have been created for
           such units, the site in question has been omitted from the count.
The stocktaking was organised so that no site or unit of study was counted more than once for the year
regardless of the number of sessions where the site or unit runs or the number of different units that an
individual site serves within a particular session. The website count for second semester has a different
status from that for the first semester total as the survey was conducted from a second semester vantage
point. The second semester count comprises sites verified by viewing of actual sites while the first
semester count includes sites no longer available at the time of survey but whose previous existence
was indicated by some sort of site remnant such as archive file. The website total for second semester
includes sites running in both first and second semester, since these were still active at the time of the
survey, while the first semester total includes sites that were available for that semester only, thus avoid
ing double counting.
Location of websites has relied primarily on a manual scan of relevant web sub-domains,
complemented by Google Search on unit code prefixes. The main locations searched were those of the
central university LMS and faculty/school websites. Sites found by scan were crosschecked against
those reported in staff interviews. Only one undetected site was discovered in this way. There may be a
number of further sites that have remained undetected but the number is likely to be small.

Staff survey
The staff survey was based around a set of questions dealing with how and where web-based resources
are being used, the quality of support received by teachers in doing so and where support might be
improved. Questions were set out in a written questionnaire (see Appendix) that contributors had an
opportunity to examine before responding. Responses were collected orally wherever possible, but
written responses were accepted where staff contributors requested to do so. The majority of
contributors were selected randomly from staff directories according to availability, though a small
number of contributors were specifically targeted for a contribution due to their known interest in the
subject. Invitations were made by phone and followed by email and direct contact. A total of 29 staff
contributed to the survey: 28 academic staff plus one non-academic working in a web support role that
included direct participation in various aspects of teaching work. The non-academic's extensive
knowledge of both teaching context and technical issues of school and faculty made her an essential
inclusion for the survey. Another two non-academic staff members involved in web support for
specific units of study were also consulted, but their contributions play a more background role.
An attempt was made to ensure comparable representation for the six faculty/schools and this was
achieved overall, though with somewhat reduced representation for EIE where there was some
difficulty contacting contributors due to a high proportion of external staff, compounded by others
being away or just returning from travel at the critical moment. In other respects, the survey
contributors provide a fairly diverse spread of staff backgrounds, including junior and contract staff as
well as more senior members and non-beneficiaries of current web support services as active
participants. A critical aspect of the survey approach was to minimise as far as possible the risk of only




Use of web-based resources in EITA undergraduate teaching                                                  4 of 32
talking to the "converted" and missing out on those in the best position to help identify where the gaps
really are in current support services.

Table 0.1 Breakdown of survey contributors
        Position                                            Location
          Professor                              1            AMME                   6
          Ass. Professor                         2            ARCH                   5
          Adj. Ass Professor                     1            CHEM                   4
          Hon. Ass. Professor                    1            CIVIL                  4
          Senior Lecturer                       14            EIE                    4
          Lecturer                               6            SIT                    6
          External Lecturer                      3
          Non-academic                           1
                                                            Level of units taught
        Usage of central LMS                                  1st year              10
          Current year                          21            2nd year              10
          Previous years only                    3            3rd year              17
          Never registered                       5            Final year            11




Terminology
The website stocktaking survey involves some specialised terms that require some initial clarification.
Apart from the term 'UoS website', which has already been defined above, the specialised terms are as
follows.
LMS (or Learning Management System) is used in referring to the central learning resource platform
commonly known as WebCT. Name changes over the past two years have rendered the WebCT label
somewhat anachronistic but substitute terms emerging since do not have comparable recognition and
are liable to cause confusion. LMS is a fairly well accepted term for the generic software family to
which the former WebCT belonged and is adopted as best available alternative at present. ‘LMS
support' is the support supplied by Sydney eLearning, the unit currently responsible for system
administration and user support on the former WebCT system.
CMS (abbreviation for Content Management System) is used in broad to describe systems that enable
web content to maintain a consistent format across different websites with different website builders.
The term is used mainly where a distinction needed to be made between UoS websites delivered under
a controlled institutional format and sites of purely personal design. It does not necessarily mean that
there was full-featured CMS software product in use, but at least some sort of mechanism for
delivering sites in CMS type manner with standardised institutional appearance.
Primary UoS website is used to distinguish sites which are the primary location for content supporting
a particular unit of study from those playing a more secondary role alongside them. The distinction is
important in assessing the extent of websites with particular types of content. To describe the extent of
websites containing content such as lecture notes, quizzes, discussions etc. as a proportion of overall
websites without distinguishing primary and secondary sites would make the extent of sites with
particular items appear misleadingly low, due to the large number of sites in secondary roles with only
minimal content.
Faculty/school is the label used for each of the six main organisational units of the Engineering-IT-
Architecture area. With one unit existing as a full faculty and five as schools within a faculty, it was
necessary to come up with a common term that applied equally in both cases. The six faculty/schools
are individually referred to by the accustomed abbreviations: AMME, Chemical, Civil, EIE, SIT and
Architecture.




Use of web-based resources in EITA undergraduate teaching                                                  5 of 32
Organisation of the report
The report comprises three sections summarising and discussing the findings of the stocktaking and
staff survey followed by final section of recommendations.
          Section 1 deals with the extent and role of UoS websites based on stocktaking data and staff
          survey responses.
          Section 2 deals with staff survey responses regarding gaps in support for academic staff in use
          of web-based resources.
          Section 3 deals with staff survey responses evaluating overall the level of support received by
          academic staff in use of web resources for teaching.




Use of web-based resources in EITA undergraduate teaching                                                6 of 32
Survey focus 1. What is the extent of UoS website use
and what are they used for?

Summary of findings
     1.    There are four main limiting factors in the way UoS websites are currently used:
           •    Website availability. Two thirds of units taught in undergraduate programs have a
                specific supporting website. One third of units miss out.
           •    Website standards. Unit of study outline and regular handouts, consistent with the
                progression of the course, are standard minimum expectations for UoS websites, but
                these expectations are not consistently met. Website contents in many cases are simply a
                handful of documents in unstructured order, for occasional use only rather than a regular
                point of reference for the unit as a whole.
           •    Website role. The role of handout repository that UoS websites most readily assume is a
                role that places them at a distance from what tend to be seen as the more central teaching
                issues of supporting students as active, reflective learners in complex problem-solving
                environments.
           •    Visibility of website impact and potential. What UoS websites are actually doing and
                what they are capable of doing beyond the basic repository role is difficult to determine
                in many cases. The quality of a website cannot always by judged by its technological
                features.
     2.    There are examples of success in overcoming the first three of these factors:
           •    In relation to website availability, universal access has been achieved at both SIT and
                EIE.
           •    In relation to website standards, SIT websites assure systematic coverage of all units and
                consistently meet minimum content standards.
           •    In relation to website role, there are a number of outstanding examples of web-based
                resources engaging imaginatively and effectively with core teaching challenges such as
                interactive group learning, reflective learning and provision of just-in-time feedback,
                particularly in Architecture and SIT.
     3.    Visibility of website roles is the most challenging gap here. The sites themselves are easy to
           see, but what they are actually doing is not always apparent. The crucial evidence of tracking
           data and student feedback is unavailable. The means for gathering this data exist, but there
           has been little use of them so far for systematic comparison of sites.
     4.    Access to diverse software and locally hosted websites are integral to the use of web
           resources in the Engineering-IT-Architecture teaching areas. Future progress requires a
           coordinated use of both central and local systems. The WEBfast system used in EIE provides
           a potential solution for areas that the central LMS is unable to reach and where other local
           systems are unavailable or of limited capabilities.

Overall extent of UoS websites in use
The websites meeting the criteria of the stocktaking count across the six faculty/schools of
Engineering, IT and Architecture were 240 in total for 2007 as a whole. There were 127 sites for
second semester (or both semesters) and 113 sites for first semester only (Table 1.1 below). The
number of units supported by these sites was somewhat smaller, due to some units having more than
one site. The number of scheduled units having UoS websites was 206 for the year as a whole,
comprising 109 units in second or both semesters and 97 units for first semester only. As a proportion
of overall scheduled units, those having UoS websites represent just over two thirds of total in each of
the semesters and for the year as a whole. Two out of every three units of study could expect some sort
of supporting website while one in three could not.




Use of web-based resources in EITA undergraduate teaching                                                   7 of 32
While website coverage spreads evenly from one semester to the next, there are some variations among
the different faculty/schools. SIT and EIE enjoy a universal service level of web support, with sites
provided across virtually all units. The other four faculty/schools lag behind at levels ranging from
roughly 70%, in both Civil and AMME, down to less than 40% in Architecture.

Table 1.1 Units of study supported by UoS websites active during 2007 academic year
                                                                                                           Joint
                                          AMME           ARCH         CHEM        CIVIL        EIE   SIT   units    Total
 Units current in Semester 2
    Units of study                            31            47         11           18          23   24     7       161
    Units with UoS websites                   20            18          7           11          23   24     6       109
    Sites outside central LMS                 16            23          0           9           27   14     4        93
    Sites on central LMS                      3             2           6           5           1    14     2        33
    All sites                                 20            25          6           14          28   28     6       127

 Units for Semester 1 only
    Units of study                            29            31         12           16          27   21     6       142
    Units with UoS websites                   21            11          7           14          20   21     3        97
    Sites outside central LMS                 17            14          0           5           26   12     2        76
    Sites on central LMS                      6             1           5           10          2    12     1        37
    All sites                                 23            15          5           15          28   24     3       113


Comparing LMS and non-LMS based sites
Non-LMS based sites accounted for over two thirds of total UoS websites in 2007, 169 out of 239
across the two semesters. As a proportion of websites playing the role of primary course site (there
were a number of units with multiple sites), non-LMS sites represented three quarters (142 out of 182
primary sites). Non-LMS sites were divided between those operating inside local content management
systems and those of personal design.

Table 1.2 Breakdown of 2007 UoS websites by web platform used and faculty/school affiliation
                                      AMME         ARCH          CHEM       CIVIL         EIE        SIT    Joint   Total
 Units current in Semester 2
    Personal                             8          23            0          1            10         11         0    53
    WEBfast                              0           0            0          0            16         0          4    20
    Other local web systems              9           0            0          8             1         3          0    21
    Central LMS                          3           2            6          5             1         14         2    33
    Total                                20         25            6          14           28         28         6    127

 Units for Semester 1 only
    Personal                             11         14            0          0            18         11         0    54
    WEBfast                              0           0            0          0             4         0          2     6
    School CMS                           6           0            0          5             4         1          0    16
    Central LMS                          6           1            5          10            2         12         1    37
    Total                                23         15            5          15           28         24         3    113

The local systems varied considerably in their capability. The simplest was the AMME system where
UoS websites took the form of a unit of study description with a list of links alongside. The most
versatile was WEBfast, developed by the IT Unit at EIE and released at the beginning of 2007. The
WEBfast system combined a high degree of flexibility in page and website layout with simple, fast
editing and minimal skill threshold for new users. The success of the WEBfast can be seen in its rapid


Use of web-based resources in EITA undergraduate teaching                                                            8 of 32
adoption across EIE. By second semester 2007, WEBfast had replaced personal sites as the main
platform for UoS website production. WEBfast had also found a foothold outside EIE in the ENGG
units of study.
Some correlation can be found between the level of website provision in the different faculty/schools
and their different website hosting arrangements. What seems to make a difference is not the use of one
specific platform per se, but an ability to combine multiple options. The common feature of EIE and
SIT, the faculty/schools best provided in UoS websites, is that personal websites and strong content
systems are both widely available: the system for EIE being WEBfast, and central LMS at SIT.
Elsewhere the options are limited in various ways and these limits seem to be reflected in the
diminished levels of website provision found there. Website hosting is largely confined to a single
option in both Architecture and Chemical, though precisely the opposite kind of single option in each
case. Architecture relies almost entirely on personal websites, which means in effect relying heavily
upon academic staff having website production skills. Units in the Design Computing area account for
most of the sites within Architecture while other streams are poorly represented. Chemical, by contrast
with Architecture, has no personal web publishing option whatsoever and relies entirely upon the
Central LMS, but again a substantial proportion of units remain without web support. Civil
Engineering and AMME both make combined use of central LMS and local CMS platforms, but to a
proportionately greater extent in Civil. AMME on the other hand is more heavily reliant on personal
UoS websites. There may well be other reasons for low website use in some areas, such as lack of
perceived need. However perception of need for websites appears to happen more often where a range
of website options is more readily found.
There is no 'one size fits all' web solution for the UoS website needs of Engineering, IT and
Architecture. However, there is room for much wider use of the systems available to support website
design and management, the central LMS and WEBfast in particular. The WEBfast system provides a
potentially quick and effective solution for areas that the central LMS is unable to reach and where
local systems have limited capabilities or are absent altogether.

Location of websites and navigational access
Apart from differences in the overall provision of supporting websites there are also some disparities in
access arrangements for the sites. While a central access point for all LMS based sites is provided via
the LMS link pages on the university website, there are no consistent arrangements for access to non-
LMS based sites.

Table 1.3 Access arrangements for non-LMS UoS websites in Semester 2
             Non-LMS sites with
                                              School website link page for unit of study websites
             direct LMS gateway

 AMME           One site                         http://www.aeromech.usyd.edu.au/current/units_of_study.shtml

 ARCH           Two sites only                   None available

 CIVIL          None                             http://www.civil.usyd.edu.au/current/undergraduate/units_of_study.shtml

 EIE            Three sites                      http://www.eelab.usyd.edu.au/eLearning/view_uos.html

 SIT            All sites                        None available

 Joint          One site                         Link as for Civil above.

There are index pages on School websites for units of study in AMME, EIE and Civil. Sites for joint
ENGG units share the Civil Engineering page. The Civil index includes LMS based sites on its list as
local sites. The other index pages include local sites only. Gaps can be found in the site listings for
each of the school index pages, probably due to difficulty in keeping up to date with changes in sites
available. Access to local sites in the School of IT is through gateway sites inside the LMS, taking
advantage of the LMS's central login location. The limitation of this system is that it does not provide
an easy way of seeing sites across the school as a whole, as is possible with index pages used in other
schools. The least accessible sites are those in the Architecture web domain where almost all sites are
locally hosted but without any index page to enable easy location.



Use of web-based resources in EITA undergraduate teaching                                                  9 of 32
In addition to the links mentioned there are collective gateway sites inside the LMS offering indirect
links to sites for particular faculty/schools as a whole. These can give the false impression that units
that are not supported on the LMS may have sites in the non-LMS area, which is often not the case.
Chemical Engineering had an LMS gateway site for its ‘non-LMS sites’ while having no actual sites at
all outside the LMS. The page simply links to the school website instead. None of collective gateway
actually link directly to any particular site but simply whatever index pages the particular school
provides, if any.

Content of UoS websites
Analysis of website content is confined to content in use at the time of the survey, which is to say
Semester 2 websites. These provide a reasonably large sample on their own without the first semester
sites and do not bring any of the retrospective uncertainties that first semester material would entail.
The content found was predominantly text, either as web page, document file, or slide presentation.
The most commonly appearing content item was the unit of study outline or an equivalent hyperlink.
Organised handout series covering the course program were the second most common content type
(Table 1.4 below).

Table 1.4 Semester 2 UoS websites with basic content items: unit outline and regular handouts
                                   AMME        ARCH         CHEM     CIVIL     EIE        SIT         Joint       Total
 Primary UoS websites                19          17          6        11       22         14           5           94

 Sites giving access to:
    Unit of study outline            11          10          5        9        21         16           3           75

    Regular handouts                  9          13          5        8        15         14           5           69

 Sites on local password
                                      5           1          0        0         3         2            0           11
 (content unknown)

The main qualitative difference between UoS websites is in the reliability with which outlines and
organised handouts can be found. At SIT, unit outlines and regular handouts could reliably be found for
all Semester 2 2007 units of study. In most of these units, there was an LMS based site linked to a local
site and in some cases double copies of outline and handouts posted in both locations. The number of
SIT sites found with unit outlines and handouts is thus higher than the number of primary UoS
websites that could be viewed at the time. EIE, Civil and Chemical had the unit outline and regular
handouts for the majority of sites but fell short of full coverage on a small number. It should be noted
in relation to low numbers for AMME that these are affected by high proportion of sites under local
password where content could not be verified.

Table 1.5 Semester 2 UoS websites with dynamic content items
                                     AMME        ARCH        CHEM      CIVIL        EIE         SIT        Joint     Total
 Sites giving access to:
    Multimedia                            3           8          0         1         3          1             1          17
    Interactive online
                                          1           4          0         1         0          3             1          10
    exercises /quizzes
    Discussion forums                     3           5          3         0         1          3             1          16
    Student pages (wiki,
                                          0           4          1         0         0          1             0           6
    blog, presentations etc.)
Table 1.5 above shows other more dynamic types of content appearing on websites. These include
multimedia (17 sites overall for second semester), discussion forums (also 16 sites) interactive
exercises with automatic feedback (10 sites) and student pages, where student work is presented and
shared (6 sites overall). The leading role of Architecture in the use of these resources was a striking
contrast with what was seen in relation to basic website provision, but consistent with the previously
noted tendency of computer related units and staff with strong computer backgrounds to be well-
represented among Architecture websites.




Use of web-based resources in EITA undergraduate teaching                                                               10 of 32
Comparing the four content types, it should be mentioned that numbers of sites where discussions were
actually used to a significant degree was much smaller than the 16 sites listed as having discussion
facilities. Only eight discussion forums showed signs of use and only five had a regular pattern of
postings. Of the four content types, interactive online exercises had the widest real impact across the
faculty as a whole, though the small number of cases was still a relatively proportion of overall site
numbers.
Another content type was print-based exercise downloads, in which students were given problems to
work through along with solutions to check their work. This might be considered yet another form of
interactive content, differing from the online variety in that the interaction happened on paper rather
than on screen. The difficulty with reporting this kind of content is that it doesn’t always show itself as
obviously as the fully digital equivalent and can be hard to spot during a quick website visit. The
numbers for this kind of material are indicative only. They represent the number of examples found,
not necessarily the only ones that exist.

Table 1.6 Semester 2 UoS websites found with exercise downloads.
                                    AMME        ARCH        CHEM       CIVIL          EIE         SIT         Joint        Total
 Sites giving access to
 exercise downloads                     4           0           1           4          0            3           0           12
Other types of material that might be seen as having a certain ‘interactive’ facet are those providing
support for interactions taking place in the live teaching environment. Examples of such material are
lecture notes with blanks to be completed during actual lectures and tutorial problems where solutions
are given in the tutorial session. Like exercise downloads, this type of material is also hard to spot
amongst surrounding handout documents and no numbers were taken for this reason. The critical point
to be drawn from this material is firstly that the interactive learning dimension of the UoS website is
not simply a matter of its collection of digital tools present and secondly, that we need to be careful
about drawing conclusions regarding the static nature of websites simply because the static features are
all that we see.

Technical issues on UoS websites
There was no time for full technical checking of UoS websites, but maintenance issues were noted
when encountered as a further aspect of website content. The types of issues encountered were dead
links, outdated content and mislabelled content. The number of current active sites affected by such
issues was relatively low, only ten in total.

Table 1.7 Semester 2 UoS websites with basic maintenance issues noted: dead links, mislabelling, outdated
content.
                                AMME        ARCH        CHEM        CIVIL       EIE         SIT         Joint    Total

 Primary UoS websites              19          17           6        11          22         14            5           94


 Sites with basic
                                    1           2           0         3           3          0            1           10
 maintenance issues

The relatively low level of maintenance bugs is a creditable result in the context of a web environment
that is predominantly local and self-supported, outside the checking procedures of central LMS. In this
context, it should be mentioned that locally hosted sites were not the only ones with maintenance
issues. Three of the sites with bugs found were LMS hosted. In terms of comparing the content features
of non-LMS and LMS based sites overall, there is not a lot to be said however. Only a small number of
LMS sites were in a position of meaningful comparability with non-LMS sites, which is to say in use
as primary websites in an area where non-LMS websites were also in use alongside. All that can be
said at the level of website content is that no significant differences have been found between LMS and
non-LMS environments.

How web-based resources are used
Descriptions of use of web resources from the staff survey show a strong association between UoS
websites and handout functions as was suggested by the initial content breakdown. The descriptions



Use of web-based resources in EITA undergraduate teaching                                                                    11 of 32
also show a strong interest in more active and interactive dimensions. However the interest is more
likely to be expressed outside UoS websites than within.
Use of UoS website as handout repositories was reported by 21 out of 28 academics interviewed (Table
1.8 below). Of the remaining seven academics, two had no site to speak of, one reported using a
personal blog for handout purposes and in the other four cases there was simply a lot more to be
described on the website, apart from handouts. The main difference between academics in use of
website handouts was more a question of whether the website stopped there or went beyond. It was
more less or less taken for granted that UoS websites would play some sort of handout role was. The
question was simply whether there was going to be anything else to them.
Description of how UoS websites and other web resources were used brought with it further
information about the use of non-web internet resources such as email and beyond that non-internet
computer based resources with functions. To see the full extent of the skills practice and student
communication aspect of ICT use, it was necessary to include these wider ICT domains as well. The
roles attributed to the various ICT types are set out in the table following, along with the number of
academics mentioning UoS websites and other ICT types in relation to each role.

Table 1.8. Roles of UoS websites and other ICT resources with numbers of academics mentioning each.
                                                Using       Using UoS   Using other         Using non-
                                               any ICT       websites   web/internet      internet based
 Roles described                                                         resources             ICTs
   Assessment                                      4            3            1                   0
    Class management                               4            3             0                  0
    Collaborative project work                     3            0             3                  0
    Demonstration                                  3            1             1                  1
    Handout repository                             22          21             1                  0
    Resource portal                                2            2             0                  0
    Student communication                          10           2             8                  2
    Student information seeking                    1            0             1                  0
    Student publishing                             2            2             2                  0
    Student skills practice                        14           5             4                  5

Outside UoS websites, the focus was more on resources for purposes of student communication and
skills practice appears at a higher level than was seen in the website content breakdown. Half the
academics contributing to the survey report use of resources for skills practice type purposes. Slightly
over one third reported using resources for student communication. In both cases, the resources used
were predominantly outside UoS websites rather than on them. For student communication, email and
blogs play a role no less important than discussion tools on UoS websites. For skills practice, non-
internet resources play a major role.

Use of UoS websites in relation to teaching and learning challenges
The staff survey provided information regarding what academics saw as the main teaching and learning
challenges of their units of study as a context for explaining their use of web based resources. Staff
descriptions of their teaching and learning context confirmed that staff were a lot more student focused
and active learning focused than the handout centred evolution of the UoS websites might suggest.
What was limiting the use of UoS websites to static content purposes was not that teaching staff were
failing to connect with the more dynamic side of learning and teaching; it was simply that UoS
websites didn’t enter into the connections they were making.
Altogether there were 18 types of learning and teaching challenge mentioned by the 28 academics
contributing to the survey. The 18 types of challenges related 5 general aspects of the learning
environment: course content, student profile, student performance, design of learning situations and
institutional frameworks. There was a relatively high level of concern with course content, marked
particularly by the frequent mention of new concepts as challenge. But this wasn't about rote learning.
Alongside the concern with mastery of basic technical elements was an even higher level of concern
about students being self-motivated in working through the basic material (mentioned by 12 out of 28
staff), and critically reflective in thinking about what they already knew and what they still needed to



Use of web-based resources in EITA undergraduate teaching                                                  12 of 32
learn, and where their knowledge may or may not apply in solving of complex problems (11 out of 28
staff). Challenges under the 'learning situation' heading related again to the need for students to learn
for themselves rather than being told directly what to do, 'not spoon-feeding' as more than one staff
member mentioned. Students were described as needing learning situations that:
     •    provided opportunities to learn by doing (3 mentions)
     •    promoted reflection(4 mentions)
     •    were realistic (3 mentions)
     •    were relevant to student backgrounds (2 mentions).
     •    enabled them to learn through interaction with others (7 mentions)
     •    supported them in learning how to manage the complex and challenging conditions that come
          with group and team activities (3 mentions)
     •    provided fast and informative feedback (4 mentions).
An underlying thread within the various concerns around the challenge of getting students to learn
independently and reflectively in advanced problem solving environments was the question of the
transparency and accessibility of the information channels on which students had to rely in doing so.
          ‘It is very difficult for students to understand what their expectations are in the problem-based
          learning [work] until they've done the first part. . . formative assessment is great, but you need
          some sort of "feed -forward" as well as some sort of feedback. And the problem is that "feed-
          forward". Certainly from everything I've looked at, everybody acknowledges it's necessary,
          but nobody knows how to do it well.’
          ' There’s no point in the students knowing everything at the end of semester, because at the
          end of semester they may forget everything. They have to have everything as fast as possible,
          so that they can make use of the information during the semester when they actually need it. . .
          . Instantaneous communication is crucial. . . . it’s no good if they get help three days later
          because they will have abandoned the whole thing and moved onto something else which they
          are able to do rather than what they intend to do. The outcome would be then only things
          which they are able to do but not reflecting the design that they want and the quality would
          not be there.'
Acceleration of information feeds and creation of just-in-time feedback loops are very much core
business of current information and communication technologies but it was only in a small cases that
academics saw any significant role for technology in these kinds issues. And where the connection was
made it just as often from a perspective of what was lacking in current technologies as from a
perspective of effective solutions having been found. While one academic saw the multiplication of
communication channels bringing levels of speed, accessibility and support for reflective learning
processes at levels never previously possible, another described currently available tools as inherently
‘passive and limited’ and incapable of matching the intensity of face to face interaction in traditional
lab settings. Both academics were in ICT related academic disciplines. Those with strong ICT
backgrounds were best placed to see the limitations as well as the possibilities of current ICT tools in
supporting more interactive aspects of learning. The majority of staff, however, were not in a position
to make any specific connection between core learning and teaching processes and potential
applications of technology. Only 9 out of the 28 staff surveyed made any link at all between the
learning and teaching challenges that they saw in their units taught and the way web and other
technology based resources were used in their teaching. Given the diversity and uncertainties in the
way this link might be made, hesitations in doing so are understandable.

Conclusion
The overall lesson of these very mixed experiences in use of web resources for teaching is that
breaking out of the conventional repository model is possible but not guaranteed and that much better
knowledge of individual contexts is necessary before we can say with any certainty what makes
innovative approaches more achievable in some cases than others. This is yet further illustration of the
need for more systematic documentation of existing practices in web resource use.




Use of web-based resources in EITA undergraduate teaching                                                 13 of 32
Survey focus 2. Gaps in support for teaching use of
web-based resources

Summary of main areas of need
     1.    Integrated ICT support. Services supporting local websites, lab-based software, central LMS,
           email and general IT are currently delivered in isolation focusing more on each technology as
           an end in itself rather than teaching environment they are supposed to support. A more
           integrated approach is required.
     2.    Pro-active LMS support. Raising LMS use above present low levels will require some effort
           in addressing access barriers currently experienced by staff.
           •    The complexity of the LMS software and the time required in learning to use it
                effectively.
           •    Time involved in having sites activated at the start of semester.
           •    Difficulty in obtaining the UniKey logins needed to have accounts initially set up on the
                LMS.
          The third is mainly a problem for new and contract staff. The other two affect all current non-
           users of the central LMS and a proportion of current users as well. The first of these,
           complexity and time required, appears to be the one with greatest impact and requiring most
           immediate attention.
     3.    Integrated systems for managing assessment information and unit of study information.
     4.    Common access to existing web-based resources used in teaching.
     5.    Recognition and support for development of new learning materials as a regular part of
           teaching duties.
     6.    Explicit service level agreements with service providers covering support of computer and
           web-based resources in teaching as a whole, along with appropriate procedures for
           monitoring and responding to service lapses.

Responses to the survey question asking staff to identify support gaps
Across the various responses on the question of gaps in current support for teaching use of web-based
resources, 12 recurrent areas have been identified. The twelve areas form three overall groups
according to whether they concern resources outside the central LMS, or the LMS itself or support
functions that are not platform specific.

Support for non-LMS based resources.
Gap Area 1. Support for existing resources run from local servers. The range includes resources used
directly in teaching of students along with management tools such as unit of study database and
assessment management software. It includes local file sharing services and software run over local
networks as well locally hosted teaching websites. The common element of the various technologies is
simply that each plays an integral part in the effectiveness of the learning and teaching environment as
a blend of technology-mediated and face to face experience. The common concern here is that the
question of how to better support the use of new web-based resources should not simply be about new
resources for their own sake but about the effectiveness of the support environment as a whole. The
gap is about supporting new technologies in ways that genuinely blend with the local environment
rather than simply rolling over the top as if it wasn't there.
Gap Area 2. Integration of existing resources and information systems used in teaching. This gap area
is partly a flow on from the previous area concerning support for locally based resources. It's not just
about integration among local resources but also integration with those running centrally. Suggestions
include common gateways for unit of study websites, cross linking of central LMS and locally hosted
websites, more efficient systems for handling student assessment information.



Use of web-based resources in EITA undergraduate teaching                                               14 of 32
Gap Area 3. Support for use of email in teaching. Responses in this areas included concerns about
disparity between level of support for email compared to central LMS support, or features affecting
usability for teaching purposes such as file-space limits, or simple lack of recognition for the central
role of email in student teacher communication.
Gap Area 4. Range of technology options available. Concerns in this area related to technology options
that exist but are out of reach, such as access to alternative learning management systems, blogs or
wikis, or options yet to be developed such as fully scalable video-conferencing.

Support for LMS users
Gap Area 5. Central LMS user help. Concerns in this area were mainly related to support for users in
dealing with initial entry barriers: the initial learning curve, the processes of set-up and activating of
new sites and obtaining login access as a new user.
Gap Area 6. Central LMS software features. Criticism of general LMS usability accounted for roughly
half of the expressions of concern in this area. The remainder were suggestions for improvement in
specific aspects of software functionality. In some cases the problem raised was not easy to solve, but
in other cases a solution was under way or in one case already taken care of in the recent LMS upgrade.

General support needs not specific to software platform.
Gap Area 7. Support for development of new learning materials. Materials mentioned were mainly
independent learning type resources with built in feedback, with varying levels of complexity from
standard quiz type exercises to virtual reality simulation.
Gap Area 8. Help in administering & maintaining web-based learning resources. Concerns in this area
related to recognition of workload carried by academic staff involved website administration as well as
a need for appropriately skilled personnel available for assistance.
Gap Area 9. IT support. Concerns in this area focused on a mix of IT staffing issues, IT skills and the
extent to which support of teaching websites is recognised as part of IT support role.
Gap Area 10. Support for resource sharing. The gap here is about opportunities for resource creators to
take their products forward as well as opportunity for others to draw on previous materials instead of
having to reinvent. Suggestions in this area include putting currently closed LMS sites on open access
to other staff members and creation of an open repository of resources developed through university
supported projects.
Gap Area 11. Support communication. This gap area was about the need for clear, consistent
communication of support conditions and availability and the stress and waste of energy occurring
when communication was lacking. Concerns include simply not knowing what resources are available
or what to expect in relation to particular resources and the losses incurred when staff are led to make
assumptions about the quality of particular resources that later prove unfounded. A website assumed to
be core infrastructure fails at a critical moment, an assessment due date, and the lecturer is left with an
inbox full of distressed emails and no satisfactory explanation. Another lecturer finds his whole
working week disappearing into website development work following instructions that somehow fail to
mention the time the job might take.
The proportion of survey contributors raising each of these 11 areas is shown in Table 2.1 below, with
breakdown by school/faculty.

The majority perspective
The overall focus seems to be on more on the basic framework of ICT support rather than the creation
of high end learning environments or the high end tools needed to build them. This is a 'bread and
butter' kind of wish list focusing on bare essentials more than luxuries, reflecting environments where
having enough of the bare essentials may be considered a bit of a luxury. Interest in new materials and
new tools is certainly present, with 30% and 17% respectively expressing needs in these areas (Gap
Areas 7 and 4). But alongside these are:
     •    52% wanting adequate maintenance for existing local ICT based teaching resources (Gap
          Area 1),
     •    14% wanting email recognised and supported as an essential teaching tool (Gap Area 3)



Use of web-based resources in EITA undergraduate teaching                                                    15 of 32
     •    41% wanting simpler more straightforward use of the central LMS (Gap Area 5)
     •    31% wanting better integration among existing resources (Gap Area 2)
     •    31% wanting to be able to look at examples of how web-resources are used in teaching by
          someone other than themselves (Gap area 10)
     •    24% wanting clear, consistent communication of the support they can expect to receive in the
          first place (Gap Area 11).

Table 2.1 Gaps in teaching support coverage mentioned by staff responding to the Engineering Snapshot survey
with numbers mentioning in each case.
                                       AMME       ARCH      CHEM   CIVlL    EIE     SIT    Total       %
 For non-LMS resources
   1. Support for existing local
                                          4          2       2       2        1      4       15      52%
   resources.
    2. Integration of existing
                                          2          1       1       2        1      2        9      31%
    resources and systems
    3. Support for use of email
                                          1          0       2       0        0      1        4      14%
    in teaching
    4. Range of technology
                                          1          0       0       0        0      4        5      17%
    options available

 For use of LMS
    5. Central LMS user help.             2          3       2       0        3      2       12      41%

    6. Central LMS software
                                          4          0       1       1        0      4       10      34%
    features.

 General support needs
   7. Support for materials
                                          2          1       0       2        0      4        9      31%
   development
    8. Help with maintenance
                                          1          1       2       2        1      0        7      24%
    of web resources

    9. IT support services                2          2       3       0        0      0        7      24%

    10. Support for sharing of
                                          3          0       0       2        1      3        9      31%
    existing resources
    11. Communication of
                                          2          2       0       1        1      1        7      24%
    support available

 No. of staff responding:                 6          5       4       4        4      6       29      100%


Local variations relating to development of new learning resources
Listing of new materials and new technology as needed improvements occurs more frequently among
SIT staff, mentioned by 4 out of 6 for both items, than elsewhere. At the same time, concerns regarding
specific types of support personnel are found only in AMME, Architecture, Chemical and Civil but not
SIT and EIE. The concentration of staffing concerns in these four faculty/schools is consistent with the
developmental differences of online support capabilities observed in previous survey of teaching web
sites. Distribution of concerns regarding access to specific support personnel is shown in Table 2.2
following.




Use of web-based resources in EITA undergraduate teaching                                                      16 of 32
Table 2.2 Number of staff identifying needs for specific support types in different faculty/schools

 Specialist support:                    AMME        ARCH      CHEM        CIVIL       EIE        SIT      Total

    General IT support services             2          2         3          0          0              0    7

    Support in educational
                                            1          0         0          2          0              0    3
    design of new resources
    Support in IT aspects of
                                            1          1         0          0          0              0    2
    new resource development

In the context of these variations across the different faculty/schools, there are two points to be made
concerning new resource development. The first is that the need exists, though not a majority concern
overall. Being a minority issue overall doesn't stop it becoming a majority issue in specific contexts.
The second point is that the resource development support needs identified by the minority are
connected to the issues of basic infrastructure concerning the broader majority. The bulk of the support
needs of new resource development concern capabilities of a basic and general nature rather than
highly specialised services. Apart from two cases of need for software programmers, the kinds of
support skills were more of a generalist nature, general IT support and general educational design.
Academics interested in new resource development are not actually asking for anything exceptional in
most cases, but simply asking something specific of the general support infrastructure. It is the needs of
those wanting support for specific purposes that best bring out the limitations of support in general. It is
those who are specifically seeking who tell us what is specifically hard to find: things like feedback
and advice on resource design of learning materials, confidence in knowing how far specific systems
may or may not be relied upon.
          'We are engineers and when we design something we need to know the constraints of the
          project, i.e. what we can do, what we cannot do, can we do something else, how we can do it?
          If we don't get these boundaries, we just keep on spinning producing stuff. In the work which
          I have been doing in the last few years thanks to university support, I have experienced this
          feeling on several occasions.'

How is development of new learning resources best supported?
When resource developers found themselves suffering from a sense isolation and lack of recognition
for their efforts, it was not just because new resource work was uniquely invisible or poorly considered
but also because teaching resources as a whole were largely invisible and poorly considered. This is
where the issue of new resource development intersects with another issue of basic infrastructure issue:
the sharing of resources and knowledge about how to use them. The need for an open accessible
repository of existing resources and the need for recognition of new resource work were two sides of
the same problem of getting resources and people to connect with each other.
The problem of nurturing a group dynamic in learning resource development is one that has been
repeatedly grappled with at this university over time but with mixed results according to most recent
review of elearning resource projects by Scott, Mahony & Peat (2007). The creation of effective
support frameworks for resource development and resource developers is still very much a work in
progress.
There have also been a number of previous attempts in the area of web-based learning resource
repositories. To mention a few examples:
          Science and Technology Spotlight: http://science.uniserve.edu.au/spotlight/
          Test Tube Wiki (Faculty of Arts): http://wiki.arts.usyd.edu.au/elearning/index.php/Case-
          Studies
          Health Sciences eLearning Resource Centre. A site hosted on the university LMS. Login on
          request from Sydney eLearning Helpdesk. http:/learn-on-line.ce.usyd.edu.au
          Case studies section of ITL’s Generic Graduate Attributes
          http://www.itl.usyd.edu.au/graduateAttributes/casestudies.cfm
          Learning Designs, sponsored by AUTC: http://www.learningdesigns.uow.edu.au/




Use of web-based resources in EITA undergraduate teaching                                                         17 of 32
          Contemporary Online Teaching Cases at Deakin University:
          http://www.deakin.edu.au/itl/teach-learn/cases/
          Designing e-learning: http://designing.flexiblelearning.net.au/
          Carrick Exchange (under development):
          http://www.carrickinstitute.edu.au/carrick/go/home/rin/cache/offonce
          MERLOT: http://www.merlot.org
          RESL: http://www.resl.ac.uk/
          Design Shop at Virginia Tech: http://www.edtech.vt.edu/edtech/id/index.html
Any new resource repository for Engineering, IT and Architecture will obviously needs to incorporate
links with resources in previous repositories. It will also require some sort of strategy for addressing
what has been the major challenge of previous repository development: building an active user base
(Philip, Lefoe, O’Reilly, & Parrish, 2007). Repository development needs to start from a process of
resource use and a group of active resource users rather than starting from the resources in isolation
and expecting process and people to follow. The interests expressed in discussions of learning and
teaching challenges and learning resource needs during the spotlight survey suggests a potential body
of users for resources in the following five areas:
     1.    Formative assessment
     2.    Problem-based learning
     3.    Creative learning
     4.    Virtual reality simulation
     5.    Integration of generic learning skills
Whether the available resources and interested users are sufficient to make resource sharing viable in
any of these areas is something that would have to be tested by hard experience, possibly not in most
cases. What is certain is without substantial investment in identifying resources and interested users
and putting them in contact with each, there is unlikely to be any momentum at all. The most promising
of the five areas as far as development of resource sharing is concerned appears to be formative
assessment. This was the area of most widely expressed interest during the Snapshot survey and also an
area where a number of known resources exist. Repository development might start firstly with a
census and comparative evaluation of existing formative assessment resources from the perspective of
transferability and potential users.

Conclusions
From a perspective of improving support for use of web-based resources in teaching, the main needs
appear to be:
     •    Integration of support service delivery at local level, addressing the needs of local teaching
          and learning environment as a whole rather than in terms of single isolated technologies.
     •    Streamlining of LMS basic access requirements in relation to the three main barriers basic
          users currently face: initial set-up of user ID, basic user training and site activation. The initial
          set up phase requires a quicker and simpler way of dealing with staff UniKey difficulties than
          simply telling them to ‘go and find out’. Staff suggestions on this point include brief written
          start-up guides dealing with basic navigation and functions as well as short, customised
          training sessions with carefully researched focus on what the particular users really need to
          know.
     •    Integrated systems for handling essential teaching and learning information, in particular
          information related to student assessment, unit of study information and degree mapping.
     •    Development of knowledge sharing resources such as repository of generic learning and
          teaching resources and open access systems for UoS websites including those based in the
          Central LMS.
     •    Support for development of new learning materials that focuses not just on the materials
          themselves but on how they relate to resources already there and the people using them.




Use of web-based resources in EITA undergraduate teaching                                                    18 of 32
     •    Development of more explicit service level agreements covering support of computer and
          web-based resources in teaching as a whole, along with appropriate communication
          procedures for ensuring that support expectations specified in the agreements are consistently
          understood and applied across the six faculty/schools.
     •    Ongoing monitoring of system performance that focuses on key groups most affected by
          irregularities in performance so far: in particular new academic staff and those undertaking
          substantial teaching resource development projects.




Use of web-based resources in EITA undergraduate teaching                                                19 of 32
Survey focus 3. How satisfactory is the overall level of
support for staff using web resources in teaching?

Summary of findings
     1     Current level of support overall for use of web resources in teaching is considered neither as
           satisfactory nor unsatisfactory by the majority of academic staff.
     2     Absence of commonly understood standards defining the level of support that academic staff
           can reasonably expect makes it difficult to form clear cut judgements on the subject.
     3     Within the context of an overall ambivalent response there are specific areas where there is
           some degree of satisfaction. The work of support staff at SIT and EIE and the central LMS
           helpdesk has received a number of favourable comments.
     4     There are also some areas of specific dissatisfaction.
           •    The usability of the central LMS and time demanded in learning to use the system
                effectively were repeated points of concern.
           •    Academics with first year classes are a much higher proportion of unsatisfied responses
                than those without first year responsibilities.




Satisfaction ratings
Responses evaluating the level of support for staff overall were first of all broken down as a linear
series of satisfaction ratings. Comparing the responses in terms of degree of satisfaction expressed, four
categories can be found.
     1.    Responses presenting a positive view with no reservations. Expressions used varied from a
           direct 'My overall satisfaction is very good' to the more distanced 'I think we have all the
           elements.' The common feature shared by all was simply that there were no counterbalancing
           statements to limit the scope of their satisfaction.
     2.    Response presenting a mixture of positive and negative views regarding current level of
           support. These followed one of four patterns: (1) 'I'm happy but I don't ask a lot' (2) 'There's
           some good aspects, some not so good', (3) 'No problem myself but I have concerns for others'
           and (4) 'I do what I want but there's no help doing it'.
     3.    Responses presenting a negative view with no reservations regarding current level of support.
           Responses in this category tended to be uniformly direct in their phrasing, responses of the
           type "not so good" or simply "there is no support."
     4.    Neutral Responses presenting neither positive nor negative views regarding level of support.
           These were mainly responses that focused on constructive suggestions for improvement,
           leaving aside the question of the effectiveness of services in their current unimproved state.
           The category also includes one case where the contributor felt contact was too limited for
           him to offer an opinion on their performance.
Numbers of responses for the four categories were as follows.




Use of web-based resources in EITA undergraduate teaching                                                 20 of 32
Table 3.1 Responses on the question of overall level of service in terms of making positive or negative content.
                                            Response type                                    No. of responses
             1 Positive without reservation                                                            6

                2 Mixture of positive and negative
                2a Happy but I don't ask much                                                          3
                2b Good in parts, not so good in others                                                7
                2c No problem myself but concerns for others                                           1
                2d I can do what I want but not much help doing it                                     2
             3 Negative without reservation                                                            5
             4 Neutral                                                                                 4
                                                                                    Total             28

The largest group of responses is the ‘Some good, some not so good’ sub-category, accounting for 7
out of 28 responses, one quarter of the total. This group sits squarely in the middle of the satisfaction
scale. The middling tendency is augmented by neutral responses (4 out of 28) that likewise fail to
commit themselves in either direction. The dominant message seems to be "We are not unappreciative
of the services available but we have significant concerns about the services that are not there." In
terms of overall numbers, we find over two thirds of staff (19 out of 28) indicating some sort of issue
with the level of service. Of the remaining third who present themselves as essentially happy with
current level of support for use of the web in teaching, a substantial proportion (4 out of 9) identify
themselves as users with limited needs who have really not put the services to the test so far.
Cutting across the middling trend is what seems like an entirely different trend for responses to polarise
towards opposing extremes, accounting for 11 of the 28 responses. On the one side is a group of six
responses expressing a quiet, untroubled contentment. On the other side, are a nearly equal number of
dissatisfied cases.
The overall picture is a somewhat fragmented one. Rather than a single unitary spectrum of views on a
single unitary reality, the responses of overall service level seem to represent a mixture emerging from
a number of different realities: different teaching contexts, different support environments and differing
personal backgrounds, all tangled together. The evidence is not sufficient, however, to tease out the
specific composition of this mix.
Only one background factor shows any sort of correlation with the distribution of positive and negative
responses, and the correlation is a limited one. The factor is 1st year teaching responsibilities. Four out
of six unreserved negative responses came from academics involved in 1st year teaching while only
two unreserved negative responses came from the other group, despite the 1st year teaching staff being
much smaller in a number, as shown in the table below

Table 3.2 Comparing number of unreserved negative responses from survey contributors with and without 1st year
classes
                                    Unreserved negative                        Survey            Proportion of
                                        responses                            contributors        contributors
Teaching 1st year                               4                 out of           11                  36%
No 1st year teaching                            2                 out of           17                  12%
Total                                           6                 out of           28                  21%
Being on first year teaching duty however does not necessarily condemn a person to a bleak view of
the teaching support services. Those tending to the negative end of the spectrum were still a minority.
Some sort of interaction with other factors may be suspected but there is insufficient evidence to say
what those factors might be.




Use of web-based resources in EITA undergraduate teaching                                                          21 of 32
Aspects of support targeted for comment
While contributors were asked to evaluate the level of support that they received overall in the use of
web resources, there was nothing to prevent them from approaching the topic in a narrower, more
selective frame of reference. There was indeed good reason to do so, when in some cases what was
important overall was not the whole field of teaching support services, but certain specific elements.
The focusing of comments varied in two ways, firstly in terms of support providers commented upon
and secondly in terms of the forms of support selected for comment. In terms of support providers,
comments could be about the central LMS support service, central IT, or local services, or a
combination or unspecified. In terms of the form of support, there were again four possibilities, support
as technical resources, support as people, support as technical resources plus people and again, leaving
the form unspecified. The distribution of responses across these two dimensions is summarised in the
table following.

Table 3.3 Providers and types of service mentioned in responses on overall quality of service
                                Support          Technical     Both tech &        Form un-      Total for
                                people           resources       people           specified     provider

   Central LMS                      4                 1              2                0            7


   Local services                   4                 1              0                0            5

   Central LMS &
                                    3                 0              4                0            7
   local services

   Central LMS,
   central IT                       2                 0              0                0            2
   & local services

   Provider
                                    1                 0              3                3            7
   unspecified

   Total for each
                                   14                 2              9                3            28
   form of support

In terms of provider, responses were divided fairly evenly between each of the four alternative focuses:
support as centrally provided, as locally provided, as a combination of the two or unspecified, although
the unspecified category was the largest of the four. Central IT was mentioned in only a couple of
cases, and then in combination with central LMS support and local services. In relation to form of
support, the field is more clearly dominated by one particular focus, support as people (14 out of 28
responses), though the broader focus of technical resources plus people is not far behind. Responses
speaking in terms of technical resources alone or leaving the form of support unspecified are even
smaller still. (2 and 3 responses respectively).
There does not seem to be any correlation between differences in the way comments were focused and
the differences in the kinds satisfaction and dissatisfaction that these comments expressed. The most
salient tendency seems to be towards focus on the people side of ICT support rather than the
technological, but what this might mean can only be speculated. Looking at central verses local
breakdown in relation to the distribution of positive and negative responses is likewise a dead end.
There is no discernable pattern to the positive and negative responses at this level, which would not be
unexpected, given the widely varying mixture represented by the local side.
The few patterns that do emerge regarding central and local services are solely related to the influence
of specific factors within those services and the evidence is still fairly thin due to smallness of
numbers. There are two main findings to report in this area. The first is that the positive comments
concerning central and local support both seem to cluster around a specific zone -and people- in each
area. For central support, the LMS Helpdesk is responsible for the bulk of positive mentions by survey
contributors.




Use of web-based resources in EITA undergraduate teaching                                                   22 of 32
Table 3.4 Aspects of central LMS support mentioned favourably in responses on overall quality of service

                          Central support                           No. of favourable
                         features mentioned                             mentions

                            New software version                              1

                            Helpdesk responsiveness                           6

                            Training                                          1

For local support, the standout performers are ICT support staff in the SIT and EIE. The five
favourable mentions given by contributors speaking on the support level item concern services
provided by local support staff within these two faculty schools. The numbers of course are small but
consistent with previous indications of differences between EIE and SIT on the one hand and the other
four faculty schools on the other.

Table 3.5 Aspects of local support mentioned favourably in responses on overall quality of service
                                              Location of service mentioned                 Total
           Local support                        SIT                EIE                     mentions
           features mentioned
           Staff responsiveness                    2                      1                     3
           Support staff skills                    1                                            1
           Software (WEBfast)                                             1                     1

The second finding concerning specific providers of elearning support relates to a possible limitation of
the central service. A compilation of critical mentions concerning central support shows a list
dominated by what seem to be software issues, in contrast to the favourable mentions focusing on
people. "Great customer service, not so thrilled about the product" would seem to be the underlying
message.

Table 3.6 Aspects of central LMS support mentioned critically in responses on overall quality of service
                    Central support features                        Number mentioning
                    mentioned critically                                critically
                    Closed to outside users                                       1
                    Lack of independent help materials                            2
                    Limited capabilities of software                              1
                    Options restricted to single
                                                                                  1
                    software solution
                    Software bugs                                                 2
                    Software usability                                            5
                                                            Total                 12

The real story behind the tally of critical mentions, however, is not so much technology verses people
as what happens when they tangle together. The most frequently mentioned point of criticism, the
usability of the LMS environment, represents an important point of overlap between technical and
human factors. The usability issue was essentially the same in each case, and about people as much as
technology. The issue started with the complexity of the LMS software but also involved the pressures
that this complexity created for the user in their working environment and the absence of human
support response. The core of the usability complaint was there were significant time costs in the use of
the LMS technology and these costs were difficult to sustain if they had to be borne alone, which was
the common experience
           ‘It’s just that it’s very hard to sell to other academics, just because you have to invest the
          time. If only I was able to explain in 10 minutes . . . '




Use of web-based resources in EITA undergraduate teaching                                                   23 of 32
          ‘It was just assumed that I know that WebCT, yeah, it's a learning content management
          system and so on. How to use it was like "Go and find out"'.
While there was no direct criticism of the human support side of the central LMS service, there was a
sense that it could have been better distributed.

Questions raised concerning how a satisfactory level of service might
be defined
The strongest overall trend in responses to the question of how satisfactory the level of service was a
tendency to treat the question as in some way problematic. In some cases, the response directly
described a difference of views: academic on one side, service provider on the other. In other
responses, it was rather assumed that clashes of this sort were an inevitable part of the relationship and
that all the academic could do was try and minimise it by curbing any expectations they had. They
understood that the support services were very busy and might have more important things to than
provide the service that the academic actually needed. They didn’t want to be too demanding. In other
responses it was more a matter of bewilderment in trying to understand what the service providers’
expectations actually were. Altogether, the number of responses tending to make the service level
question in some way problematic represented over three quarters of the total, 22 out of 28. Six
different ways in which the question could be problematic were identified in the 22 problematic
responses. These were in effect six sub-questions within the overall question of what satisfactory level
of service might be.
     1.    What is the basic level of support services that academic teaching staff should be able to
           expect as a bare minimum in use of ICTs? To what extent should basic service expectations
           including making access quick and easy, not simply possible with perseverance? Should it
           include auxiliary resources like instruction guides? Should it include help in developing
           resources or should it be limited to providing spaces to keep them and tools for building?
     2.    What level of co-contribution can be reasonably expected from teaching staff in accessing
           services supporting the use of ICTs? What is reasonable maximum time to expect academic
           time to spend in securing working access to a basic teaching website? What is a reasonable
           amount of time to expect from staff in turning a university web space into an effective
           teaching resource without additional outside support? A sense of time requirements that were
           excessive or impossible to meet or insufficiently appreciated was an element in large
           proportion of responses, shown by numbers raising this issue in the frequency table below.
     3.    What level of support for technological diversity can be expected for academics using ICTs
           in their teaching? What are the resources to be supported apart from central learning
           management system? What is the place of local systems performing essential functions that
           the central system cannot replicate at an equivalent level? What is the place of competitive
           benchmarking of systems where the only effective evaluation is experience over time? What
           is the place of email and other web services used in teaching but traditionally managed under
           the general IT portfolio rather than learning and teaching support? Can class discussion lists
           on the university email system expect the same level of support as discussion forums on the
           university LMS? Can we envisage some sort customisation of central support to the varying
           circumstances of different disciplines, schools and faculties? Or is "one size fits all" a
           necessary condition of university wide service?
     4.    What level of support for user diversity can be expected in services supporting the use of
           ICTs in teaching? What sorts of academic user can support services reasonably expect to
           cover? What sort of support can be expected for those at the margin of the current system:
           newly recruited staff, contract staff, staff lacking background technical skills or background
           institutional knowledge that everybody else takes for granted? What sort of support can be
           expected for technological diversity?
     5.    What level of active engagement with academic users can be expected from ICT support
           services? Is ICT support primarily a reactive service, waiting for problems to be reported, or
           should there be a proactive element as well, actively seeking out user and needs and issues as
           they occur?
     6.    What level of user perspective can be expected in the way the level of ICT support for
           academic users is measured? Is it reasonable to expect standards to be set not simply in terms




Use of web-based resources in EITA undergraduate teaching                                               24 of 32
           of what the service provider does on behalf of academics, but also in terms of what
           academics are able to do for themselves as a result of the service they've had?
The extent to which these questions were raised in academic staff responses on the question of overall
satisfaction is shown in Table 3.7 below.

Table 3.7 Responses raising questions regarding how satisfactory service level is defined?

    Questions                                                                                Responses

                                                                                                19
       1. What is the basic level of support to be expected?
                                                                                                14
       2. What level of co-contribution can be expected from teaching staff?
                                                                                                 5
       3. What level of support can be expected for technological diversity?
                                                                                                 5
       4. What level of support can be expected for user diversity?
                                                                                                 8
       5. What level of user engagement can be expected?
                                                                                                 7
       6. What level of user perspective can be expected?
Most frequently raised questions (Table 3.7) are those concerning basic level of support to be provided
and the effort expected from academics in return. Those raised least frequently are those concerning
diversity either of technologies or the people using them. Standing between are questions how far
support services should go in seeking to engage with users and incorporate their perspective in the way
services are designed.
The broad tendency that emerges from the breakdown of responses in terms of problematic verses non-
problematic approach is a tendency to meet the question of ‘How satisfactory overall is the level of
support that you receive in using web resources for teaching’ by bringing up yet another unresolved
question in various forms: ‘How are we supposed to judge the level of support as satisfactory or not?’
The underlying message about the current support level is that it is not yet at a stage that allows
meaningful ratings of satisfactoriness to given, lacking the basic pre-requisite for such ratings to be
given, a sense of settled norms against which such things might be judged. The underlying need that
the responses point towards is for a set of explicit service level specifications as a guide for judgements
in the future. In the absence of explicit standards, 'normal' level of support will simply mean whatever
support is received, which means effectively no support at all for a number of academic staff.
Part of the purpose of having the survey in the first place was to gather some ideas on what standards
would be appropriate for the Engineering-IT-Architecture teaching area but survey responses make it
clear that simply getting people to give us their ideas is not in itself going to provide a meaningful
answer, when the ideas simply lead us back in a circle again to the question of what standards are to be
expected. What the responses do provide is a stronger starting point in terms of specifying the kinds of
question that ICT support standards will need to address. The six questions of minimum service, co-
contribution, technological and user diversity, user engagement and user-perspective provide a
checklist for evaluation of any service standards that may be proposed.




Use of web-based resources in EITA undergraduate teaching                                                25 of 32
Recommendations
The findings show inconsistencies at three levels of infrastructure support for use of web-based
resources teaching: base level access for students, base level support for academic staff and more
advanced support for web-resource development.
Base level student access:
     •    Consistent access to website support is only found in two out of six faculty/schools.
     •    Basic standards of website content are consistently met in only one of six faculty/schools.
     •    Navigation pathways enabling students and staff to locate UoS websites are fragmented,
          arbitrary and sometimes missing altogether.
Base level staff support:
     •    Differences in availability of ICT support from one faculty/school to the next.
     •    Absence of commonly understood standards of support for academic staff in use of ICT
          resources.
     •    There are substantial groups of academic staff not reached by current support services and
          unable to make effective use of existing resources while their specific information and
          training needs remain unrecognised and unaddressed. Groups 'falling through the cracks'
          include new academic staff, contract staff, staff on first year units of study, staff in teaching
          areas with little or no history of previous website use and limited opportunities for developing
          web use practices through peer example.
     •    There is a high proportion of staff reporting difficulty with basic LMS access and site
          management tasks. The number of staff reporting problems that prevent them from making
          effective use of LMS sites is equivalent to the number of current active users.
     •    Support services involved in use of web-based and related resources are fragmented and
          lacking in coordination.
     •    Poor integration of basic teaching information systems adds to complexity and workload
          while hampering quality assurance. Assessment information processing is split between
          central database, the LMS ‘Gradebook’ and various local systems running on faculty networks
          or individual computers.
Support for resource development
     •    UoS websites are often seen as playing only a minor teaching and learning role as document
          repositories, only vaguely connected with the learning needs and teaching strategies of the
          courses that the sites are supposed to support.
     •    Development of new learning materials happens through sporadic individual efforts, isolated
          from each other, without sense of continuity or common purpose and with little recognition of
          value that these efforts may have for student learning and the institution.
     •    Support currently provided for materials development tends to focus on inputs for specific
          learning materials projects rather than assistance in making best use of available resources
          generally. In the absence of a framework that recognises the value of learning resources
          generally, support focusing on the input side of learning materials only reinforces its low
          institutional status by treating them simply as running costs rather than assets.
     •    Evidence needed to judge the real impact and effectiveness of web resources is not collected
          in any systematic way.
     •    Lack of opportunities to view and compare resources used by others makes it difficult to
          develop any sense of what is best practice in the use of web-based resource.
Addressing these issues requires coordinated action from both the faculty and central university
services. How that co-ordination happens will be for negotiation between them. The following
recommendations focus on the agenda items that they will need to co-ordinate about.




Use of web-based resources in EITA undergraduate teaching                                                26 of 32
Base level student access
Recommendation 1. Access to web-resources should be maintained on a consistent basis for all units of
study through common list pages for sites used in each faculty/school and through systematic checking
of sites available against Units of Study in session every semester. Remedial assistance for cases where
sites are not available would be part of the checking process.
     1.1 Access to some sort of supporting website should be standard on all units of study, with
         choice of using LMS or non-LMS platforms up to the unit coordinator.
     1.2 Access to unit of study website via central LMS portal should be standard for all units of
         study regardless of whether the main website exists in the LMS itself or outside. Where there
         are non-LMS sites already in use, the LMS site would simply provide a link from the central
         university portal as is already done in a number of cases. Where there is no website currently
         in use, the LMS site would display a message informing students that the unit relies on other
         communication channels at present and that they should contact their lecturer for further
         details. This message can be replaced as soon as staff are ready to start making more use of
         the site. The default LMS site would also contain links to unit of study outline and university
         timetable page. Procedures for setting up and maintaining new LMS sites would be subject to
         negotiation with Sydney eLearning.
     1.3 Access to websites for units of study offered within a particular school via index page on
         school website should be standard for all units of study. The list would identify units with
         LMS base sites along with outside sites as is currently done on the Civil Engineering link
         page.
     1.4 The practice of having shared link pages for non-LMS based units of study across whole
         schools should be discontinued.
Recommendation 2. Content of Unit of Study websites should be maintained on a consistent basis
through systematic checking each semester to ensure that the web resources for each unit of study
include at minimum a well organised, clearly labelled set of basic course documents including unit
outline, assessment details and list of topic materials consistent with the scope of unit. Remedial
assistance would again be provided as part of the checking process.
     2.1 Quality assurance checking needs to focus on ensuring that staff have the necessary means of
         meeting the expected minimum standard without exceptional time or effort, rather than
         achieving compliance by force.
     2.2 Website quality assurance needs to include positive measures of recognition and
         encouragement for sites with qualities over and above the minimum expected standards.
         Measures might include letters of commendation, inclusion of sites in ‘Featured Website’
         section of UoS website link pages on faculty/school websites, or use of ‘star’ symbols on
         website link pages to flag sites of interest or outstanding value.
     2.3 Quality assurance processes currently confined to LMS sites at the stage of preparation need to
          focus more broadly on course sites generally in use and course sites at the stage of active use.
          Non-LMS sites including sites currently under local password should be included with
          consent of academic owners.

Base level staff support
Recommendation 3. Minimum standards of support for academic staff in the use of ICT across the six
faculty/schools should be negotiated with the two main central providers, the university’s ICT service
and Sydney eLearning. Agreed standards should be specified in Service Level Agreement with support
providers, as is normal IT practice in regulating service standards. Agreed standards for staff support
should be at a level commensurate with the minimum standards website use expected under the previous
two recommendations.
     3.1 Service agreements should be communicated in the form of explicit service quality
         statements addressed to students and staff. The statements would be published prominently
         on school websites and in orientation material presented to commencing students. Statements
         would be phrased as demonstrations of practical commitment to ensuring high quality
         learning environment for students on the one hand and high quality teaching support for




Use of web-based resources in EITA undergraduate teaching                                               27 of 32
           academic staff on the other. Statements would set service quality expectations not just for
           electronic resources but also for the learning environment and teaching support as a whole.
     3.2 Service quality statements addressed to students would set minimum expectations
         concerning:
           •    Global uninterrupted availability of electronic resources including email and websites on
                all units of study through the university portal
           •    Function of website as central resource hub and range and quality of resources to be
                found there.
           •    Availability of technical assistance and response to service breakdown.
     3.3 Commitments regarding the level of service provided for students would be matched by
         equivalent commitments regarding support for academic staff in meeting student needs.
         Service quality statements addressed to academic staff would set further expectations in
         relation to:
           •    Prompt execution of routine requests for website set up and activation
           •    Maximum time for basic access & site management procedures.
           •    Integrated web-access to information resources essential for effective use of websites and
                electronic communications, in first place full student records including current contact
                details and previous assessment performance.
           •    Choice of non-LMS as well as LMS based resources or both as appropriate for teaching
                context.
           •    Specific services for support and recognition of innovation in use of web-based and
                related resources in teaching.
           •    Active engagement of support services in identifying and addressing support needs of all
                teaching staff regardless of employment classification or teaching area
Recommendation 4. A pro-active facilitation program needs to be undertaken directed towards
academic staff having difficulty in making use of the university's web-resources as well as those who
have no access at all. The ‘Teaching Websites Made Easy’ program should aim to streamline access
procedures for teaching websites and extend participation in the use of web resources by actively
seeking out staff who experience difficulties in accessing the resources that the university and the
faculty make available.
     4.1 Downloadable help guides addressing needs of intending LMS users unable to attend
         sessions. The guides would need to be compact, easy to follow and closely targeted to cover
         basic site management tasks encountered by beginning users.
     4.2 Streamlined LMS site activation procedures. Precise arrangements would be part of service
         agreement negotiations with Sydney eLearning. The overall thrust would be minimisation of
         delay at start of semester, automation of site activation and shifting of quality assurance away
         from compulsory start of semester compliance towards focus on mid-semester site checking
         and site design consultant services.
     4.3 LMS orientation needs to systematically target areas of low LMS use, previous LMS users
         who have given up on the system after early experiments, as well as new staff. Training
         format would probably combine occasional workshops with ‘at elbow’ sessions and use of
         Quick Start materials from 4.3.1 above. The approach will need to be flexible and focused on
         how the system can best meet the needs and expectations of academic staff, not just getting
         academic staff to meet the needs and expectations of the system. Staff need not only to be
         told but also to see that provision of basic website support can be done without significant
         addition to workload.
     4.4 Orientation to WEBfast should be provided alongside LMS orientation in faculty/schools
         without strong local content management systems (Architecture and AMME in particular).
         There should be chance to decide which resource would be most appropriate for the
         particular context.




Use of web-based resources in EITA undergraduate teaching                                                28 of 32
Recommendation 5. Focus for technical improvement in central university web systems for teaching
and learning should be first of all on achieving integrated management of assessment information. This
is currently split between student administration systems, the central LMS and local school and faculty
systems.
     5.1 As a first basic step, the ‘Gradebook’ screen on central LMS websites should include student
         numbers to facilitate data entry for marks recorded against student numbers rather than
         names. This is the single most useful improvement that could be made in central LMS
         functionality at present, but still an initial.
     5.2 Creation of an effective integrated assessment information will require more just tweaking of
         data items fed to the Gradebook. What is needed is an actual assessment information
         management system. Requirements of this system would include elimination of double
         handling of student results, issue tracking for contested grades and special consideration
         cases, support for varied grading sheets and grade weighting formula, long term monitoring
         of individual and cohort progress against outcomes of units and whole degree and mapping
         of pedagogical alignment of assessment tasks, learning outcomes and generic attributes.
     5.3 These assessment information management requirements should be at the forefront in any
         discussion of assessment related web tools for purchase by the university, such as ‘e-
         portfolios’. The focus of evaluation in dealing with purchases decisions such as e-portfolios
         should not be on finding which is the ‘best’ product in the genre but rather finding out if
         there are any at all which deal with assessment information at the comprehensive level
         required. If no such tool exists, there would be a good case developing one independently.
Recommendation 6. A full-time ICT in Learning and Teaching coordinator position should be created
on a two-year project grant basis with the task of developing longer term systems and capabilities in
the management of ICT use in teaching & learning.
     6.1 Integrated delivery of teaching support services would be facilitated by locating support
         personnel as close as possible to point of delivery and each other in the individual
         faculty/schools. It is not expected that this kind of resource distribution would happen easily
         or quickly but the principle of having logistical support for teaching delivered from a
         common source based as close as possible to the teaching itself should at least be well
         understood.
     6.2 A critical part of the coordinator’s task would be the development of local ICT coordinator
         roles at faculty/school level. Academic staff in every faculty/school should have access to an
         onsite person with effective oversight and control over ICT resources needed for teaching
         and learning purposes.
     6.3 The provision of the new services, resources and people recommended here must be done
         without additional charge on the existing resources of the faculty and as far as possible by
         drawing upon external resources or by improved focus in use of resources already available.
         The task of harnessing and focusing these resources will be the challenge of the ICT
         coordinator.

Development of new learning resources for specific student needs
Recommendation 7. Support for development of innovative materials and approaches in teaching
should be targeted in the first instance towards systematic evaluation of innovative materials or
approaches that already exist from a perspective of wider application and of cases where innovative
approaches or materials need to be applied. Establishing an evaluation framework for teaching
innovation generally is an essential precondition for ensuring value and recognition for individual
innovators. Those who can demonstrate successful innovation in practice and those who can help in
identifying needs and opportunities for future development are the core assets of an innovative teaching
culture and should be primary focus of support for innovative teaching work.
     7.1 The primary form of assistance for effective teaching innovation should be an on demand,
         real-time evaluation service that reports in reasonable time (no more than two weeks) with an
         assessment of the effectiveness of the targeted resource or approach against known
         benchmarks plus proposed plan for either development or dissemination according to the
         needs or opportunities identified. Evaluation reports would be provided in a documentary
         form suitable for teaching portfolio and grant applications purposes. Evaluations would



Use of web-based resources in EITA undergraduate teaching                                              29 of 32
           thereby carry substantial intrinsic value in themselves, apart from any further assistance that
           might be given as a result of the evaluation.
     7.2 Procedure for requesting assistance with development of existing or new learning materials
         should not be more complicated than a statement identifying the issue for which assistance is
         required and its teaching context. There should be no requirement for a specific project
         proposal or any other threshold barrier in the way of academic staff presenting needs or
         resources for investigation.
     7.3 Decisions regarding whether and when to proceed with particular proposals emerging from
         particular evaluation cases will still be made according to current priorities and resource
         availability as these decisions always have been, but with priorities and resource availability
         informed and strengthened by the ongoing investigation of needs and resource capabilities
         provided by the evaluation service.
     7.4 The area of formative assessment is suggested as initial focus of learning materials evaluation
         and development due to store of existing materials and shared staff interest in this area.
         Within the formative assessment area, the Reflect tool developed in SIT appears most likely
         to find successful applications in other contexts. Investigation of Reflect tool as potential
         solution for wider formative assessment needs should be undertaken at the earliest
         opportunity.
     7.5 Development of a resource repository would be part of the work of the evaluation service.
         The repository should not be seen as answer in itself to the problem of promoting innovative
         practice and should be closely coordinated with existing repositories, starting with the
         Sciences and Technology Spotlight website.
     7.6 The open access approach adopted by the majority of non-LMS UoS websites should be
         maintained and extended to LMS-based websites by making available generic guest logins
         for LMS sites in each faculty/school as well as a generic guest login for sites across the
         faculty as a whole. LMS site owners with particular reasons for keeping resources under
         closed viewing would have the choice of opting out from the open access principle, as is the
         case for staff with local sites at present. However, open access would be the default
         approach, rather than closed access as currently happens with LMS based sites.




Use of web-based resources in EITA undergraduate teaching                                                30 of 32
References

Joint Information Systems Committee (JISC) (2007) Student expectations study.
http://www.jisc.ac.uk/publications/publications/studentexpectations [viewed 28 January 2008].
Kennedy, G. E., Judd, T. S., Churchward, A., Gray, K. & Krause, K.-L. (2008) First year students'
experiences with technology: Are they really digital natives? Australasian Journal of Educational
Technology, 24(1), 108-122. http://www.ascilite.org.au/ajet/ajet24/kennedy.html [viewed 28 January
2008]
Caruso, J. B. & Kvavik, R. (2005) ECAR study of students and information technology 2005:
Convenience, connection, control, and learning. ECAR, 6.
http://connect.educause.edu/Library/ECAR/ECARStudyofStudentsandInf/41159 [viewed 28 January
2008]
Philip, R., Lefoe, G., O’Reilly, M. & Parrish, D. (2007) Community, exchange and diversity: The
Carrick Exchange. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore
2007. http://www.ascilite.org.au/conferences/singapore07/procs/philip.pdf/ [viewed 28 January 2008]
Scott, K. (2006) eLearning focus group: Schools of Civil; Electrical and Information; and Aeronautical
Mechanical & Mechatronic Engineering, 19 October 2006.
Scott, K. & Mahony, M.J., (2007) Investigating impacts of elearning projects: Do they improve
collaborative teaching developments? AARE 2007 Conference. Fremantle.
http://www.aare.edu.au/07pap/sco07267.pdf[viewed 9 February 2008]
The University of Sydney (2005) eLearning Audit. College of Sciences and Technology, The
University of Sydney.
The University of Sydney (2001). Academic Board Resolutions: The Management and Evaluation of
Coursework Teaching. http://www.usyd.edu.au/ab/policies/Mngt_Eval_Cwk_Teaching.pdf [viewed 8
March 2008]




Use of web-based resources in EITA undergraduate teaching                                            31 of 32
Appendix

Staff survey questions
     1.    What undergraduate units are you
           currently involved in teaching?




     2.    How would you classify the
           teaching-learning focus of each in
           relation to the 3 categories shown?




     3.    Is there another way of categorising
           units in your faculty/school/discipline? What is it?




     4.    What do you see as the main learning challenge for students in tackling these units?




     5.    What do you see as the main challenge for you as a teacher in helping students meet the
           demands of these units?




     6.    What role do online resources play in addressing the teaching & learning challenge(s) that
           you’ve mentioned? (Note that ‘online resources’ here includes any unit material provided to
           students over the web – not just WebCT sites – not just Uni of Sydney).




     7.    Do you see any gaps in online learning support? Is there anything missing that you’d like to
           see provided? Is there anything provided that you’d like to see improved? What is your wish
           list for elearning in your school/faculty (in priority order if possible) for students and for
           teaching staff? Please don’t hold back!




     8.    How satisfactory, in your view, is the level of support that you currently receive as a teacher
           in addressing your students’ learning needs online? (All observations welcome on adequacy
           of the current elearning support for teaching staff!).




Use of web-based resources in EITA undergraduate teaching                                                32 of 32

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:10/6/2011
language:English
pages:32