ltsn reportnewsletter article objects for osces by lindash


									Vicki Dale, Gill McConnell, 15 July 2005

Subject Centre Funded Mini-project: Objects for OSCEs (Generating Reusable Learning Objects (RLOs) from existing CLIVE courseware to support Day One Competencies) Final Report, July 2005
Purpose of the project This was a pilot study, kick-started by Subject Centre (formerly LTSN-01) funding, to develop a limited number of reusable learning objects (RLOs) for veterinary education. In particular we wanted to create RLOs targeted at the veterinary benchmark „Day 1 competencies‟ for new graduates, with the potential for use in OSCEs (Objective Structured Clinical Examinations). The RLOs were to be extracted from over 70 existing CAL packages created by CLIVE (Computer-aided Learning In Veterinary Education) This approach, we argued, would be more cost-effective than creating RLOs from scratch, which we estimated to be £1k-2k per object. Overview / Summary Consultations with academic and library staff, the Centre for Educational Technology Interoperability Standards (CETIS), and reference to the available literature on RLOs provided the framework for development work, beginning in December 2003. Objects were extracted from Authorware packages, to form smaller Authorware subunits and/or were recreated using Dreamweaver and Flash, to create web pages. The RLOs, in both formats, were then packaged using the JISC RELOAD editor, and uploaded to the University of Edinburgh‟s Intralibrary repository, before being tested in Virtual Learning Environments (VLEs) such as Moodle, WebCT and Edinburgh‟s in-house system EEVeC. Twenty-four RLOs were created from these packages: Anaesthetic Circuits, Diagnostic Procedures in Dermatology, and General Pathology Test and Tutorial. Work is continuing to identify the potential uses of these objects, prior to creating more RLOs from other existing CAL programs. This report documents the methodology for creating RLOs and importing them into a repository and different VLEs. Getting started Work on this project began in December 2003. Initially, all available CLIVE packages were mapped onto RCVS/QAA Day 1 competencies. We then consulted local academic and library staff. An important first step was to obtain permission from authors to re-work existing CAL packages. All authors were agreeable to this, but pointed out that some of the content would need to be reviewed and revised to bring the packages up to date with current knowledge. A number of CAL authors had moved on to different institutions, and had to be contacted at their new affiliation. Librarians were able to comment on classifications and vocabularies – very important with regards to the meta-data used to describe RLOs. We also had consultations with the Centre for Educational Technology Interoperability Standards (CETIS), with regards to the format of RLOs – whether SCORM1 or IMS2 compliant. We also found the


IMS Global Learning Inc publishes internationally agreed specifications within the scope of distributed learning Internet environments: 1

Vicki Dale, Gill McConnell, 15 July 2005

CETIS website to be an invaluable source of information ( and would suggest that anyone supporting the development of e-learning at their institution takes a look. We also referred to the available scientific literature on RLOs to guide and inform our work. At the start of this project, there were only a handful of relevant references, but over time the literature has grown as the wider scientific community has taken note of this relatively new aspect of e-learning development. Defining the Objects As well as technical issues to consider, there were also pedagogical aspects to take into account. First of all, we had to decide what a „reusable learning object‟ consisted of. A look at the literature revealed that there was no widely agreed definition. Wiley (2002, p.6) defined a learning object as “any digital resource that can be reused to support learning”. This was good enough for our purposes! Although outwith the scope of this particular e-learning project, we also recognise that reusable learning objects could be paper-based, or even physical objects, particularly objects that might be included in OSCEs. A key issue in defining our electronic RLOs was the level of granularity, or complexity, of an object. Smaller objects (for example a photograph or a vocabulary entry) can be reused in many different ways. Larger objects (for example a self-contained quiz or mini-tutorial) are less flexible for reuse. A balance needs to be struck, between the reusability of an object, and the time required to create it and provide the appropriate meta-data. Creating the Objects

Initially we attempted to re-create different objects using different software. Content and context-free objects, such as ConceptTutor: mapped onto CLIVE “template” applications, but were rather too complex for adaptation. We decided to stick with Macromedia software for creating the objects. For example, a small section (one or more „screens‟) of a larger CAL package created in Authorware (v7.0) could either be isolated and kept in Authorware format, or be re-designed as a web page in Dreamweaver (MX 2004), using Flash (MX 2004) to replicate small interactions, such as drag-and-drop quizzes. We used the following methodology for creating and delivering RLOs: 

Sub-divide CAL programs into smaller units: o Either keep in Authorware format  Publish the Authorware file for local use (.a7r) and for web (.aam). Then run the Authorware Learning Object Content Packager, making the .htm file in the web version the „entry point‟ for the learning object. We chose to „save the manifest only‟ rather than packaging the object in Authorware, because we did not add the metadata in Authorware. Instead we opted to open the manifest in the RELOAD editor ( and „zip‟ the contents from there). The HTML pages can then be organised using the RELOAD editor, in which you create a manifest file. The manifest file keeps track of all the resources used (e.g. html pages, graphics), as well as how they are organized, and metadata. All this


Or recreate Authorware file as a series of HTML pages (using Dreamweaver) 


ADL SCORM is a suite of standards that allows the aggregation of complex objects, branched paths through content and learner tracking. SCORM packages are intended for use in Learning Management Systems: 2

Vicki Dale, Gill McConnell, 15 July 2005

files can then be „zipped‟ in RELOAD. Users who do not have access to a VLE or an object repository can also preview RLOs in the RELOAD player.   Upload „zipped‟ objects into repositories and add metadata (if not already added in RELOAD). Download and import objects into VLEs for testing.

All the RLOs created were essentially „simple sequences‟ – linear constructs. We did not test aggregated objects.
Estimated time to create objects

The time taken to extract new objects from existing Authorware packages was anywhere between an hour and several days, depending on the size, format and complexity of the new objects. Authorware - It took approximately an hour to extract a small mini-tutorial (several „screens‟) from a larger Authorware package when all the text and graphics were already embedded in the document, and the extracted subunit was kept in Authorware format. For packages with external text and images, these components had to be pasted into Authorware before the subunit could be published for web delivery. On average this took about two and a half hours per mini-tutorial. When extracting objects from the original CAL packages, some re-programming was required to maintain functionality within the subunit. Interface design was also improved for both aesthetic and functional reasons, and image quality was improved where appropriate. Dreamweaver/Flash - Re-creating a single object (or „screen‟) in Dreamweaver/Flash format took about two hours compared with 20-30 minutes to isolate and repackage the object in Authorware. However we found that we got quicker at designing similar interactions – for example it took 1½ hours to re-create a drag-and-drop interaction in Flash, but only 50 minutes to do a second, similar one. Re-creating interactions in Flash is time-consuming but offers the advantage that users (and network administrators) do not need to keep downloading the latest Authorware web plug-in software (although Flash plug-ins, required by a high proportion of websites, still need to be downloaded). Recreating an object as a series of HTML pages generally took longer than isolating a subunit in Authorware, because new design and presentation issues arose, however the result was easier to search, print and update. We used common cascading stylesheets for packages with related content e.g. all dermatology objects used a specific design theme.
Naming convention for manifest files

We developed a specific naming scheme for our manifest files: “MANIFEST” – [Day 1 competency] – [School] – [2-letter CAL package ID + 00N] – [file format] – [interoperability standard] e.g. MANIFEST-B.15-GLA-GP001-HTM-SCORM is a SCORM object, created as a series of HTML pages, (the first object) from the General Pathology CAL package, at Glasgow, corresponding to Day 1 competency B1.5 .
Importing the objects into repository and VLEs

Around fifty objects are now available for use within the Intralibrary repository, which will be uploaded to the Virtual Learning Environments of the two schools – Moodle in Glasgow, and WebCT and EEVeC in Edinburgh. Adding RLOs to Moodle and WebCT was very straightforward – the content and manifest were packaged as a zip file in RELOAD, and uploaded as a SCORM activity within an individual course. As yet, there is no direct RLO import facility in EEVeC, but RLOs can be submitted through the developers.


Vicki Dale, Gill McConnell, 15 July 2005


Around fifty objects, covering anaesthetic circuits, general pathology, dermatology, and diagnostic imaging, available for use within the Intralibrary repository, and uploaded to the Virtual Learning Environments of the two schools – Moodle in Glasgow, and WebCT and EEVeC in Edinburgh.
Conference presentations

World Veterinary Anaesthesia Conference, 2003, Knoxville, Tennessee – “Avoiding the pain in developing Anaesthesia Computer-Assisted Learning for veterinary undergraduate students.” (poster), V.H.M. Dale and G. McConnell. Association for Medical Education in Europe (AMEE) Conference, 2005, Amsterdam – “A case study in creating and using reusable learning objects” (short communication), G. McConnell, V. Dale and C. Newlands. Issues arising from the project
Quality Control

Maintaining motivation to re-purpose „old‟ materials (up to ten years old in cases) was difficult at times. Although it takes longer, and costs more money, it always seems more exciting to create new materials from scratch. Packaging and cataloguing objects satisfactorily took several iterations and in the worst cases two to four hours per RLO. Peer review of content and learning design by teachers and educational technologists and evaluation by end-users remains critical so that RLOs maintain educational value.

RELOAD (version 2.02) is an excellent tool for creating and previewing RLOs, however it is subject to some limitations. It is more stable on Windows XP than earlier windows platforms. For example, we experienced a problem with RELOAD continually crashing in Windows 2k that we have been unable to resolve.

This is a joint Glasgow-Edinburgh collaboration, a continuation of the partnership arising out of the CLIVE project. The University of Edinburgh uses Intralibrary as their repository (LORE – Learning Object Repository for Edinburgh), to which veterinary objects from Glasgow and Edinburgh have been added arising from this project. The University of Glasgow did have access to the LRC3 repository, but its license has expired. This exercise has flagged the importance of having a shared, long-term repository between the schools. The JORUM repository, to be launched later this year, offers this possibility, at least within the UK. There also exists the possibility for sharing RLOs with other veterinary schools on an international scale, in which case an international biomedical repository would be required, with appropriate licencing models available for educational and commercial use. LORE is a pilot project under development, so usability improved as the project progressed, and further work is needed to optimise the interface for users. For example, copying metadata from a published object required the object first to be „unpublished‟, metadata copied, and then republished. Desirable enhancements would include thumbnails for search results and recognition of commas as delimiters for keywords.

For this project, for convenience, we took the decision to add meta-data (LOM 6.2) to the objects as they were uploaded to the Intralibrary repository, rather than within RELOAD. The information provided within Intralibrary includes mandatory and recommended fields – title, description, identifier, language, keywords, aggregation level, lifecycle, contributors, format, and copyright restrictions.


Vicki Dale, Gill McConnell, 15 July 2005

The RLO was then classified (discipline as defined by the Library of Congress, and educational level by those set by the Scottish Curriculum Qualification Framework). We mapped the veterinary curriculum to the following JORUM SCQF levels:
   

Level 8 = Higher Diploma = Years 1 & 2 Level 9 = Ordinary Degree = Year 3 Level 10 = Honours Degree = Year 4 Level 11 = Masters =Year 5

The meta-data were comparable with the profiles offered in RELOAD Much more detailed meta-data could be added, however cataloguing was time consuming and unpopular with developers. Vocabularies for veterinary keywords are not nationally agreed, so this is an area for future work.

There are complex licensing and IPR issues relating to the modification, aggregation and reuse of materials which may have multiple authors. These are outside the scope of the project, however licensing models are being devised for projects such as LORE (Learning Object Repository Edinburgh) and JORUM. Importantly, objects cannot be released for general use until these licences are in place.

Different VLEs handle RLOs in differing ways – not always as one would expect. The handling of more complex objects may well raise further issues. Interpretation of standards is variable. Simple sequences imported as SCORM activities in Moodle played without paging navigation. WebCT (version 4.2) supports the import of IMS and both IMS and SCORM 1.2 were imported successfully, although warning messages are given, for example, this message for an IMS import:
Warning: The package type cannot be determined. Importing as WebCT package type. Success: "Skin biopsy ims" has been successfully imported to your course.

Import facilities may have been smoothed out in WebCT Campus and Vista, however SCORM packages may not be handled as expected in WebCT Campus – they are treated as „black boxes‟, meaning that, for example, timed-release of components is not possible – for this objects should be imported as IMS objects.
‘Objects for OSCEs’

We did not test the use of RLOs in OSCE stations, however we believe that self-assessment objects (e.g. mini-quizzes) may be more amenable to this role than, for example, animations or mini-tutorials. It remains a challenge to get scores from SCORM objects into student records within VLEs.
Learning and pedagogy

Outwith the scope of this particular pilot study is establishing a sound pedagogical framework, within which students will be able to personalise and make effective their online learning through interaction with RLOs within VLEs. Future work will attempt to evaluate the usefulness of RLOs from the learner and teacher perspectives, as soon as a sufficient number of objects become available, with the necessary supporting infrastructure to enable RLOs to become a mainstream delivery mechanism for e-learning. Conclusions Staff and students, when asked informally, liked the idea of having a searchable repository of RLOs, and readily acknowledged potential benefits. However, working with multiple pieces of software to produce and catalogue RLOs is likely to deter all but the enthusiast and the developer. Repurposing is a quick (if tedious) way of generating RLOs, although time/cost issues mean that at present, it is really only worthwhile cataloguing high quality materials. Once a repository is populated, it is then quite easy to use. Out of date content will remain an issue, however the ability for any user to add comments to RLOs may be a powerful way to inform use. Further research needs to be conducted into the educational uses of RLOs.


Vicki Dale, Gill McConnell, 15 July 2005

References Wiley, D. A. (2002). Connecting Learning Objects to Instructional Design Theory: A Definition, a Metaphor, and a Taxonomy. The Instructional Use of Learning Objects. D. A. Wiley. Bloomington, Indiana, Agency for Instructional Technology and the Association for Educational Communications and Technology: 3-23.


Vicki Dale, Gill McConnell, 15 July 2005

Illustrative screenshots These screenshots have been included as a means of illustrating the development process.

The RELOAD editor – resources (files) are dragged into the right-hand pane as the basis for the content package. If desired, „organisations‟ can be added to define the order and structure of presentation of the resources in a player.


Vicki Dale, Gill McConnell, 15 July 2005

RELOAD allows the content package (Learning Object) to be previewed


Vicki Dale, Gill McConnell, 15 July 2005

Screenshot of Glasgow Moodle course with links to RLOs. „Recent activity‟ box on right draws attention to recently uploaded objects. Although this is a course created specifically for testing RLO‟s, in reality these would be embedded within undergraduate veterinary courses.


Vicki Dale, Gill McConnell, 15 July 2005

Screenshot of anaesthetic circuits object within Moodle, uploaded as a „SCORM activity‟. A component of an original larger Authorware package, but with 24 screens, this is one of the larger objects! The copyright information, included as the SCORM „Summary‟ information, is displayed above the object. Learning objectives or instructions could also be added here for the student.


Vicki Dale, Gill McConnell, 15 July 2005

Screenshot of general pathology quiz object in Moodle. Each object, extracted from an original larger Authorware package, is given its own „title‟ page with appropriate copyright information.


Vicki Dale, Gill McConnell, 15 July 2005

A trial page in WebCT with a number of imported objects. These would normally be placed within a customised course page.


Vicki Dale, Gill McConnell, 15 July 2005

A SCORM object made up of html pages, images and video displayed in WebCT. The rather long title is truncated in the breadcrumb bar and, unlike in Moodle, no copyright information is displayed. Navigation is simple and straightforward, with paging buttons provided by the VLE.


Vicki Dale, Gill McConnell, 15 July 2005

Screenshot of LORE (implementation of Intralibrary in Edinburgh) showing some of the veterinary objects in the repository.


Vicki Dale, Gill McConnell, 15 July 2005

Preview of dermatology learning object (Authorware) in the LORE repository.


Vicki Dale, Gill McConnell, 15 July 2005

The same object re-created in HTML format. A common stylesheet is used for a number of objects for consistency and to preserve the „CLIVE‟ identity.


To top