Summary of D2L Demo Day final

Document Sample
Summary of D2L Demo Day final Powered By Docstoc
					Summary – D2L Demo at UWaterloo
Feedback on the meeting held at UW on 24 January 201 1, compiled by Liwana
Bringelson, with input from Jane Holbrook, Sean Warren

This summary is an aggregate of information gathered, questions to be answered and next steps in the
investigation of Desire2Learn (D2L) as a Learning Management System solution for the University of
Waterloo, as successor to UW-ACE based on ANGEL.

On January 24, 2011, Desire2Learn provided a demonstration to the LMS Selection Project team and
invited stakeholders from the UW community. A detailed agenda was developed which included
specific questions, and activities for D2L to demonstrate on a live instance of their learning environment.
This agenda included general areas of: Migrating UW-ACE courses to D2L; Administrative processes;
Teams; Various functionality; and Usage reporting.

There were approximately 30 people participating in this Demonstration Day: 4 from Desire2Learn, 16
members of the LMS Selection Project teams (core and resource, with David Bean and Nick Soave not
able to attend), and 9 other UW stakeholders. The latter additional participants were from CTE, CEL,
Math and Statistics, and the Professional Development program. D2L was represented by four people
for the full day (John Baker, President and CEO; Stephan Meyer, Sales Rep; Mike K, Technical
Engineering; Kenneth Chapman, Sr. Dir for Product strategies). At lunch, four additional representative
joined us ( , National Sales Rep; Dennis Croft , Services, Support & Information Technology (including
hosting facility); Janna Cameron, Senior Usability Specialist; , ??).

Prior to the demo D2L was provided with the Angel export of 6 UW courses (3 representative on-campus
courses, 2 CEL courses including the CEL repository and GEOG 333 which is an on-campus course well-
known to a UW instructor who is blind.) During the demonstration, they demonstrated the automated
conversion process from an Angel export course into the D2L LE, as well as the manual process which
allows importing and exporting of course components.

In planning the day, the UW LMS Selection Project Team developed a detailed agenda (included as
Appendix A), and asked D2L to address 5 specific areas, with a live system:
     Migrating UW-ACE (ANGEL) courses to D2L
     Administrative Processes
     Teams
     Various Functionality
     Usage Reporting

This Summary is organized along the project lines. Each participant was solicited for feedback and Team
Leads for project areas provided a summary of their area. Each Team Lead (covering Functionality,
Integration, Orchestration, Reporting, Reuse (including Migration)) was asked to reflect on four
questions for their area:
    1. What did we learn?
    2. What are we done with (no more information needed)?
    3. What questions are still un-answered?
    4. What would you like to see as "next steps"?

The conclusion of this Summary includes overall next steps, as well as the outstanding issues that the
Project Team needs to resolve in order to solicit further information from D2L. This collection of
information will be used in finalizing next steps in the LMS Selection Project.

Functionality (with input from Sean)
The Demonstration Day was designed to explore specialized functionality, and in some cases, it also
validated expectations of basic functionality. As we have been using UW-ACE (ANGEL) for many years,
our processes have been set up to make that system meet the users’ needs. This demo day provided an
opportunity for the UW team to explore with D2L how their system can best meet our/users’ needs.

Overall, the functionality appears to address the technical functions that we expect. All basic content is
available, including quizzes, dropboxes and discussion forums. In addition, D2L has functionality such as
the ability to define roles in a very granular way that may allow us to do things differently than we have
with UW-ACE. Following are some specific comments for each of these areas.
      Quizzes:
              o Provides all the question types, including and algorithmic question type
              o Can adjust settings for individuals or teams, with most of the options that are currently
                  used: Start and end date, Completion time limits, Password and IP protection
              o Ability to create banks of questions, from which questions can be randomly selected
              o Re-grading questions is confusing and requires lots of steps, although it can be done
      Drop boxes, including TurnItIn
              o Allows expected functionality, such as individual and group submissions, with ability to
                  allow students to overwrite previous submissions
              o TurnItIn is tightly integrated (allows access to TII submission and reports within LMS)
                  which may impact Training
      Discussion Forums
              o Provides general type (individual and group)
              o The display of multiple forums (based on teams) is different from UW-ACE and may not
                  be as convenient for access by instructors as in ANGEL
              o Does not include some types of forums (e.g., Post First, Fishbowl), although D2L seemed
                  open to discussion of possible expansion of their functionality
      Overall content functionality
              o Editor for math equations appears much better than what is available from UW-ACE,
                  including LaTeX support in every multi-line communication field (email, quiz, discussion
              o Ability to easily embed content (e.g., YouTube video)
      Gradebook
             o   Did not get a lot of exposure during the demo; it is possible to post a grade to a field in
                 the gradebook for an individual, or to individuals of a team based on a group
Additionally, D2L has an active research focus toward making their system accessible to people with
disabilities. One of the Senior Usability Specialists who works on issues of accessibility attended the
meeting during lunch, and is willing to review a course with our team and appropriate stakeholders.
Toward allowing all people to work with their systems, they also have recently announced language

Integration (with input from Lorne)
Integration of the LMS requires Administrative communication to and from the Student Information
System (Quest), authentication (LDAP) as well as integration with Academic tools. D2L has worked with
clients using PeopleSoft, Oracle and LDAP, so there don’t appear to be major hurdles with the
Administrative integration. They are working on a SAIP integration that may expand the current
functionality, and increase the “tightness” of the integration (e.g., a “button” in Quest that would pull
marks from LMS. For populating courses, D2L proposes a “holding tank” for student information,; that is
an XML representation or middleware to move data from the UW Oracle database into the LMS.

Regarding Academic Integration, D2L stated they have a “standards first” philosophy of integration, and
explore other integration options according to the following:

       Standards based – using IMS course packages, IMS LIS, LTI, Single Sign-on, etc.
       Generic Interfaces – interfaces created by D2L (e.g., web services)
       Targeted Interfaces – API, subject matter, e.g., Synchronous communication tools (Adobe
        Connect, Wimba, ...), mobile solution frameworks, Respondus (quizzing API)
    Direct Integrations – collaborating with different vendor, e.g., Turnitin, Banner, Lockdown
There are not direct or targeted integrations for some of the UW applications (e.g., iClicker and Ares
eReservers) although D2L representatives appeared to be fairly confident that web services could be
used in making these connections.

If we move forward with D2L as the UW LMS, we’ll need to determine whether UW staff will do the
programming for D2L to integrate with our information and academic systems, or if we contract with
D2L for those services.

Orchestration (with input from Jan)
This area provides the “glue” between the system, users (e.g., staff, faculty and students) and policies.
In general, the D2L representatives communicated a “collaborative spirit”. They also stated interest in
assisting us in the work of supporting learning, and noted that changing systems may provide an
opportunity to re-think current practices and policies. If the D2L LE system is selected at Waterloo, it
appears that current resources could be utilized to support and advance its effective use on campus.
The flexibility of the D2L LE regarding organization, templates and roles may have policy implications.
The system allows for multiple “organizations” from the same instance, and UW staff (e.g., UW-ACE
Operations) will need to determine what organizations should be created, and how they are structured
and differentiated. These implementation decisions and policies can be phased in, first configuring the
system and look-and-feel to support migrated courses, and then developing the policies (and expanding
the functionality) to meet the needs of the UW Community with appropriate support. Similarly, the D2L
LE system may allow for more efficient support of Community Groups (e.g., creation, population and
management of cohort or non-roster-based groups) which may allow for an evolutionary change in the
current UW policies for using the LMS as a collaborative workspace.

From a Training perspective, it appears that documentation strategies used in the past could continue to
work well. A Train-the-trainer structure will allow UW staff to be very familiar with the system, and then
share that information across the campus. A “Where’s Waldo?” document, similar to what was created
during Angel migrations can be developed as a resource to help users find familiar functionality within a
different system. A two-tier approach to training would first address issues for migrated courses and
users, and then develop and expand materials and offerings for the new system. Additional
functionality, such as the rich repository features may allow for pedagogical and time-saving advances
that can be shared with regular LMS users. An advantage to D2L having head offices in Kitchener is the
ease and low expense for getting to them, either at their offices, or on UW campus.

To introduce and transition a new LMS a comprehensive communication plan will be designed. There
are two primary aspects that would be key to the success of this campaign: 1) successful migration and
support for current UW-ACE courses and users (this is imperative regardless of the LMS which would be
selected as the target, and D2L conversion/migration issues will be discussed under “Reuse”) and 2)
specific information about D2L that is attractive to the UW Community. For example, there are many
characteristics people at UW would appreciate:

       Product offered by a company started and run by a former UW student
       Company headquartered locally (in Kitchener)
       System also used at local institutions including the “Tri-University Library” partners (University
        of Guelph, Wilfrid Laurier University) and McMaster
       Philosophy of expansion and inclusion of open source tools through standards-based format
        (e.g., IMS LTI)

Tracking and Reporting (with input from Jane)
The area of tracking and reporting relates to current and future uses of the LMS to support UW faculty
and staff to support teaching and learning at UW. Currently, this type of functionality is used
infrequently, as it is not readily accessible. Recent developments on campus will benefit from data the
LMS can provide. Two examples include the Office of Student Success is interested in student progress
and students at risk, there is Faculty-level interest in Outcomes Assessment and documentation. The
new LMS must make access to data easy and well-organized.
UW-ACE instructors are able to view a limited amount of “usage data” from students, such as “last time
since login”. UW-ACE support staff members are able to extract data from the database and analyze
information about frequency and use of types of activities across courses. With the D2L LE, usage data
appears to be more accessible and robust both within a course and across courses. D2L maintains a
“data warehouse” which is different than the system database, to allow for generating reports without
impacting the production LMS performance.

The system also allows an instructor to send an email to students based on their usage patterns. For
example, settings can be arranged for an instructor to send a “personalized” email to all students who
haven’t completed a certain activity (or completed with a failing mark, or other characteristics). For a
group such as the Office for Student Success, an “auditor” role can be created so that a mentor could
access a student’s usage (or set of students, an athletic team, for example) to review and support their
progress with activities in the LMS.

With regard to programmatic or Faculty-level reviews D2L appears to have the Competency Tool
functionality that would aid in the record-keeping and summarizing of student activities across courses
in specific curriculum. This Tool allows for the ability to achieve competency in multiple ways, and
gather that data into a report. They also have an additional software package, “Analytics”, that provides
ways of looking at the data that might also support this goal. During the Demo Day we did not look at
the Analytics functionality, and the base function of the D2L LE appears to provide robust data summary.

There are still a few unanswered questions regarding data collection and reporting, including how the
system reports student time on an activity, specifically related to closing a browser versus logging out
and clarification about what is required to track technical issues (e.g., quizzes that didn’t get submitted
due to a system blip).

Reuse (including Migration) – (with input from Anuja)
The area of Reuse includes many aspects of the LMS selection and implementation process. The
component of this area at the forefront is “reuse of UW-ACE content” or conversion/migration, because
it is a huge hurdle in moving from UW-ACE to any new LMS. Additional aspects of reuse include reuse of
templates, environment variables, and activities.

With respect to migration, D2L has been working with other academic partners to develop an
automated tool for converting courses saved in the Angel export format (versions 7.3 and 7.4) to D2L.
They have also explored the IMS Common Cartridge format as the input, although it appears that the
Angel export is the “most promising.” During the Demo, they converted one course. They want
feedback from us to learn how to get it closer to what the course is supposed to look like. While the
conversion was relatively quick and the resulting course had the content present, the organization was
awkward and confusing. They promised to convert the remaining 5 courses again and allow UW staff an
opportunity to go through them and provide feedback regarding the accuracy and appropriateness of
the conversion. Following the Demo Day, communication must still go through Procurement as the RFP
has not been closed. The LMS Selection Committee has not heard official news on the progress on the
migration of the courses, though queries have been sent through Procurement (28 January and 1
February). When the course conversions have been completed, the UW Team will review from these

       CEL staff: accuracy of CEL course, impact on objects drawn from the Angel repository and look-
        and-feel of CEL templates.
       CTE staff: on-campus courses for accuracy of organization and presentation of content, as well
        as functionality of advanced/complex learning activities, including quiz formats that draw from
        question pools or algorithmic questions.
       ITMS staff: accuracy of organization, presentation of information and overall functionality
        replicating UW-ACE functionality with an eye to what instructors and students in migrated
        courses will need to know to use the course.
       OPD staff: the GEOG 333 course, with instructor and OPD? staff to evaluate impact on
        accessibility of the migration.

D2L seemed eager to work with UW to further develop an effective way to migrate content from Angel
to D2L. The better the automated conversion tool, the less “fixing” will be required following the
conversion. D2L includes migration in setting up the new system (no extra charge) so it is in their best
interest to understand what is required and to be able to deliver it.

With respect to templates, the ability of the D2L system to allow for multiple “organizations” in one
instance may allow for streamlining the “look and feel” and reuse of templates. This will require further
investigation with respect to functionality as well as policy/use issues at UW.

The learning object repository functionality available with D2L LE will be useful at UW, especially in CEL,
and also on campus. Leveraging this type of functionality may allow for reuse of content and activities
between fully online and blended courses. Of course, this will require further investigation to explore
the policy and usage implications. Additionally, D2L provides an additional software package, Learning
Repository, which UW may wish to explore to address needs of sharing and reuse for individuals,
departments, programs or offering modalities (e.g., CEL).

Another type of reuse is the use of environment variables across the system. Angel has a very flexible
system to allow for defining and using environment variables. D2L has variables which allow data (such
as {firstname}) to be used within the system. There are approximately 30 of these variables defined in
the D2L LE, and during the presentation there were mixed messages about the ability to define
additional, customized variables. This functionality will need to be further explored, as UW-ACE
currently takes advantage of the ability to reuse variables across many courses, especially in CEL.

Next Steps
At the end of the Demonstration Day, it was agreed that there were a few ways to move forward in
exploring the potential adoption of the D2L LE at UW. From the UW Project team (and stakeholders)
perspective, progress on the 6 UW-ACE course conversions is very important, including examination and
feedback to D2L about the inconsistencies in a converted course. In addition, follow-up meetings with
focused groups from both D2L and UW were identified for the following topics, some of which may not
be requisite in closing the RFP but are included on this list for completeness:

       Technical integration issues (including members of SISP) including discussion of SAIP and
        connecting with Oracle
       Hosting with D2L or UW self-hosting
       Repository structure and function, exploration of D2L LOR package, and shared vs managed files
       Analytics
       Eportfolio

UW can also provide use-cases for additional functionality for D2L, including ways that intelligent agents
are currently used in UW-ACE, e.g., adding students to a team when he/she has successfully completed
a learning activity, thereby unlocking that content.

Overall, UW participants (from the Selection Team and other invitees) found the D2L Demo Day helpful
and appreciated the approach and intention of the D2L Team. The general consensus is that D2L
appears to be a viable candidate for replacing UW-ACE (no “show stoppers” or “red flags”) , although
there are some issues to clarify. If D2L can be as responsive as they suggested, UW would have a local,
engaged partner, willing to support the transition to a new LMS, and ideally, helping to take it to the
next level.

Shared By: