Step 1 Please list all LMS you are evaluating in this work sheet. The Product Name field is
mandatory. The Company Name field is optional. This spreadsheet supports up to 10 products
without modification. You may add additional columns, e.g., Web Site URL.
Product Names Company Name
MagicTutor MagicSoft
Step 2 This worksheet contains a set of general evaluation criteria. Each criterion is further described by a set of questions. T
step, evaluate each product: (1) Record the answers to the questions in the corresponding cells. You can leave a cell blank
answer. (2) Enter a rating for each criterion as a whole. Rate it using a scale of 1 to 5 (5 being most satisfactory; 1, le
Criteria Questions/Rating
Cost of Ownership
What are the costs for licensing,
software, hardware and custom
development requirements?
How fast can you be up and running?
What level of expertise is required?
What kind of support and assistance
are available?
Misc. comments
Rating
Maintainability
How many valuable resource hours will
this take to administer and maintain at
the server level?
How granular and distributed is the
administration (the more granular the
better)?
Are all of the data processes automated
and will they integrate easily with your
other systems?
Does the program run on a server
platform on which your staff already has
excellent expertise?
Misc. comments
Rating
Usability & Support
How available is documentation, how-to
guides, training and online help?
How responsive will support be?
Will the program require lots of training
or is it fairly intuitive to use?
How long will it take faculty to set up
their courses at a minimal level?
Misc. comments
Rating
User Adoption/Vendor Profile
Will the vendor be around tomorrow?
How much market share?
(If product is Open-Sourced) Is there a
strong development community
associated with the program?
Are comparable institutions currently
utilizing the program?
Misc. comments
Rating
Openness (For Open-Sourced Programs Only)
Is it written in a modular format that is
designed for easy modification and
new, custom modules? Are there clear
code specifications for writing new
modules?
Misc. comments
Rating
Standards Compliancy
Does the LMS adhere to specifications
like SCORM, IMS, OKI, AiCC? Can the
LMS import and manage content and
courseware that complies with
standards regardless of the authoring
system that produced it?
Is XML support available?
Misc. comments
Rating
Integration Capacity
Does the solution allow for ready
integration with other systems?
Misc. comments
Rating
Learning Object Metadata
(LOM) integration
How available is compatible content?
What is the capacity to integrate with
existing and newly created learning
objects?
Misc. comments
Rating
Reliability & Effectiveness
Is the solution reliable? How well will
this program help an average group of
faculty deliver their materials online?
Misc. comments
Rating
Scalability
Is the program suitable for both small
and large installations? How easily does
the solution allow for growth of users,
content, functionality?
Misc. comments
Rating
Security
Will it handle security or authentication
schemas?
Are there tools for digital right
management (DRM)?
Are there provisions for privacy issues?
Misc. comments
Rating
Hardware & Software
Considerations
Does it support multiple Operating
System platforms (including Open-
Sourced OS)?
What are the client browser
requirements?
What are the database requirements?
What additional server software is
required?
What are the hardware specifications?
Misc. comments
Rating
Multilingual support
Does the system support additional
languages?
Misc. comments
Rating
criteria. Each criterion is further described by a set of questions. To complete this
questions in the corresponding cells. You can leave a cell blank if you don't have the
te it using a scale of 1 to 5 (5 being most satisfactory; 1, least satisfactory).
MagicTutor
Open-source software. No extra-ordinary
hard/software costs.
Install is easy.
Minimal Linux experience required to install, and
operate.
Paid support is offered through web site.
4
Minimal. Upgrades are painless.
Satisfactory.
Integration should be fairly easy with the data-
base driven approach.
Implemented in php. Multiple platforms:
Windows/Linux.
4
Adequate
Two types of support: voluntary and paid.
Questions are usually answered promptly (even
with voluntary support).
Satisfactory.
Depends on level of computer sophistication. For
computer-savvy persons, hours, For novices,
days.
3
Being part of an educational institution provides
some level of stability.
Gaining popularity in the open-source LMS
segment
Yes
3
Yes
4
SCORM Compliant. Modules can be imported
and exported into SCORM compliant LMS.
Yes
4
Source Code available. Database driven
4
Repository available (but sparsely populated)
3
Used in many educational institutions
4
Most suitable for small and medium sized
organizations.
3
Yes
No
Yes
3
Coded in PHP. Very portable
Recent versions of IE, and Mozilla.
Currently mysql only.
mysql, php, apache
No extraordinary requirements
4
Yes
4
Step 3 This worksheet contains a set of feature-related evaluation criteria. Each criterion is further described by a set of statem
this step, enter a weight for each feature. Existing weights are examples only. The weight is on a scale of 1 to 5 (5 means
important to you; 1, least important). Note that you only need to enter the weight once for each feature. Then, evaluate
any comments you may have regarding the feature statements in the corresponding cells. You can leave a cell blank. (2) Ente
feature as a whole. Rate it using a scale of 1 to 5 (5 being most satisfactory; 1, least satisfactory). The worksheet will
each product the total weighted score (weighted sum over all features), and a re-scaled score (new scale is 0 -5).
Feature
Features Weight Feature Statements MagicTutor
Administration 3 Manage user registrations Both batch & individual
registration.
Set curricula, chart certification paths No
Administer internal budgets, user payments,
and chargebacks. No
Create standard and customized reports on
individual and group performance. Reports
should be scalable to include the entire Standard reports available
workforce. for Usage & Login Data.
Print Certificates.
Build schedules for learners, instructors,
and classrooms. No
Misc. Comments
Rating 3
Security 4 Encryption
Authentication Logins are authenticated.
Misc. Comments
Rating 3
Access 4 Individual/Group Login and Password Yes
Manage user profiles, define roles. Assign Can mix & match privileges
tutors. to form roles for users.
Browser accessible Yes
Course Authorization. Instructor approves Private vs Public vs
enrolment Protected.
Registration Integration. Prerequisite
Screening, Cancel Notification Simple model of registration
Misc. Comments
Rating 3
Integration with Other 3 Integration with HR Systems
Systems No
Integration with CRM. Student listing.
Maintain student information. No
Misc. Comments
Rating 3
Course Design, 3 Customizable look and feel
Development and Can build own themes. Use
Integration of stylesheets.
Support classroom and virtual courses No classroom support.
Course templates
Use and access Learning Object Yes
Web authoring Yes
Support multimedia types No streaming video.
Accessibility Compliance Yes
Instructional design tools Yes
Curriculum Management No
Easy Navigation/linking Many ways to navigate. Can
turn off.
Easy Course structuring
Extensible Architecture
Support style sheets Yes
Misc. Comments
Rating 4
Course Monitoring 3 Course Listing/Catalogue Available.
Course Descriptions One paragraph description
Schedules and Availability No
Usage Tracking
Misc. Comments
Rating 3
Assessment Design 3 Creates Test Questions and Facilitate Test Random questions. Auto-
Administration marked tests.
Automated Testing and Scoring Yes
Course Path Maintenance keep track of how users
navigate
Competency Mapping/Skill Gap Analysis No
Self-assessment Yes
Misc. Comments
Rating 4
Online Collaboration 4 Email
and Communications Yes
chat rooms Yes
help desks
file exchange Yes
online journals Commentary provided for
files.
notes Yes
whiteboard No
discussion groups/forums Yes
Misc. Comments
Rating 4
Productivity Tools 3 Bookmarks Keep context of last access
paged.
Calendar/Progress Review Yes
Orientation/Help Yes
Search Yes
Work offline/Synchronize Export course contents.
Misc. Comments
Rating 3
Total Weighted Score 100
Re-scaled Score ( 0-
5) 2.2
er described by a set of statements. To complete
scale of 1 to 5 (5 means the feature is most
eature. Then, evaluate each product: (1) Record
n leave a cell blank. (2) Enter a rating of the
. The worksheet will automatically compute for
w scale is 0 -5).
0 0 0
0.0 0.0 0.0
0 0 0 0
0.0 0.0 0.0 0.0
0 0
0.0 0.0
Step 4 This worksheet summarizes the scores from both the general and the feature-related sets of criteria. To complete this
a weight for each criterion. Existing weights are examples only, and should be modified. The weight is a number between 1 an
the criterion is most important to you; 1, least important). (2) Optional. Enter comments in the Comments field. This work
calculates a combined score of each product.
>
>
>
>
Features &
Functionality 5 2.2 0.0 0.0 0.0 0.0 0.0
Cost of Ownership
5 4 0 0 0 0 0
Maintainability 3 4 0 0 0 0 0
Usability &
Support 3 3 0 0 0 0 0
User Adoption 2 3 0 0 0 0 0
Openness 3 4 0 0 0 0 0
Standards
Compliancy 2 4 0 0 0 0 0
Integration
Capacity 4 4 0 0 0 0 0
Learning Object
Metadata
integration 2 3 0 0 0 0 0
Reliability &
Effectiveness 3 4 0 0 0 0 0
Scalability 4 3 0 0 0 0 0
Security 4 3 0 0 0 0 0
Hardware &
Software
Considerations 3 4 0 0 0 0 0
Multilingual
support 1 4 0 0 0 0 0
Combined Score 152 0 0 0 0 0
criteria. To complete this step, (1) Enter
is a number between 1 and 5 (5 means
ents field. This worksheet automatically
>
>
>
0.0 0.0 0.0 0.0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
CC-BY-SA - 3waynet Inc. and the Commonwealth of Learning, 2004