Document Sample
vine Powered By Docstoc
					A collaborative workspace environment: experience of evaluation and
selection in the Agora project
by Tracy Gardner and Rosemary Russell, UK Office for Library and Information Networking
The Agora project recognises that effective communication is a key to achieving successful project
outcomes. One of its objectives is to develop an integrated communications environment for both
internal and external communications. Collaborative workspace software to support asynchronous
communication is seen as the core application within such an environment. Requirements for such
software were developed, together with an evaluation plan. Three collaborative workspace products
(Web4Groups, BSCW and AltaVista Forum) that passed preliminary criteria were selected for detailed
evaluation. Results showed that BSCW met Agora’s needs most closely and the project is currently in
the process of implemention.

Introduction: the need for a collaborative workspace
Agorai is one of the ‘hybrid library’ projects which is funded by the Electronic Libraries Programme
(eLib)ii of the Joint Information Systems Committee. It is developing a hybrid library management
system (HLMS) to provide integrated access to distributed heterogeneous information services. It will
allow users to access both print and electronic resources more easily, which may be held either locally
or remotely. The HLMS is based on the ‘broker’ model which was developed by the MODELS
projectiii; it includes discovery, location, request, and delivery services.
Agora is led by the University of East Anglia, with UKOLN, Fretwell-Downing Informatics and
CERLIM (the Centre for Research in Library and Information Management) as partners. The project
also works closely with several associate groups: libraries, service providers, and systems developers.
With such a wide cross-section of participants involved in various elements of the project, effective
communication is a key to achieving successful outcomes.
Agora’s central communications aims are therefore firstly to support effective dialogue amongst project
participants, thereby enabling a shared understanding of the hybrid library management system under
development; and secondly to adopt an appropriate dissemination programme in order to share learning
experiences with the relevant communities.

You call it groupware, I call it computer-supported cooperative work…
A web-based solution is the obvious choice for communication between a geographically dispersed
group of people who are familiar with the web as a working environment. Products that enable such
communication are generally known as groupware or computer-supported cooperative work (CSCW)
A large number of products, both commercial and non-commercial, fall into the category of web-based
groupware; there is a range of useful resources on the web for sourcing groupware and CSCW
softwareiv. It is clearly impossible to perform a detailed analysis of all such products. Instead some
basic criteria for product selection were identified (see below) and three products that met those criteria
were chosen for further analysis.
This text focuses on Agora’s requirements for groupware products, together with software evaluation.
Both synchronous (realtime) and asynchronous (non-realtime) activities were considered. Full
definitions of terminology used by the project are included at Appendix A.

Basic general requirements
The general requirement was therefore an integrated workspace for both internal and external
communications. As indicated, web-based groupware is particularly suitable for supporting wide-area
group collaboration. However, few projects to date appear to use groupware; it was planned to use
state-of-the-art web technologies to demonstrate the possibilities of this type of software.
At the same time, another clear requirement was ease of use; the project team and associates should be
comfortable with using the facilities, although it is appreciated that there will be a learning curve for
any new technology. Similarly, public users should find the site inviting to explore; it should encourage
public interaction and feedback. It is hoped that Agora will demonstrate a state-of-the art project
communications environment.
It was felt that the solution should preferably not require users to install additional software. It should
also be platform independent, not relying on the features of a particular web browser or a particular
operating system environment. Server side software must be available for either Solaris or Windows
NT (with Solaris being preferred since the communications software is hosted at UKOLN where
Solaris is the main operating system).
For consistency it was hoped that a single (core) product would meet the majority of Agora’s
requirements. However, no product will provide all features so the ease with which external products
can be used to augment the core product is also of concern.

Immediate feature requirements
As well as meeting the basic requirements outlined above the chosen solution must support certain
features. It should be noted that these features all fall into the category of asynchronous groupware.

Access Control
There is a need to provide areas with different levels of access to materials and facilities within the
workspace. It should be possible to allow the following forms of access:
Public            Contributors can read materials and can contribute without the need to join a service
read/write        first.
Public read       Interested parties have access to materials but cannot add material of their own. Only
access            Agora members can place resources in such areas.
Member only Agora members (who will need to join a service and be authenticated) will be able to
access      participate fully. Member-only areas will not be accessible to non-members.
Restricted    A specified subset of Agora members will have access to restricted member areas.
member access Such areas will be used for restricted discussions which should not involve all Agora

Notice boards
It should be possible to post articles (ie messages), documents and links to other online materials.
Discussion forums
It should be possible to create discussion forums for online conferencing. Discussions should be

Document sharing and versioning
It should be possible for multiple authors to work on a document (asynchronously) uploading new
versions to the shared workspace. The workspace should provide a mechanism for determining and
accessing the current version of the document and for storing the version history.

Calendaring and scheduling
The workspace should provide calendaring features to support the recording and sharing of events of
interest to Agora members.
It should also provide support for the scheduling of meetings for which multiple Agora members are
required to be present.

Medium term feature requirements
Once the features described in the previous section have been established as a means of
communication, the project is expected to progress to the following features:

Real time chat
It should be possible for Agora members to communicate in real time using text based chat.

Voting and rating
Voting and rating features would allow user responses to be collected as structured metadata, to allow
easy analysis.
Voting refers to the process of determining the opinion of a group by selection between a number of
alternatives. The vote takes place over a fixed period of time within which each member has a single
Rating refers to the long term collection of metadata based on the evaluation of resources by group

Long Term Feature Requirements

Video conferencing
Members with access to suitable equipment (desktop cameras and microphones) would be able to
communicate in real time with other participants via audio and video.

Real time presentations
Agora members who are not physically present at a presentation would be able to view the presentation
over the Internet. The presentation could also be recorded and stored within the Agora workspace for
playback at a later date. Real time presentations could be combined with real time chat with the speaker
after the event.
Shared whiteboard
Multiple members of the group have access to the same whiteboard. Each member can gain access to
the board and make notes upon it (either text or diagrams).

Shared windowing
An application (such as a web browser or word processor) could be shared by multiple group members.
All participants would see the application and participants may be able to gain control of the
application. Only one participant can control the application at any one time.

Product evaluation

A large number of products were initially informally evaluated to determine suitability for use within
Agora. It quickly became clear that most software provided limited functionality. The focus has been on
selecting a core product that will provide a web-based workspace from which all groupware facilities
can be accessed.
The field was therefore narrowed to produce a small number of products that were considered to be
potentially capable of meeting Agora’s communication requirements as outlined above. The three
products selected were: BSCWv, Web4Groupsvi, and Altavista Forumvii.
The products were evaluated against tasks selected to be representative of the immediate requirements
identified. The tasks are described in the following section. For each task the feasibility (is it
achievable?) and usability (is it intuitive and straightforward?) of each product were assessed.

 Set up a workspace (including registering members) with sub-areas with varying levels of access.
   Create a resource area containing URLs, documents and articles relating to the Agora project.
   Set up a threaded discussion forum. Post initial articles and replies.
   Create a document and place it under version control. Download the document, revise it and upload
    it as the new current version.
   Set up a face-to-face meeting and invite members.
These tasks are intended to be general enough to allow different products to offer different ways of
working and yet still be able of drawing meaningful comparisons.


Web4Groups categorises itself as `non-simultaneous web-based groupware'. It is an EU-funded project
to develop an advanced, web-based groupware system. Figure 1 shows a screenshot of a newly created
Web4Groups workspace.
Web4Groups primarily supports different-time communication, i.e. communication where all users
need not be active at the same time. Participants are of course not prohibited from participating at the
same time, but the software does not provide facilities for immediate exchange of data between
simultaneous participantsviii.
Figure 1: A Web4Groups Workspace

Evaluation Details
Evaluation was performed at the Web4Groups public demonstration siteix.

Task 1
Set up a workspace (including registering members) with sub-areas with varying levels of access.
Users can enter public areas of Web4Groups anonymously or they can choose to login (new users are
asked to register with a username and password).
Once logged in users have access to their own private area. In this area it is possible to create new
`activities'. When creating an activity the user must provide a name, some text and an email address (all
users and activities within Web4Groups have an associated email address). A new activity can be a
`public forum', a `private workspace', a `private folder' or an `information page'. These types of activity
are sufficient to support the different access levels required by Agora.
Once an area has been created, it can be entered and members can be added. It is only possible to add
members who have already registered with Web4Groups so prospective members must be asked to
register first. External members may be added but they can only participate via email (it is not obvious
that this feature would be useful for Agora).
Summary: Web4Groups scores highly on feasibility, it was possible to set up areas with various access
levels. The task was reasonably straightforward to complete although the use of the term `activities' is
non-intuitive and does not seem to apply to all of the possible activity types.

Task 2
Create a resource area containing URLs, documents and articles relating to the Agora project.
One of the activities created in Task 1 was used as the location for the resource.
Firstly, a URL was added. A subject and the URL must be provided. The URL is added to the resource
area, clicking on it takes you to the specified web page.
In order to add an article to the area, 'Write Message' must be selected. A subject and the content for the
message must be provided, also specifying where it should be sent. Web4Groups uses the idea of
mailing a message to an activity. An image can also be selected to denote the kind of article posted.
Once added the article can be read by clicking on it.
`Add/Send File' is selected to upload a document. A file upload box appears so that a local document
can be selected. Again the activity must be selected to which the document will be sent. The document
appears in the resource area.
Summary: The completion of the task was entirely feasible but again seemed non-intuitive. Also on
completed of a task Web4Groups did not always return the user to the page last being used. Deleting an
item took the user to another page confirming the action, involving removal from the resource area.

Task 3
Set up a threaded discussion forum. Post initial articles and replies.
Any article can be the beginning of a discussion forum, so the article created in Task 2 was used as the
basis for this task.
When viewing an article, a `Reply' option appears at the top of the page. A new article can be createed
by clicking on reply. The reply article simply appears in the same folder. The discussion is not threaded.
Summary: Web4Groups does not appear to support threaded discussions.

Task 4
Create a document and place it under version control. Download the document revise it and upload it
as the new current version of the document.
Documents can be uploaded as described under Task1. However there is no option to place them under
version control.
Summary: Web4Groups does not support document versioning.

Task 5
Set up a face-to-face meeting and invite members.
There is no direct support for the setting up of meetings.
Summary: Web4Groups offers no real advantage over email for the setting up of meetings.

General issues and conclusions
The overall appearance of Web4Groups is reasonable. Graphics are kept to a minimum and many
options are selected by clicking on a textual link. Web4Groups did not feel particularly user friendly
and the use of confusing terminology was distracting. Performance was reasonable considering the
software was remotely hosted.
Functionality that was not highlighted in the above set of tasks includes: index of areas, find facility,
trial voting, annotations (for discussing web pages).
Web4Groups offers some groupware functionality but does not meet several of Agora’s requirements
set out above.

BSCW - Basic Support for Cooperative Work

BSCW describes itself as a shared workspace system which supports document upload, event
notification, group management and much more.
A screenshot of a BSCW workspace is shown in figure 2.
 Figure 2: A BSCW Workspace

Evaluation details
This evaluation was carried out on a locally installed copy of BSCW downloaded from the BSCW web

Task 1
Set up a workspace (including registering members) with sub-areas with varying levels of access.
Registration is initiated via email. An existing BSCW member invites a new user by email. The
invitation message includes a URL at which the user can register. Once a member, new `Folders' can be
created; if further members are invited to a folder it becomes a 'Shared Workspace'.
Existing BSCW members can be invited to join a shared workspace or non-members can be invited by
Folders inherit the access permissions of the folder in which they are contained. Fine-grained access
control is possible on a per folder or per resource basis.
Summary: BSCW provides suitable access control. This is slightly awkward to set up but very
Task 2
Create a resource area containing URLs, documents and articles relating to the Agora project.

One of the folders created in Task 1 was used as a resource area for this task.
Articles can be added to a folder by choosing `Add Article'. The title and contents must be provided and
an image representing the nature of the article must be selected. Added articles appear in the folder,
clicking on them takes you to the article.
URLs can be added to a folder by choosing `Add URL'. The title and URL are requested and a
description of the web page being linked to may be provided. There is an option to include a link
showing the text of the URL as well as its title. URLs appear in the folder, clicking on the takes you to
the appropriate web page.
Documents may be added to a folder by choosing `Add Document'. A file upload page appears for
selection of a local file. A description other than the file name may also be provided. Documents appear
in the folder, clicking on them will display or download the document.
Summary: BSCW provides a consistent interface for adding resources of different kinds to a

Task 3
Set up a threaded discussion forum. Post initial articles and replies.
Any article in BSCW can form the basis of a discussion forum so an article created in Task 2 was taken
to be the start of a discussion.
When viewing an article there are options to reply to the current article or the original article.
Discussions are threaded and it is possible to view all replies to an article on a single page.
Summary: The creation of a threaded discussion forum is possible in BSCW. It is also relatively

Task 4

Create a document and place it under version control. Download the document revise it and upload it
as the new current version of the document.
Any document that has been uploaded to BSCW can be placed under version control so this task was
carried out using a document uploaded in Task 2.
Documents can be placed under version control by clicking on `Version' under the document link. Each
version of a document has a version number and a descriptive comment can be added. New versions of
a document are uploaded by clicking on 'Revise'.
Summary: BSCW provides basic support for document versioning. It is straightforward to use.

Task 5
Set up a face-to-face meeting and invite members.
BSCW does provide some support for organising meetings. It does not help with scheduling the
meeting but once a meeting has been scheduling it can be added to BSCW using the `Add Meeting'
feature. Members and non-members can be invited to a meeting and an email invitation will be sent
Members can confirm or decline the meeting. Members can access a list of the meetings to which they
are invited.
Summary: BSCW provides some support for arranging meetings but does not help with the problem of
scheduling the meeting. Setting up a face-to-face meeting is fairly straightforward.

General Issues and Conclusions
The overall appearance of BSCW is clean and professional. Performance is reasonable (it was also
tested using the public server at the BSCW web site). Online help is available although it is not context
sensitive, there is also an FAQ and a mailing list for BSCW users.
Additional features not covered in evaluation: online meetings can be arranged although external
software is required for the meetings, there is support for converting documents from one format to
another, some filters are provided with BSCW but others can be added, BSCW meetings can be
accessed in the vCalendar format, the workspace can be searched and web searches can be performed
and saved in BSCW.
BSCW supports the majority of Agora's immediate requirements for groupware in a well-designed

AltaVista Forum

The AltaVista Forum website describes the product as `a collaborative and knowledge-based teamware
web application. From the browser you are using today, you can search and retrieve documents, update
information in that document, and add your comments.’
Figure 3 shows a screenshot of the AltaVista Forum public Demo Area.
Figure 3: The AltaVista Forum Demo Area

Evaluation Details
This evaluation was carried out on the AltaVista Forum public demonstration site. Consequently it was
not possible to test some advanced features of the software - the documentation has been consulted
where this is the case. A trial version of the software is available for download from the AVF website
but time constraints meant that it was not possible to download and install the product for this

Task 1
Set up a workspace (including registering members) with sub-areas with varying levels of access.
Setting up new workspaces on the public demo site of AVF was not possible. However, the
documentation states that it is possible to `Set up private work areas for different teams'. The online
help shows that this is done using the `Add Folder' option on the `Tools Menu'.
Public areas can be seen on the demo site; once logged in, additional access is available.
Summary: AVF appears to be capable of meeting Agora's requirements for this task but it was not
possible to test how easily this can be done.
Task 2
Create a resource area containing URLs, documents and articles relating to the Agora project.
This task was carried out using an existing demo area on the AVF web site as the resource area.
Articles can be added by choosing `Add Discussion Topic' from a pull-down `Tools Menu'. Subject and
content must be provided and further access control information may be supplied. Attachments may be
added to the article. Once added the article appears in the resource area.
Documents can be uploaded by choosing `Add Document' from the tools menu. A title and abstract
may be entered and a local document can be selected using a file upload box. Once added the document
appears in the resource area. The name of the document is not given in the resource area so it is
necessary to provide a meaningful title.
URLs can be linked in by choosing `Add URL' from the tools menu. A title and abstract may be entered
along with the URL. Once added the URL appears in the resource area.
Summary: AVF allows resources to be added in a straightforward manner. Access to resources can be
finely tuned.

Task 3
Set up a threaded discussion forum. Post initial articles and replies.
Any article in AVF can be the basis for a discussion. When viewing an article a `Reply' option appears
on the tools menu. By selecting this option a reply with subject, content and, optionally, attachments
may be added.
Viewing the original article now also shows the replies. The number of replies appears in brackets after
the original article in the resource area.
Summary: AVF supports the creation of threaded discussions in a straightforward manner.

Task 4
Create a document and place it under version control. Download the document revise it and upload it
as the new current version of the document.
AVF does not currently have direct support for doument versioning.
Summary: Although documents can be shared using AVF they cannot be placed under version control.

Task 5
Set up a face-to-face meeting and invite members.
AVF has support for individual and group calendars and allows meetings to be arranged and
participants invited. It was not possible to test this feature on the public demo site but the
documentation describes extensive calendaring features.
Summary: AVF would probably meet the needs of Agora for basic calendaring features.

General Issues and Conclusions
AVF is a professional looking product that meets a large number of Agora's current communications

Product selection
Each of the CSCW solutions evaluated had positive and negative points and each would be suitable for
certain kinds of project. However, in the context of the Agora requirements BSCW appears to offer the
most complete functionality combined with a professional appearance and a relatively straightforward
user interface.
Having selected BSCW as the most likely contender it was implemented on a trial basis. A limited
group of project participants were ‘invited’ to use the notice board facility and discussion forum, to
support a specific project work package. The aim was to help ensure that there were no major usability
issues which were likely to cause problems in the future. Although a relatively small number of people
was involved, it was helpful to prove that a group of users could register themselves and carry out basic
functions such as accessing documents, without experiencing problems.

On the basis of the trial use, Agora has decided to adopt a phased implementation of BSCW, to allow
people to become familiar with facilities on a gradual basis. Groupware requires certain alterations in
working habits, such as remembering to visit the server on a regular basis. Encouragement may be
needed in the initial stages. However the benefits can already be seen – the distribution of several large
documents to an extended group of people as email attachments has been avoided, and documents are
held in a secure environment; comments can also be collected in the same workspace.
It seems (based on anecdotal evidence) that a surprisingly small number of projects and workplaces are
currently using groupwarex. However it is only relatively recently that a range of software has become
available and they are not yet mainstream products. With remote and project-based working becoming
increasingly commonplace, it is likely that takeup will rapidly increase.

Appendix A: Terminology
It is useful to define the terminology used by the Agora project; the following definitions are taken from
the groupware glossary hosted by Usability Firstxi.
CSCW: computer-supported cooperative work. The field of study that examines how technology
affects group interaction, and how technology can best be designed and built to facilitate group work.
Groupware: any type of software designed for groups and for communication, including email,
video-conferencing, workflow, chat, and collaborative editing systems.
Asynchronous groupware: software used to help people to work in groups, but not requiring those
people to be working together at the same time (asynchronous = not coordinating at a single point in
Realtime groupware: software used to help people to work in groups, while working together at the
same time. Examples include phones and video-conferencing, playing arcade games together, and
shared drawing tools that allow people to see what their conversational partners are drawing as they
draw. Also known as synchronous groupware
To these we add the following definition:
Workspace: a web area that provides an integrated communications environment for carrying out
(synchronous and asynchronous) CSCW activities.

UKOLN is funded by the British Library Research and Innovation Centre (BLRIC), the Joint
Information Systems Committee (JISC) of the Higher Education Funding Councils, as well as by
project funding from the JISC's Electronic Libraries Programme and the European Union. UKOLN also
receives support from the University of Bath where it is based.

    Information about Agora: <URL:>
     Information about eLib: <URL:
      Information about MODELS: <URL:>
     We found the following resources useful for sourcing groupware and CSCW software - Web
Conferencing Software: <>; Conferencing software for the web:
    BSCW: <URL:>
     Web4Groups: <URL:>
      Altavista Forum: <URL:>
       Web4Groups summary at: <URL:>
     Web4Groups public demonstration site is at: <URL:>
    It is interesting that an introductory course on groupware is now being offered by Netskills; see
     Usability First groupware glossary:

Shared By: