Project management

Document Sample
Project management Powered By Docstoc

                Project management

As we saw in Chapter 2, the development of a digital collection can be
broken down into a number of distinct phases:

1. Define goals and scope of the collection (covered in Chapter 2).
2. Evaluate and select source material (Chapter 2).
3. Clear permission to use the source material (Chapter 3).
4. Define project objectives and preliminary milestones (this chapter).
5. Determine technical specifications (Chapters 4–7).
6. Develop workflows (Chapter 10).
7. Develop budget (this chapter).
8. Execute the project (this chapter, Chapter 9).
9. Evaluate the project (this chapter).
10. Evaluate the collection (this chapter).

So far, I have covered in detail a number of these phases, such as defining
the goals and scope of the collection, evaluating source material, dealing
with copyright issues, deciding what types of metadata to use, determining
how users of the collection should find and display items in the collection,
determining what file formats to use, and selecting a CMS. This chapter
introduces a number of topics not reflected in the high-level outline of
phases provided above that deal specifically with project management,
including planning the implementation of a project, writing funding
proposals, staffing, vendor relations and project evaluation. The following
two chapters, ‘Project operations’ and ‘Developing workflows’, focus on
the operational aspects of project management.
   The term ‘project’ can be misleading, as it implies a period of activity
with a defined start and end. In many cases, particularly with collections
that are intended to grow over time, there is no specific end to the

Putting Content Online: A Practical Guide for Libraries

      work – it continues indefinitely. In other cases, the work stops when
      either the goals defined for the collection have been met or the money
      allocated to the work has been spent. In this chapter (and in others
      throughout the book), ‘project’ is used primarily to describe the work
      required to create and maintain a collection of digital content. ‘Project’
      also refers to the entire set of activities surrounding development of a
      digital collection, including promotion, user education and ongoing
      evaluation, but in general this chapter focuses on the narrower meaning
      of the word. ‘Project management’ includes overall planning of this
      work, overseeing of project operations, project staff and project funding.
      Whether or not the actual work stops at a specific point in time or
      whether it continues as the collection grows is not important because
      both types of projects need to be managed.
         Typically one person (a project manager) or a small group of people (a
      management committee) is responsible for the planning and execution of
      a defined project. Depending on the size of the library or the number of
      staff involved in the development of locally created digital collections,
      this person may or may not be the same one who is responsible for other
      aspects of the collection, such as advocating for its development, gaining
      financial commitment from their library, or defining the collection’s
      goals and scope. The production of content and the related work
      required to make the content available to users frequently occur quickly,
      with firm deadlines and limited resources, and it is important that
      libraries creating collections take the role of project manager seriously,
      as managing any project is usually a time-consuming and intense activity.

      Sequence and timing
      Not surprisingly, the above list of phases hides a good deal of
      complexity. To begin with, the fifth phase, ‘Determine technical
      specifications’, is really a catch-all that involves defining the metadata
      that the collection will use, determining the search and display
      functionality, determining the file formats that will be used, and
      selecting a CMS. As we have seen, all of these tasks are themselves fairly
      complex, and ideally should be completed before actual production of
      content begins, given that each has implications for tools, workflows
      and procedures. For example, during this phase, desired search and
      display functionality may influence file formats, which may in turn
      determine the tools that are used during the creation of the files.

                                                              Project management

Likewise, search and display requirements and the functionality of the
CMS will determine what types of structural metadata need to be
   Second, even though it is convenient to view the phases as a sequential
list, in practice it is not always possible to complete one and move on to
the next sequentially. Two examples:

  The original goals for a collection may need to be revised later because
  of technical or budget issues that arise.
  Securing the rights to include a large amount of material could
  take more time than initially anticipated, resulting in delayed
  digitisation and the need to alter workflows in order to finish the
  work on time.

In addition, some of the activity that is defined early in our list of phases
may continue past the beginning of later phases. The best example of this
is the clearance of permission to include material in the collection.
Depending on national copyright laws, material should not be digitised
before permission to do so has been secured, which means that some
material may be ‘held back’ from the project execution phase. However,
this material can remain in the clearance queue while the project moves
forward and other material is digitised and made available. One of the
most challenging roles of a project manager is balancing the complex
relationships between the various phases in a project.
   One phase that should happen at a specific point in the planning of a
digital collection is the development of the budget. This activity should
happen after the workflows and project objectives have been finalised,
not before. Later in this chapter I will cover ‘evidence-based budgeting’,
which requires that workflows and procedures be sufficiently developed
to perform representative samples of the work in order to determine how
much time they take. It is important that the creation of a budget follows
the development and testing of workflows; otherwise, cost and estimates
and production milestones will be highly speculative. The more
information available before the budget is created, the more accurate the
budget will be.
   To summarise, even though we can identify a fairly linear progression
between distinct phases in the development of a digital collection, in
many cases it will be necessary to adopt a more flexible approach. The
exception is developing the budget: accurately projecting the cost of
executing a project requires that as many decisions are finalised early in
the planning process as possible.

Putting Content Online: A Practical Guide for Libraries

      Planning the implementation
      Another area of complexity not reflected in the list of phases provided
      above is planning the implementation of the project itself. This activity,
      broken down into specific tasks, clusters around phases 5, 6 and 7:

      a. Define project objectives and preliminary milestones
      b. Determine technical specifications
      c. Develop workflows
      d. Determine preliminary procedures based on workflows; begin project
      e. Determine what resources you need (hardware, software, staff)
      f. Decide if you will outsource
      g. Develop budget
      h. Evaluate and acquire necessary resources
      i. Finalise milestones
      j. Finish project documentation
      k. Hire and train staff, if necessary
      l. Start production

      This series of tasks begins with identifying objectives and milestones and
      ends with the start of production work. The remainder of this section
      will cover each of these aspects of planning the implementation.

      Defining project objectives and preliminary
      A collection’s goals are the starting point for defining the objectives of
      the project. Objectives should describe the desired outcomes of the
      project, specifically the deliverables expected at the end of the project (if
      a specific end point has been defined). Milestones describe desired
      outcomes at specific points in time throughout the project and are used
      to assist in monitoring the progress of the work being completed.
        As an example, I will use the following statement of a collection’s goals
      developed in Chapter 2:

            The Southeastern Regional News Collection contains selected
            issues of the County Herald and Crighton Daily newspapers

                                                              Project management

     published between 1900 and 1920. The collection, which will be
     freely available to everyone over the World Wide Web, will be of
     interest to local historians, to genealogists, and to the students
     seeking primary source material from the early part of the 20th
     century. Access to the major articles and in each issue will be aided
     by the addition of subject keywords. Each newspaper issue will be
     presented as a single Adobe Acrobat file for easy printing.

Once we have a clear statement of the project’s goals, we can determine
how much work needs to be done by compiling an inventory that lists all
of the major groups of content that will be included in the collection
based on physical format, and within each group, the number of
documents that need to be digitised. It is useful to define these groups by
physical format (text, image, audio, video), as the ultimate purpose of
this inventory is to help us plan the amount of work that will need to be
competed and how long it will probably take. Each format group will
require its own workflow (or at least major section of a workflow).
Finally, in addition to the number of documents within each format
group, it is necessary to calculate how many components or parts of each
document will need to be processed. For text, the countable unit is pages;
for graphic materials, it is usually the entire document (e.g. a single
photograph); for audio and video, it is each separate document, but as
these two formats must be digitised in real time they are generally
quantified by their duration. Again, the goal of this inventory is to
determine the amount of work that will need to be completed during the
project, so the inventory needs to be fairly granular. It is better to define
more format groups than fewer because you can combine similar groups
later; by contrast, defining your format groups too narrowly can
introduce unnecessary complexities into your cost estimates.
   To proceed with our example, during the content evaluation and
selection phase of collection planning, the following calculations were
used to estimate the total number of pages published in the period
1900–1920. As stated above, the two newspapers are of the same
physical format, but we are putting each newspaper in its own group
simply to assist in determining the total number of pages (Table 8.1):
   The number of issues to include in the collection was determined
during the evaluation phase. The staff evaluating the material concluded
that in general one issue per month of the Herald and one issue every
2 weeks of the Daily would provide a representative sample of the
articles, advertisements and other content in the newspapers. Based
on this assessment, approximately 9840 pages need to be digitised.

Putting Content Online: A Practical Guide for Libraries

        Table 8.1        Sample calculations for estimating extent of
                         source material

                                                                           Approx. no. of
                 Average no. of        No. of issues No. of issues in        pages in
       Newspaper pages per issue        per week     collection              collection
       County       15 pages/issue           1            240 (one per        3600
       Herald                                             month)
       Crighton     12 pages/issue           5            520 (one every      6240
       Daily                                              2 weeks)

      The project is to be performed over a period of 10 months (that is the
      period during which the funding is available). Therefore, 984 pages need
      to be scanned every month (or 246 every week), on average, during this
      time. This figure will be used to determine preliminary production
        Based on the information we have at this point, the preliminary
      monthly milestones for the project can be represented as in Table 8.2:

        Table 8.2        Sample production milestones

                       End of month No. of pages to be digitised
                              1                     984
                              2                    1968
                              3                    2952
                              4                    3936
                              5                    4920
                              6                    5904
                              7                    6888
                              8                    7872
                              9                    8856
                             10                    9840

         These milestones are preliminary at this point because we have not
      performed the detailed calculations necessary to determine how many
      staff we will need (and we do not have enough information to do that
      at this point). We will almost certainly need to revise these milestones as
      information about funding and resources becomes available, but it is
      good to have a general idea of how much content we will be expected to
      produce at this point so that we can at least start planning for staff,
      hardware and other resources.

                                                              Project management

  In some cases, the project is not intended to complete all of the work
required to fulfil the collection’s goals, such as when the collection has
open-ended goals or when the collection is intended to be completed in
specific phases over an extended period of time. In these cases, defining
milestones and using them to track project status is still essential, if only
to track how much is being spent in staff salaries.

The role of workflow
As suggested by its position in the implementation list, workflows need to
be developed as early as possible. From a project management perspective,
a clearly defined workflow provides information necessary to:

  Determine the tasks required to achieve a specified set of outcomes
  and the order in which these tasks must be completed.
  Determine costs. As already mentioned, the only sound method of
  determining the resources required to perform a given set of tasks
  (and the costs of performing those tasks) is to perform a
  representative set of those tasks and document actual costs carefully.
  Workflow modelling and evaluation is therefore necessary prior to
  estimation of costs associated with the production of digital content.
  Clarify duties and responsibilities. Systematic workflow definitions
  can help clarify who is responsible for what and can serve as the basis
  for negotiations between partner libraries, vendors, consultants and
  other participants in the creation of a digital collection.

Chapter 10, which deals exclusively with workflow development, will
expand on these three functions, but at this point it should be apparent
that workflow development is a crucial component of project

Project documentation
In the next chapter I will cover documentation as it relates to project
operations: documentation of detailed procedures, rationales for
decisions, administrative metadata and general best practices for creating
operational documentation. A number of other kinds of documentation
that are not particular to digital collection development must be
managed efficiently. These types of documentation are required for any
type of project (e.g. they apply to retrospective cataloguing of print

Putting Content Online: A Practical Guide for Libraries

      materials as well as digitising content) but are worth mentioning here
      nonetheless. These include:

        Finances: Financial aspects of a project need to be documented very
        carefully. Financial audits are a real possibility, particularly if funds
        have been received from public agencies, and any funder has the right
        to ask for documentation on how project managers are spending its
        money. Many institutions rely on centralised financial services (at the
        library or parent organisation level) to assist in this documentation.
        Periodic reporting to funders is much easier if sensible documentation
        procedures are established early in the project.
        Staffing documentation: Job descriptions, documentation regarding
        position creation, applicants’ forms and resumes, and other types of
        documentation should be handled in accordance with your
        institution’s normal practices.
        Results of monitoring the project: I will discuss the importance of
        documenting the results of monitoring later in this chapter.
        Meeting minutes: Notes and minutes from project staff meetings
        should document the status of the project, issues that require attention
        and action items.

      Documentation should be viewed as a regular activity and not one that is
      performed on an as-needed basis, and it should be built into each of the
      phases identified earlier as part of large-scale collection planning, from
      defining the goals and scope of the collection to project and ongoing
      collection evaluation. Every phase needs to be documented in some way,
      but in particular, evaluating and selecting source material, clearing
      permission to use the source material and determining the technical
      specifications for the collection all require considerable documentation.
      Given that creating and maintaining useful documentation is time-
      consuming, it can take considerable staff resources, and therefore should be
      included in the overall staffing and budget for a given collection or project.

      The tasks that operational staff perform will be defined in project
      workflows. Similar tasks should be grouped into roles, which are general
      types of staff positions that can be used in planning and in developing
      job descriptions. Stephen Chapman provides the following list of staff
      roles typically involved in digitisation projects:

                                                            Project management

  Project manager
  Conservator, curator, or other analyst of the source materials
  Preparations technician (may also be curator, who, in turn, may also
  be the selector)
  Cataloger to create or enhance bibliographic records and to withdraw
  materials for conversion
  Scanning technician or photographer
  Quality control technician (may also be the scanning technician)
  Metadata analyst (may also be the cataloger)
  Data entry technician
  Programmer or other database expert who integrates metadata and
  images into a coherent resource (also known as the digital object)
  Systems administrator or other manager of electronic records and
  Network administrator to implement security and other access
  requirements (may also be the systems administrator)
  Developer or designer of the user interface1

These roles may not apply to every project, depending on the nature of
the source material and the workflows that you have defined. For
example, the staff required to work on a collection of born-digital
material will differ from the staff required to work on a digitisation
project. Also, some of these roles, such as cataloguers and network
administrators, may be assumed by existing staff who perform these
functions as part of their regular jobs. Having existing library staff
perform work in short-term projects is a good way to save on staff costs,
and many funders consider existing staff time to be ‘in kind’
contributions (discussed further below). However, if existing library staff
are to work on the digitisation projects, they must be allowed to devote
the required time and not have the extra work simply added to their
existing responsibilities. In some cases this means that the digitisation
project takes priority over other activity or that someone else can take on
work displaced by the digitisation project without exceeding their own
normal workloads.
  A number of factors should be considered when creating job
descriptions for project staff. First, new positions may be based on
existing ones if they exist, particularly for roles such as cataloguers,

Putting Content Online: A Practical Guide for Libraries

      systems administrators and software developers as the skills required for
      these positions are in general not specific to the work that is required in
      digital library development projects. On the other hand, many positions
      in Chapman’s list are quite specialised and project managers may have
      difficulty finding models for suitable job descriptions. Second, much
      digitisation work is repetitive and detail-orientated, particularly that of
      scanning or digitisation technicians. Therefore, it is important to build
      variety into their job descriptions. Two strategies for ensuring variety are
      (1) have each staff member fill multiple roles or (2) have each role
      perform a variety of tasks. The disadvantages of setting up positions in
      this way is that it may be more difficult to fill such positions because of
      the variety of skills required, and when a person leaves a project,
      multiple roles are left vacant.
         Planning for staff training in digitisation projects can be challenging.
      In short-term projects that have a defined duration, staff filling positions
      created specifically for the project will need to be trained early in the
      project period. For training to occur, hardware and software must be
      operational, and procedural documentation must be ready. Staff taken
      from other parts of the library will also need to be trained on project
      operations but this can sometimes be done before the training of new
      staff happens, provided documentation and tools required for their work
      are ready.
         As stated earlier, an individual project manager should be identified as
      being responsible for the project. This person is directly involved in the
      planning of the project, assembling and allocating the required resources,
      monitoring the progress of the work, managing the budget, and for
      evaluating the project and often for evaluating the collection after the
      work is complete. Project staff should report directly or indirectly to the
      project manager; if there is a separate supervisor of project operations,
      most project staff will report to this person, who in turn reports to the
      project manager. In small operations, the project manager and supervisor
      can be the same person, but in larger operations, the demands of both of
      these roles can quickly surpass a single person’s capacity. In fact, larger
      projects will probably need more than one supervisor, especially if the
      production work is done in shifts that exceed a normal work day or if it
      is done at more than one physical location.
         In some cases a committee, instead of a single individual, performs
      the role of project manager. Committees are common in large-scale or
      long-term projects, and in multi-institution projects. However, smaller,
      shorter projects are often best managed by a single individual who can
      draw on the expertise of others when needed.

                                                                Project management

  In Chapter 9, ‘Project operations’, I will cover the roles of project
supervisor, digitisation technician and quality control technician in

Hardware and software resources
If your library does not already have the hardware and software
resources necessary to create the desired content, you will have to
acquire them. Unfortunately, acquiring scanners and other digitisation
hardware is not as straightforward as it might sound, so project plans
must include suitable lead times if production is to start on a given date.
   A number of factors will help you determine the requirements for the
hardware and software you will need to create your collection’s content.
First, information gathered during the source material evaluation phase,
such as format (still images, sound, etc.), physical dimensions and physical
condition, should be considered when selecting digitisation hardware.
Second, fully developed workflows can assist in the selection of hardware
and software by helping define the most efficient set of steps in which to
create content and metadata. The desired set of steps can then be used to
define criteria for evaluating hardware and software. For example, if you
know that you will in general be using TIFF as the format for the master
versions of image files you are creating, you can include the ability to create
TIFF files in your scanner evaluation criteria. Limitations in hardware and
software can often be compensated for in workflows, but usually this
compensation involves additional tasks, which add to overall project costs.
The next chapter, ‘Project operations’, will provide additional, detailed
information on evaluating suitable hardware and software.
   It may be tempting to evaluate and acquire hardware and software
based on the requirements of a single collection or the workflows of a
single project, particularly if these resources will be purchased from
funds allocated specifically for that collection or project. However, if a
library intends to develop additional digital collections or participate in
collaborative collection development initiatives, acquiring hardware and
software that can produce files meeting a variety of criteria is a much
sounder strategy. Acquiring a scanner whose maximum output
resolution is the exact resolution required in the current project is less
preferable to acquiring a scanner that can output images at much higher
resolutions. Typically, the better the hardware, the more expensive it is,
but all other things being equal, if you can afford hardware that exceeds
the requirements of your current project, you should consider acquiring

Putting Content Online: A Practical Guide for Libraries

      it as future projects may have more demanding technical specifications
      than the current project. Realistically, however, you need to work within
      your budget, and if you can only afford digitisation hardware and
      software that meets but does not exceed the requirement of your current
      project, then that is what you should buy.
         To a large extent, exactly when you evaluate and acquire any new
      hardware and software does not matter, except that you will need to
      know the minimum technical specifications that content in your
      collection will adhere to, and you need to know if you have the funds to
      buy the required resources. Obviously you will need to have all tools in
      place in time to document procedures prior to training new staff. This
      applies to hardware, software used in the production of the content and
      metadata, and the CMS you will be using. Project deadlines and staffing
      levels can have an impact on how much hardware and software you will
      need. For this reason, many project managers wait until they know the
      final deadlines and staff budgets before acquiring new hardware.

      Producing digital content, even on a small scale, requires a substantial
      investment in hardware, software, space and staff resources. Many
      libraries are not in the position to make these investments or are
      otherwise happy to avoid making them. Instead, they send original
      material to a vendor, who creates the digital content and then delivers it
      to the library, ready for loading into the library’s CMS. Alternatively, the
      vendor may be responsible only for digitising the material and the library
      will perform other required tasks, such as creating derivative versions for
      presentation on the web. The latter case applies to libraries that wish to
      avoid investing in digitisation hardware but still want to be involved in
      the production of their content. Some libraries also outsource the
      creation of descriptive metadata, whereas others choose to create their
      own even if they outsource digitisation or conversion. In some cases, the
      library does not host its own collections, but contracts with a vendor to
      host them on its servers.
         When deciding whether to invest in the infrastructure necessary to
      develop and maintain a digital collection, a library may choose from a
      number of options:

        performing all work internally,
        hiring a commercial vendor to perform all or parts of the work
        required to produce the content,

                                                                Project management

   hiring a consultant to plan and oversee the project, with the work
   done either internally or by a vendor,
   collaborating with other libraries, and
   a combination of the above options.

Janet Gertz’s chapter cited in the ‘Further reading’ section at the end of this
chapter provides extensive information on the relative benefits of doing the
work internally or contracting with a vendor, on selecting a vendor and on
developing the various documents that are necessary when outsourcing
such as Requests for Information and Requests for Proposals. The
information Gertz provides is thorough and detailed and will prove to be
extremely useful to libraries considering contracting with commercial
vendors for the production of their digital content and metadata.
   Contracting with other libraries that act as ‘vendors’ is becoming
increasingly common. This type of relationship differs from
multi-institutional projects (discussed below): the ‘client’ library is
simply contracting with the ‘vendor’ library to do specific types of work
instead of contracting with a commercial vendor. Many libraries that
have invested in digitisation hardware, software and staff have the extra
capacity to do external work, particularly during periods of low activity
from their own projects. In these cases, the client and vendor libraries
must come to an agreement that is satisfactory to both parties, and the
vendor library must be able to perform the work at rates comparable
with those charged by vendors. This type of relationship may be
preferred when the client libraries feel most comfortable dealing with
other libraries, particularly if they cannot find a commercial vendor that
meets their needs. Also, it is not uncommon for larger university and
public libraries to act as digitisation and metadata production centres for
smaller libraries within their geographical region that may have
interesting collections but not the resources to digitise them. On the
other hand, some libraries may prefer to deal with commercial vendors
because they feel that it is easier to establish business-like relationships
with commercial entities or that it is best to collaborate with other
libraries on a more equitable basis than the client–vendor relationship
often implies.
   Hiring a consultant to assist in planning and to oversee project
operations is yet another option. A good consultant will have experience in
digital library content development, will balance the goals of the collection
with the resources available for the project, and will demonstrate flexibility
in how problems are solved. Avoid consultants who are tied to a particular
set of technologies (unless the library would adopt those same technologies

Putting Content Online: A Practical Guide for Libraries

      independent of the consultant’s recommendations). Keep in mind that a
      consultant cannot assist a library with a digital collection project (or any
      other type of project) unless library staff assist the consultant: clearly
      communicating the expected outcomes of the consultant’s work and freely
      providing the information the consultant needs are essential. The best
      possible outcome of hiring a consultant for this type of project (other than
      a successful digital collection, of course) is successful knowledge transfer;
      in other words, library staff should be able to plan and manage similar
      projects on their own in the future, if doing so is part of the library’s
      long-term digital collection development programme.
        In addition to outsourcing the production of digital content and
      metadata, some libraries choose to make their collections available using
      CMSs hosted externally. Options for making the collection available to
      end users include:

        host the collection on servers maintained by the library’s parent
        host the collection on a commercial vendor’s servers, or
        host the collection on another library’s servers.

      Like selecting an external source for content production, selecting a
      vendor or other library to host a collection allows libraries with content
      to create digital collections without investing in the required
      infrastructure. Many libraries do not run their own servers or do not have
      the technical staff required to maintain a CMS. On the other hand, many
      libraries would rather host a CMS than acquire scanners and specialised
      software, and employ the staff necessary for creating digital content.

      Preparing a budget
      Preparing a budget for a digitisation or conversion project can be
      somewhat complex because of the following factors:

        Variability in workflows for different projects: Digitising or converting
        still images, text, video and audio source material all require different
        sets of tasks, and even within each type of source material there can
        exist a wide range of workflows (we have seen, for example, that there
        are several substantially different approaches to digitising texts).
        Collection functional requirements can vary widely as well, as can
        options for creating metadata. This matrix of implementation variables
        can result in real challenges for project managers who need to estimate

                                                               Project management

  how much it will cost to create a given collection. Costs are determined
  by the nature of the project workflow, and if mangers cannot predict
  workflows, they cannot predict costs with much accuracy.
  Inconsistency in source material: Even though careful project
  managers will base cost estimates on representative samples of the
  source material they are working with, many collections of source
  material contain hidden surprises that complicate production
  significantly and therefore have a real impact on a project’s overall
  budget. This impact can be considerable given the cumulative effect of
  high numbers of unexpected exceptions.
  Limitations of technology: Whether due to temporary problems or
  due to systemic limitations, problems with technology can
  dramatically affect the likelihood that a project will be completed on
  time and within its budget.
As already stated, by far the safest way to estimate the costs involved in
producing specific digital content is to develop appropriate workflows and
procedures, perform trials, document the cost of the trials, and then using
those costs extrapolate the cost of processing the entire collection of source
material. Any other method of determining costs is an educated guess. Even
in the unlikely event that your institution will accommodate budgets that
are off by 50–200% (to pick arbitrary figures), it is safe to say that few
external funders will. They will typically want to know exactly how much
it will cost to accomplish the goals you describe in the grant applications.
   This type of evidence-based budgeting is not easy. It is difficult if not
impossible to do on short notice, it can require resources (such as
hardware) that are not available prior to receiving the funding you are
applying for, and it can be inaccurate if staff performing the trial runs are
not proficient at using the available workflows and tools, which they may
not be without a few weeks’ experience. It can also create a misleading
sense of confidence in cost estimates, particularly if the workflows and
procedures are not fully developed or if the source material being tested
is not typical of the entire collection. Finally, even though evidence-based
budgeting provides fairly accurate data for estimating the costs of staff
who are performing the tasks, it does not necessarily provide accurate
data on how much time supervisors and project managers spend on tasks
not directly related to production, such as documentation, supervision
and writing reports. Despite these drawbacks, using empirical evidence to
estimate costs lends authority to budgeting, and even if these informed
estimates prove to be inaccurate, the information derived from trials will
allow you to identify the areas that are inflating actual production costs.

Putting Content Online: A Practical Guide for Libraries

        In some cases it is not necessary to perform trials on representative
      samples of source material in order to derive an accurate budget if the
      project you are planning is very similar to ones you have performed in
      the past. In these cases, the known costs from the previous projects can
      act as a reliable indicator of the likely costs in the new project. However,
      this method is reliable only to the extent that the source material,
      metadata requirements and workflows of the two projects are the same.
      Even slight differences between source material and technical
      specifications can cause estimated costs to be inaccurate, particularly if
      the number of items in your collection is large.

      Cost components
      The largest costs for a digitisation project are operations staff (staff
      performing permissions clearance, metadata production and content
      creation; supervisors; project managers, etc.), hardware, software,
      copyright clearance fees and a CMS, whether a vendor product or open
      source. If your library already has suitable hardware or software, or a
      suitable CMS, those items will not represent additional real costs (although
      they may be applicable as in-kind contributions on grant applications, as
      described below). The most common type of cost for any project is staff. It
      is difficult to generalise about the relative proportions of the various types
      of cost in a project because of the wide variability in hardware, software
      and CMS prices, and because staff costs are unknown for any project until
      workflows have been tested. Using documented costs from a series of large-
      scale digitisation projects, Steven Puglia found that roughly one-third of
      the cost of the production of those collections accounted for digitisation,
      one-third for metadata creation, and the final third for administration,
      quality control and other activities.2 Although these findings were derived
      from actual recorded costs, they should be used as a general guide only and
      not as an accurate planning formula, for reasons stated earlier.
         After production is complete, ongoing costs include collection
      maintenance, promotion and evaluation. Preservation of the digital
      content will also have real costs, and as we saw in Chapter 5, these are
      at best difficult to predict.

      An example budget
      The following example illustrates how project managers can develop a
      budget based on performing trials of the tasks identified in a project’s

                                                              Project management

workflow. The example is simple but realistic, and includes every aspect of
the digitisation and metadata creation activities necessary to put a small
collection of audio tapes online. It includes estimates for two positions
other than the staff involved in content and metadata production, a web
developer who is configuring a CMS for the collection, and a project
manager who is responsible for supervision, documentation, training and
quality control. It does not include the work necessary to clear permission
to mount the material on the web. The collection consists of 110 tape
recordings of authors reading short fiction or excerpts from longer works.
  In preparation for developing this budget, we have developed a
workflow and digitised a number of samples. From these trials, we have
determined that the following are reasonable average amounts of time
for processing each reading (Table 8.3):

  Table 8.3       Workflow tasks and average times for processing
                  sample audio recordings

 Task                                                    Estimated time
 Retrieving and preparing each tape                      15 minutes
 Converting audio tape to WAV and naming the file        20 minutes
 according to documented procedures
 Converting each WAV file to mp3 format for the web      10 minutes
 and copying it to the designated archive directory
 Creating descriptive metadata for each sound file       20 minutes
 Adding administrative metadata to the CMS, copying       10 minutes
 the master WAV file to the archive drive, and adding the
 mp3 file into the CMS

  A note about converting from audio tape to WAV: the conversion must
be done in real time (i.e. a tape that is 15 minutes long takes 15 minutes to
convert). Even though the technician does not have to sit at the workstation
during the conversion but can do other tasks – in theory he or she could be
preparing the next tape while the previous one is playing and being
digitised – we have decided to account for the times as if the technician did
not perform other tasks while waiting for the tape to finish playing.
  Another note, this time about how to perform the budgeting trials: for
most projects where textual content is being produced, the number of
pages is a useful unit to base timelines on as each page needs to be
scanned separately. For image-based collections, the number of images

Putting Content Online: A Practical Guide for Libraries

      that are required would be an appropriate unit; for collections of audio
      or video that must be digitised, the number of minutes or hours may be
      more appropriate as these formats must be digitised in real time.
        The Conversion Technician, who retrieves each tape, digitises it and
      converts the files from WAV to mp3, earns $15.00 an hour plus 8%
      benefits, and the Metadata Specialist, who applies all metadata and adds
      the files to the archive directory, earns $20.00 an hour plus 9% benefits.
      To calculate the cost of digitising the entire collection and creating the
      accompanying metadata, we multiply the average times identified above
      by the number of tapes (Table 8.4):

        Table 8.4        Estimated time required to digitise entire collection
                         and create metadata

       Task                               Estimated time   Total time required to
                                          required to      complete the project
                                          complete the
       Retrieving and preparing each      15 minutes/tape 0.25 hours/tape × 110
       tape (Conversion Technician)                       tapes = 27.5 hours
       Converting from audio tape to      20 minutes/tape 0.33 hours/tape × 110
       WAV and naming the file                            tapes = 36.3 hours
       according to documented
       procedures (Conversion
       Converting each WAV file to mp3 10 minutes/tape 0.17 hours/tape × 110
       format for the web and copying                  tapes = 18.7 hours
       it to the designated archive
       directory (Conversion Technician)
       Creating descriptive metadata      20 minutes/tape 0.33 hours/tape × 110
       for each sound file (Metadata                      tapes = 36.3 hours
       Adding administrative metadata 10 minutes/tape 0.17 hours/tape × 110
       to the CMS, copying the master                 tapes = 18.7 hours
       WAV file to the archive drive, and
       adding the mp3 file into the
       CMS (Metadata Specialist)

         The costs for the Web Developer and the Supervisor are calculated
      separately (Table 8.5). The former is a freelance consultant who charges
      a flat rate and has supplied a quote indicating that the work will not take
      more than 10 hours to complete. The Project Manager/Supervisor’s salary
      is more difficult to estimate accurately, but based on similar work done

                                                                 Project management

  Table 8.5       Tasks and salaries for the Web Developer and

Type                                      Estimated time      Estimated cost
Web Developer, who has to modify          10 hours            $40/hour, no
some of the CMS web templates for                             benefits
publication of these files. This activity
does not include any other activity.
(Position 3)
Administration, including project         50% of all staff   $32/hour, plus
development, documentation, staff         time including the 12% benefits
training and supervision, and quality     Web Developer
control (Position 4)

at the library, we estimate that the Supervisor will need to spend half of
the total time required by the Conversion Technician and Metadata
Specialist to digitise the material and create metadata. This estimate is
relatively high but it includes the time required to write the procedures
manual and to perform quality control on the digitised sound files (by
listening to the beginning, middle, and end of each reading, for example).
   Now that we have estimated costs for all of the staff, we can add them
together in the following spreadsheet (Table 8.6; numbers of hours are
rounded up to the nearest hour):

  Table 8.6       Total staff costs for processing the audio

Title                                   Conversion Technician
Number of hours                         28 + 37 + 19 = 84 hours
Hourly rate and total salary            $15.00/hour × 84 hours = $1260.00
Benefits (8% of total hourly wage)      $100.80
Total cost for Position 1               $1360.80
Title                                   Metadata Specialist
Number of hours                         37 + 19 = 56 hours
Hourly rate and total salary            $20.00 × 56 hours = $1120.00
Benefits (9% of total hourly wage)      $100.80
Total cost for Position 2               $1220.80

Putting Content Online: A Practical Guide for Libraries

        Table 8.6          Total staff costs for processing the audio
                           recordings (Cont’d)

       Title                                 Web Developer
       Number of hours                       10
       Hourly rate and total salary          $40.00
       Benefits (0% of total hourly wage)    $0.00
       Total cost for Position 3             $400.00
       Title                                 Supervisor
       Number of hours                       70
       Hourly rate and total salary          $32.00 × 70 hours = $2240.00
       Benefits (12% of total hourly wage) $268.80
       Total cost for Position 4             $2508.80
       Total staff costs                     $5490.40

         This example illustrates a linear workflow, which means that all of the
      tasks are performed sequentially. To budget for parallel workflows, in
      which independent tasks are performed at the same time and the output
      from each is combined later, we need to add additional rows in our
      spreadsheet for each of the parallel tasks and then add up all of the costs.
      In other words, the time required to digitise the source material using
      parallel workflows is cumulative – we need to pay the salaries of all the
      staff who contribute to the creation of a digital document, even if they
      are working at the same time but on separate tasks. For example, if we
      had someone transcribing the audio tapes as they were being digitised in
      order to create a searchable ‘full text’ for each tape, the salaries for both
      of these staff would need to be included in our budget.

      Additional planning considerations
      Once the amount of staff time required for a project is known, we can
      estimate the number of staff we need to hire, the amount of hardware
      and software we need to procure, and the amount of physical space
      required to do the work.
         After fully developed workflows, the next issue to resolve is how much
      time is required to complete the project. The number of work days
      available to you will let you determine how many staff to hire, which in
      turn will determine how much hardware you need and how much

                                                                 Project management

physical space you need. These numbers can be determined by various
factors, such as the amount of time a funder gives you to spend
their money, the end dates of fiscal years, the availability of staff
(e.g. scheduling work shifts around major holidays is often problematic),
or other factors. Ultimately, you will need to determine when you plan
to launch your collection. This date may or may not be flexible, and you
may need to launch the collection before all the content and metadata
are ready, but in practice you will probably have to juggle all of the
timing factors listed here in order to get a fairly accurate idea of how
long your project will take.
   Given the total amount of time required to complete the tasks defined in
our workflow, and the number of days available within the project timeline,
we can estimate the number of hours of work per day that need to be
completed. Given the total number of hours of work, we can determine the
number of staff required, and from that number, the amount of hardware
and physical space they will need to complete the work. This formula is only
a guide – adjustments may need to be made in response to contingencies such
as unpredictable availability of staff (e.g. most students do not want to work
at exam time) and delays due to hardware and other problems. However, the
total number of hours of work required to complete a project is a useful basis
on which to estimate associated resources.
   First, for the conversion technician and metadata specialist, we need to
calculate the number of work hours per day required to complete our

      Number of hours per day required to complete all work = Total
      number of hours required to complete all work/number of
      available days

For the sake of this example (Table 8.7), we are told by the library
administration that we need to spend all of the money allocated to this
project within 4 weeks (or 20 working days):

  Table 8.7         Hours of work per day required to complete all work

                  Hours required to                    Hours per day required
 Position title   complete all work   Days available   to complete all work
 Technician              84                 20                   4.2
 Specialist              56                 20                   2.8

Putting Content Online: A Practical Guide for Libraries

        Using this figure of 4 weeks and the number of hours in a standard
      working day (in this case 7 hours per day for full-time equivalent staff,
      or FTE), we can determine the number of positions we need for the
      conversion technician and the metadata specialist in order to complete
      work within the given timelines (Table 8.8):

        Table 8.8        Positions required to complete all work
                         in days available (FTE)

                                                               Positions required
                           Hours in standard Hours per day     to complete all
                           working day       required to       work in days
       Position title      (1 FTE)           complete all work available (FTE)
       Technician                  7                      4.2          0.6
       Specialist                  7                      2.8          0.4
       Supervisor                  7                      3.5          0.5

            Number of positions required to complete all work = Number of
            hours per day required to complete all work/number of hours in a
            standard working day (for 1 FTE)

      The right-most column shows that the number of positions required is
      less than one FTE for each position title. This figure is useful in
      determining how many people to hire, and whether their positions will
      be full time or part time. In our example, because all of the positions are
      less than one FTE, we may be able to complete the project in fewer than
      20 days if we make the positions full time; if the number of positions
      were more than one FTE, we would need to hire more than one person,
      or increase the number of days in which we would be able to complete
      the project (which the library administration says is not an option).
         At this point we should finalise our project milestones. In the earlier
      example in which we devised some preliminary milestones for the
      digitisation of the local newspapers, we did not have much information –
      only how much content was intended to go into the collection and an
      estimate of how long we had to complete the project. In this example,
      however, we know an additional and important piece of information:
      how many staff we will need to complete the project. Once we know
      that, we can finalise our project milestones. If we monitor the project’s

                                                              Project management

progress carefully, we will be able to detect any significant deviations
from these refined milestones and take corrective measures if necessary.
   Also, now that we have determined the number of staff we will need
to hire, we can determine the amount of work space and hardware
required. In our example, we will need a separate workstation for the
Conversion Technician and Metadata Specialist, and a separate
workstation for the Supervisor (the Web Developer will work at home).
The Conversion Technician will need to have the tape player arranged
near his or her computer workstation in order to capture the audio. If
circumstances were such that we needed to hire more than one FTE staff
member for each position (if, for example, we had 80 hours of audio to
capture and catalogue in the same number of days), we would need to
hire more than one person for each position, which would require more
work space, more computer workstations (each with the necessary
software), and in the case of the Conversion Technician, more than one
tape player and audio capture card.

Executing the project
At this point we are ready to finish project documentation, hire and train
staff, and begin the actual production of our content and metadata. Any
required hardware, software and work space must now be in place as
well. The next two chapters deal with project operations and workflow
development, and expand substantially on the topics introduced here.

Proposal writing
Project managers are usually required to prepare formal proposals for
funding. Funders can be internal (to the institution), private (individuals,
foundations, corporations or other organisations) or public (government
bodies at any level, from municipal to international). In this section I will
introduce some general information about proposal writing. The
jurisdiction that a library is part of can have a significant impact on its
eligibility for a particular source of funds, so I will not identify specific
sources of funding.
   The process of applying for funding will be defined differently by each
funder. In some cases, the funder will provide a standard form that needs
to be completed, sometimes with attachments or appendices containing

Putting Content Online: A Practical Guide for Libraries

      narrative responses to particular questions. The form and all supporting
      documentation must then be submitted by a given date. In other cases,
      particularly where the amount of available funds is high and the time
      over which they can be spent is long (e.g. from a full year to several
      years), the application process may involve multiple submissions
      delivered at different deadlines. In these cases, funders will issue
      invitations to applicants to submit to the next phase of the process, with
      each submission containing more detailed information. Finally, some
      funding organisations rely on a less formal application procedure that
      does not involve standardised forms, but instead defines criteria (ranging
      from general to quite specific) that applicants address in their proposals.

      Proposal components
      Despite this variety of requirements, most funding proposals contain at
      least some of the following components. Both this list of components
      and the list of terms described in the next section are general and not
      necessarily comprehensive. However, they do include some of the more
      common aspects of funding proposals that you may encounter.

        Letter of intent: In application procedures where multiple components
        must be submitted at different times, the first component often takes
        the form of a ‘letter of intent’ indicating that the institution intends to
        pursue the grants in question and that may give very general
        information about the proposed collection or project. The funder
        responds to this letter with an invitation to proceed with the
        remainder of the application or a statement of why the applicant is
        not being invited to proceed.
        Executive summary: A brief summary of an application, typically
        containing information such as the name of the applying institution,
        names of major partner institutions, title and nature of the collection,
        fund(s) being applied for, amount being applied for, and a brief
        indication of the relevance of the collection or activity to the funder’s
        stated objectives.
        Description of your organisation: Brief and selective description of the
        library applying for the funds. Frequently this information is detailed in
        a form that may ask basic information such as principal contact person,
        year founded, charitable organisation status, number of employees, and
        so on, while in other cases the funder may require detailed information
        about the library’s previous projects, partners and funders.

                                                              Project management

  Project objectives: A clear statement of what the funds will be spent
  on, using language that the funders will understand.
  Description of how funder’s objectives will be met: Not all funders
  support ‘digitisation’, but many support job creation for students,
  building databases of local content or some other activity that you can
  incorporate into your project. Funders want to know that their money
  is being spent on things that matter to them, and you should be
  prepared to explain why your project meets their objectives.
  Project work plan: Some funders require a work plan, which is
  typically an outline or table indicating major milestones in the project.
  Project budget: Budget formats vary widely, from simple forms to
  sophisticated Excel spreadsheets that include formulas to calculate
  items such as administration charges, salary calculations, and
  maximum allowable amounts in various categories such as hardware
  or copyright licensing fees.
  Appendices: Some applications may require other documentation,
  often grouped together as appendices, attachments or schedules.
  These documents can vary widely depending on the requirements of
  the application, but can include letters of support from partners (both
  for the current project and for previous projects), letters or special
  forms from officials in the library’s parent institution, screen captures
  or printed pages from prototype websites if applicable, and sample
  job descriptions or job advertisements.

Assembling the information required for most applications can be time
consuming and difficult. For large grants it is not uncommon to treat the
application process itself as a ‘project’ that requires hiring consultants or
providing temporary replacements for existing staff. Although the
amount of effort required to complete most application procedures
should not require additional staff, you should assume that any formal,
structured application process will take considerable coordination,
particularly if you must involve people outside your library for letters of
support, financial information or other reasons. Most funders will not
accept applications past their stated deadlines.

Proposal terminology
The following terms are used frequently in funding applications. In some
cases, application documentation will actually contain definitions of the
terms used throughout the application and even some examples of how

Putting Content Online: A Practical Guide for Libraries

      the terms are intended to be used. Obviously, paying close attention to
      the instructions for completing applications is extremely important.
      Missing components or misinterpreted questions can disqualify an
      application, and if the funder receives many more applications than they
      intend to fund, they will be looking for opportunities to make their
      adjudication easier.

        Partners: Many funding agencies favour applications submitted on
        behalf of multiple institutions – in fact for larger grants this may be a
        requirement. Typically, one institution must be identified as the ‘lead’
        or ‘principal’ partner, if for no other reason than to simplify the
        funder’s accounting procedures (the lead partner would be responsible
        for distributing funds to partners). The term ‘partner’ will probably be
        defined within the application documentation, but participants of all
        types, from individuals to entire organisations such as commercial
        companies, universities or government departments, may be
        considered partners.
        In-kind contribution: ‘In-kind’ describes resources that the institution,
        as opposed to the funder, is expected to supply. As this term is
        generally used within the context of a project budget, it will usually be
        defined very specifically; for example, some funders consider staff
        salaries to be in-kind, whereas others do not. In-kind contributions are
        also often subject to formulas that qualify the value of the
        contributions in some way, such as reducing the value of a $5000
        scanner to $500 that can be applied to the institution’s contribution. In
        addition to staff resources, other typical in-kind contributions include
        hardware and physical space for digitisation and related activities.
        Cash: In contrast to in-kind contributions, cash is typically money
        that an institution must contribute to a project that they would not
        spend if they were not participating in the project (in other words, it
        must be additional to their normal operational costs). The most
        common costs that cash contributions apply to are staffing and
        hardware. In general, the same resources cannot be considered cash
        and in-kind contributions, although some funders allow claiming staff
        time as a ‘cash in-kind’ contribution.
        Outcomes: Funders will frequently ask for a statement of ‘outcomes’,
        which are generally equivalent to your project’s goals. To be effective,
        outcomes should be worded such that they are understandable to
        readers of your application (i.e. free of library jargon) and consistent
        with the funder’s objectives. Many funders will use language that is

                                                              Project management

  tied very closely to their mandate or organisational objectives. For
  example, government agencies at various levels may allocate funds to
  supporting projects that employ youth; some large private
  foundations have been requiring any software created with their funds
  to be released under a specific open-source licence. When describing
  outcomes, it can be advantageous to adopt some of the language used
  in the proposal documents, particularly when discussing how your
  project’s objectives align with those of the funder.
  Deliverables: Deliverables tend to be more concrete and countable
  than outcomes. Items that might be expected in response to a question
  about what deliverables a project will produce include a description
  of the website/CMS, the number of documents, any standards or
  reports, or other documentation or software developed during the
  creation of the content (CMS platforms, software utilities, etc.).
  Evaluation: Many grant applications will ask how you plan to
  evaluate the success of your project, and may even ask you to write a
  final report and sign a release so they can use the report for their own
  purposes. I will cover project evaluation later in this chapter.
  Sustainability: Finally, some funders are interested in how long their
  investment will pay off. If they fund early stages of a project, they may
  ask what its long-term sustainability is and what the library’s long-
  term goals for the project are. Your response to this type of question
  will depend on your institution’s and your partner’s commitment to
  the project. If a collection is open (content will continue to be added),
  a statement of sustainability should address how the ongoing activity
  will be funded; if a collection is intended to be closed (for example,
  the scope of the collection is narrow and the content appropriate to
  that scope is likely to make it into the collection within the defined life
  of the project), a statement of sustainability may only need to address
  issues of preservation and ongoing evaluation.

If your grant application is successful, you will be expected to report to
the funders periodically. Many funders will require clear and
comprehensive reports of how their money is being spent. This is as true
of funds allocated internally from the library as it is of funds awarded
from an external organisation. Every funder will define its own reporting

Putting Content Online: A Practical Guide for Libraries

      requirements, but typically they will include periodic statements of how
      much money has been spent, statements of periodic quotas or milestones
      for deliverables, and progress on other work that is being funded.

      Monitoring is essential for any type of project, not just projects
      producing digital collections. Periodic checks on the progress being made
      and the identification and correction of any problems as they occur will
      dramatically increase the chances that a project will meet its defined
      goals and will do so without exceeding the allocated resources.
         The most important aspects of monitoring digital content creation
      projects are ensuring that objectives and milestones are being met,
      ensuring that the available budget is not being overspent, holding regular
      staff meetings and documenting the outcomes of issues resulting from
      those meetings. First, it is more efficient to keep production on schedule
      throughout a project than to try to make up for low production at the
      end of the project. Available space, hardware and staff resources will
      probably be insufficient to increase the output required toward the end
      of the project to meet overall objectives. Second, production milestones
      should be linked to ongoing costs, so that at any time during the project
      you can predict if the funds allocated to the work will be sufficient.
      Third, communicating with production staff regularly through regular
      meetings ensures that problems they encounter are dealt with as early as
      possible. Finally, because one of the goals of monitoring a project is to
      address potentially serious issues as early as possible, documenting
      problems as they arise and are resolved will allow even faster resolution
      of similar problems should they arise again.

      Evaluating the production phase of the
      In a general sense, the content production phase of a digital collection
      project should be considered successful if it reaches the project’s stated
      goals. However, evaluating projects according to specific criteria is
      desirable for a number of reasons: evaluating a project after its
      completion can demonstrate to funders that their money was spent the

                                                            Project management

way they intended it to be spent, can validate a project and can identify
areas for improvement. The most useful strategy for evaluating a project
is to build evaluation into the project’s goals. In other words, one of the
project’s goals should be to undergo a systematic evaluation. Explicitly
stating this goal will help keep the project focused and will encourage
project managers to maintain accurate and thorough documentation.
   The exact criteria by which projects will be evaluated should be
determined by the desired outcomes of the evaluation, which should be
defined by the project manager(s), the library administration and the
project funders. Criteria will fall into the following categories:

  Production milestones: Typical criteria include the project’s ability to
  meet defined production milestones and objectives, and what
  constitute acceptable deviations from defined objectives.
  Budget: Obviously, the project’s ability to accomplish its goals within
  the defined budget is important. Using substantially less money than
  the amount defined in the budget is less likely to happen than
  overspending, but funders may not look favourably on what they
  would perceive as an overestimation of the funds required to complete
  the project. Careful monitoring will help avoid this situation.
  Quality benchmarks: Evaluating the ability to meet defined levels of
  quality over the duration of the production activity is relatively easy
  (provided standards of acceptable quality have been clearly defined
  and documented) and should provide no surprises if quality control
  has been performed throughout the project.
  Operational aspects: Evaluation of the suitability of the hardware,
  software and workflows is useful as a planning tool for subsequent
  projects. Staff job descriptions, scheduling and other aspects of
  staffing should also be reviewed at the end of each project.

To summarise, careful monitoring during the project is a form of formative
evaluation, but summative evaluation of the work involved in creating
digital content is an important aspect of overall project evaluation.

Evaluating the overall project
Content production is only one aspect of the ‘project’ associated with a
digital collection. Initial and ongoing promotion, the benefits to users,
financial sustainability and ongoing maintenance issues should also be

Putting Content Online: A Practical Guide for Libraries

      evaluated. The NISO Framework of Guidance for Building Good Digital
      Collections defines four principles by which projects should be evaluated:

      1. A good collection-building project has a substantial design and
         planning component.
      2. A good project has an evaluation plan.
      3. A good project produces a project report and broadly disseminates
         information about the project process and outcomes.
      4. A good project considers the entire lifecycle of the digital collection
         and associated services developed through the project.3

      The second principle may sound tautological, but it actually emphasises
      the need for systematic evaluation. As the Framework states, ‘An
      evaluation plan demonstrates the commitment of a project to its stated
      goals and objectives.’ To support development of effective evaluation
      plans, the Framework cites the Institute of Museum and Library Services’
      list of resources on Outcome-Based Evaluation (OBE).4 As the IMLS
      describes it, OBE determines the impact of digital collections on users’
      knowledge, attitudes, skills and behaviours. This impact is evaluated by
      using techniques such as online questionnaires that test users’
      knowledge. The IMLS website offers this example:

            In order to know if online availability had a benefit, an institution
            needs to measure skills, attitudes, or other relevant phenomena
            among users and establish what portion of users were affected.
               To capture information about these kinds of results, a library or
            museum could ask online visitors to complete a brief questionnaire.
            If a goal is to increase visitor knowledge about a particular
            institution’s resources, a survey might ask questions like, ‘Can you
            name 5 sources for health information? Rate your knowledge from
            1 (can’t name any) to 5 (can name 5).’ If visitors rate their
            knowledge at an average of 3 at the beginning of their experience,
            and 4 or 5 (or 2) at the end, the sponsoring institution could
            conclude that the web site made a difference in responders’
            confidence about this knowledge. It should be clear that such a
            strategy also lets you test your effectiveness in communicating the
            intended message!5

      Regardless of whether the narrowly defined activity involved in content
      production is being evaluated or whether the entire set of activities
      surrounding the development of a digital library collection is being

                                                            Project management

evaluated, a thorough and effective evaluation plan should be part of the
overall project goals.

Evaluating the collection
The third type of evaluation that project managers should perform is
evaluation of the collection. This evaluation is actually part of ongoing
collection management, and will happen after the ‘project’ has ended,
whether defined narrowly by its production phase or more broadly by a
larger set of activities.
   The NISO Framework of Guidance for Building Good Digital
Collections provides seven principles of ‘good’ collections (good
meaning generally appropriate for the user groups the collection is aimed
at) that are independent from aspects of projects described above. These
principles can be used both as planning tools and as evaluation criteria:

1. A good digital collection is created according to an explicit collection
   development policy that has been agreed upon and documented
   before digitisation begins.
2. Collections should be described so that a user can discover
   characteristics of the collection, including scope, format, restrictions
   on access, ownership, and any information significant for determining
   the collection’s authenticity, integrity and interpretation.
3. A collection should be sustainable over time. In particular, digital
   collections built with special internal or external funding should have
   a plan for their continued usability beyond the funded period.
4. A good collection is broadly available and avoids unnecessary
   impediments to use. Collections should be accessible to persons with
   disabilities, and usable effectively in conjunction with adaptive
5. A good collection respects intellectual property rights. Collection
   managers should maintain a consistent record of rights-holders and
   permissions granted for all applicable materials.
6. A good collection has mechanisms to supply usage data and other
   data that allows standardised measures of usefulness to be recorded.
7. A good collection fits into the larger context of significant related
   national and international digital library initiatives.

Putting Content Online: A Practical Guide for Libraries

      Project managers should develop mechanisms during production that
      will capture the information needed for testing these attributes, such as
      entrance or exit surveys on the collection website, usage loggers,
      in-person interviews, focus groups or general feedback forms that users
      can complete. Part XII of the NINCH Guide to Good Practice in the
      Digital Representation and Management of Cultural Heritage Materials6
      provides detailed discussion of the types of mechanisms that can be used
      to collect information from a collection’s users. OBE, described above,
      can also be applied to the collection on an ongoing basis if desired.

      Multi-institution projects
      Increasingly, libraries are collaborating on digital collection-building
      activities. This collaboration is different from the client–vendor
      relationships between libraries described earlier. Partner libraries come
      together to create digital collections for the same reasons they come
      together co-operatively to subscribe to commercial databases and
      electronic journal collections or to share virtual reference services: to
      form strategic alliances, to pool resources and to please funding agencies,
      which to a large extent favour multi-institution projects over ones
      undertaken by a single institution.
         The advantages of collaborating that are specific to digital collection
      building are that the costs of specialised hardware and software can be
      distributed, expertise can be shared, large projects can be delivered in
      shorter timeframes, and libraries that are not able to invest in hardware
      and software can participate by performing other types of necessary
      tasks, such as clearing copyright permissions and creating descriptive
      metadata. Collaborative efforts also have the disadvantage of increased
      administrative overhead, which can manifest itself in a number of ways,
      including record-keeping requirements that are not compatible with all
      partners’ local institutional practices, scheduling meetings and holding
      them via teleconference or video conference, etc., and making sure that
      all partners meet project milestones and deadlines.
         From a practical perspective, multi-institution projects introduce a
      number of complications that need to be taken into account. First,
      development of workflows that are distributed among multiple partners
      can be challenging simply because having multiple sites provides more
      ways of getting the work done: in some multi-institution projects, most
      or all of the work is carried out in parallel at multiple sites, whereas in

                                                             Project management

others, each partner performs different sets of tasks that are then
integrated centrally. In order to maximise efficiencies, the implications of
distributed workflow should be explored carefully during the planning
phases of multi-institution projects.
   Second, all participating institutions must conform to the project’s
technical specifications. Care must be taken that file formats, file
densities, colour depths, sampling rates, metadata formats, the
application of metadata creation guidelines and file naming conventions
are consistent across all partners. Use of the same hardware and software
at all contributing institutions promotes consistency, but in reality being
able to use identical resources at each site is not common, particularly if
each site possessed its own hardware and software prior to the start of
the project. However, regular checks of output from partners will ensure
that inconsistencies are minimised.
   Third, the number of participating institutions can have a considerable
impact on the project’s ability to meet production milestones. The more
partners fall behind, the harder it is to recover; by contrast, the more
partners are involved in producing digital content, the more likely it will
be that shortfalls can be taken on by other partners if they have the extra
capacity. All of these issues can be mitigated with sufficient planning and
continuous communication among partners.

Summary: managing digital collection
Developing a digital collection encompasses a relatively complex set of
tasks. However, if the collection’s goals have been defined clearly, those
tasks can be grouped together and performed in an effective order. The
group of tasks surrounding the production of the digital content can be
considered the core of the overall set of activities required to make a
collection available to users, but it cannot happen without considerable
planning and preparation. In particular, the development of workflows
early in the overall project timeline is essential for the success of the
project. Budget, the role of digitisation vendors, the roles of partners in
multi-institution projects, local hardware, software and staffing
requirements all depend on effective, tested workflows (so much so that
I will spend an entire chapter on this topic).
  The production of the digital content is not the only aspect that must
be planned carefully. Effective documentation, proposal writing,

Putting Content Online: A Practical Guide for Libraries

      evaluation of both the production phase of the project and the project as
      a whole, and ongoing evaluation of the collection over time are also
      tasks that project managers must perform or oversee.

      Further reading
      Gertz, J. (2000) ‘Vendor relations’, in Handbook for Digital Projects: A
         Management Tool for Preservation and Access. Edited by M. K. Sitts.
         Andover, MA: Northeast Document Conservation Center, pp. 141–53.
         Available at
      Gertz explains the benefits of both digitising in house and outsourcing,
      and covers all important aspects of working with digitisation vendors
      not covered in this chapter, including selecting a vendor, writing a
      Request for Information and Request for Proposal, evaluating responses
      from vendors, drawing up contracts and dealing with quality control

      Library Hi Tech. Special issue on collaborative digitization projects. Vol.
        23:2 (June 2005).
      This special issue contains eight articles describing collaborative
      digitisation projects from the USA, including articles on sustainability,
      partnerships between libraries and museums, multi-institution training
      and support, CMSs, and benefits to end users.

      New York Public Library. Picture collection online project documents.
        Available at
      This site contains links to documents NYPL submitted to the IMLS for
      their Picture Collection Online funding application, including the
      proposal abstract and narrative, general workplan, deselection criteria
      (for identifying images that would not be included in the collection),
      quality control procedures, metadata guidelines and OBE proposal and

      NISO Framework Advisory Group (2004) A Framework of Guidance
        for Building Good Digital Collections, 2nd edn. Bethesda, MD: NISO.
        Available at
      Cited several times in this chapter and elsewhere in this book, the NISO
      Framework provides a number of principles of digital collections,
      objects, metadata and projects that will be useful to project planners and

                                                                  Project management

managers, and also provides links to selected resources that support and
expand on those principles. Each section contains a brief case study.

1.   Chapman, S. (2000) ‘Considerations for project management’, in Handbook
     for Digital Projects: A Mangement Tool for Preservation and Access. Edited
     by M. K. Sitts. Andover, MA: Northeast Document Conservation Center,
     p. 27. Available at
2.   Puglia, S. (1999) ‘The costs of digital imaging projects’, RLG DigiNews, 3:5.
     Available at
3.   NISO Framework Advisory Group (2004) A Framework of Guidance for
     Building Good Digital Collections, 2nd edn. Bethesda, MD: NISO.
     Available at
6.   The Humanities Advanced Technology and Information Institute and the
     National Initiative for a Networked Cultural Heritage. (2002) The
     NINCH Guide to Good Practice in the Digital Representation and
     Management        of    Cultural     Heritage     Material.    Available   at


Description: Project management