Bench-marking on a national scale by 34xmeUU

VIEWS: 15 PAGES: 19

									the 2007 LibQUAL Canada Experience

BENCH-MARKING ON A NATIONAL SCALE

                 2008 Library Assessment Conference




                  Sam Kalb, Library Assessment & IT Projects Coordinator
                  Queen’s University Library, Kingston, Ontario, Canada
                   Email: kalbs@queensu.ca
WHY NOT JUST DEVELOP A CANADIAN SURVEY?

   LibQUAL+™ established survey instrument for
    academic libraries
   Challenges & costs to build a better Canadian survey
    instrument & national support infrastructure
LIBQUAL+™ AND THE CANADIAN CONTEXT
   20 Canadian LibQUAL+™ participants to 2006 but
    never more than 10 in any given year
   Need to develop relevant comparators reflecting the
    realities of Canadian education
     All   Canadian universities publicly funded
     Education    a provincial (state) jurisdiction
   By 2006, LibQUAL+™ was the primary instrument
    used by Canadian academic libraries to assess library
    service quality
ORIGIN OF THE LIBQUAL+ CANADA CONSORTIUM

   Est. & funded by Canadian Association of Research
    Libraries (CARL) in Jan. 2006
   Goal: create a larger database of Canadian content
    that would offer more meaningful benchmarking of
    services for Canadian academic research libraries
   Unique Opportunity: to engage the broader Canadian
    academic and research library community in
    developing a national service quality assessment
    survey
THE 2007 LIBQUAL+ CANADA CONSORTIUM

   Largest LibQUAL+™ consortium: 46 universities, 7
    community colleges and 3 federal government
    libraries from across Canada
   66% of the libraries had never done the survey
    including some smaller institutions who would not
    have considered participating on their own
   Bilingual Environment: English-language, French-
    language and bilingual institutions.
BUILDING THE LIBQUAL CANADA CONSORTIUM

What factors went into establishing and conducting this
 large and successful consortial project?
   Governance and Support
   Project Organization & Management
   Communication & Engagement
   Active recruitment of participants
GOVERNANCE & SUPPORT
   Governing body: CARL Committee on Effectiveness
    Measures and Statistics
   Funding: annual budgets for 2006 &2007
   Admin. Support: CARL staff
PROJECT ORGANIZATION & MANAGEMENT

   Coordination: Dedicated Project Leader working in
    consultation with participants (official contacts)
   Underlying assumption: most members did not have
    dedicated assessment staff to manage the process
    successfully on their own
   Project management objective: guide consortium
    members through the planning process, via discrete,
    manageable sets of actions; each stage with its own
    timelines and deliverables.
COMMUNICATION & ENGAGEMENT
   Moderated discussion/announcement list
   Members encouraged to contribute in shaping each
    phase of the project
   Timelines and action items were revised at each stage
    based on member input.
   Highest priority: Every query answered in a timely
    fashion &, in most cases, exchange shared with the
    membership
ACTIVE RECRUITMENT OF PARTICIPANTS
   Building critical mass (invitations to join via national
    & regional library councils)
   Individual invitations to encourage maximum
    participation by leading Canadian institutions
   Rapid response to queries from potential participants,
    incl. support documentation to help persuade
    reluctant or wary administrators
WEB PRESENCE
Major recruitment & project management tool
 Goal: to provide an easy to use, one-stop resource for
  member libraries – with material, relevant to
  Canadian libraries, that could be readily adapted by
  individual libraries for their use.
 Updated “look” throughout the project (from early
  focus on attracting participants to final focus on the
  survey results & their analysis
LIBQUAL & BEYOND: A WORKSHOP
                                OTTAWA, ONT. CANADA, OCT. 2007

   Helped consortium participants to analyze their
    LibQUAL+™ results effectively
   1st Canadian library assessment conference
   Provided 1st forum for Canadian librarians
    engaged/interested in assessment to meet &
    network
   Attempted to encourage libraries to start building a
    “culture of assessment”
CONSORTIAL DELIVERABLES
   Standard LibQUAL™ consortial notebook, aggregated
    by user category, library type, and survey language
   The Consortial, on behalf of CARL and regional
    councils in Ontario (OCUL) and Quebec (CREPUQ),
    contracted with ARL for custom consortial notebooks
    representing their member libraries
   The councils all approved the posting of the aggregate
    notebooks on the consortial web site.
DATA SETS
   The Consortium received the complete data set
    representing the results for all 48,000 consortium
    respondents
   Data set and subsets made available to all
    consortium participants in spreadsheet or SPSS
    format (with individual identifiable data, such as the
    institution name, names of campus libraries, local
    discipline groups, etc. replaced with masking codes)
SURVEY OF CONSORTIUM PARTICIPANTS
   93.6% of wanted to take the LibQUAL+™ survey again
    as members of the consortium
   80% preferred LibQUAL+™ over developing a home-
    grown alternative; slight preference among
    respondents for a more abbreviated LibQUAL+ ™ Lite
    over the full survey
   Members split evenly between 2 & 3 year options for
    preferred frequency of future consortial surveys
   Ratings for consortial support and responsiveness
    were very high
CHALLENGES FOR CONSORTIUM & ITS MEMBERS

   Demands on staff time to plan the survey & to review,
    analyze & act on the results – greatest for libraries
    with fewer staff. Limited data analysis expertise.
   Few community college participants in the 2007
    survey & widely differing mandates among the
    Canadian provinces as to clientele served and types
    of academic and non-academic programs. Need for
    more web resources aimed at community colleges
   Limited benchmarking value for federal government
    libraries who each have such widely different clientele
    and mandates
HOW TO IMPROVE THE SURVEY FOR OUR MEMBERS

   Alternative, briefer LibQUAL+™ surveys
   Alternative delivery mechanisms
   Customizable set of user types linkable to a set of
    standard user categories (similar to discipline group
    mapping)
   Customizable labels mapped to the same survey
    concepts for different cultures (e.g. “gender” instead
    of “sex”)
   More effective mapping & management of survey
    questions in different languages
CONCLUSION: IT REALLY IS WORTH DOING!




  Despite the challenges, the 48,000 consortial responses to the 2007
  survey have provided a rich, unique resource of assessment data for
  Canadian academic and research libraries that can only grow more
  valuable each time the consortium runs the survey.
QUESTIONS?



             Thank You!

								
To top