Learning Center
Plans & pricing Sign in
Sign Out

New Ways of Listening to Library Users Workshop LibQUAL


									Technological Tools for Assessment
                   Jonathan D. Sousa
       Technical Applications Development Manager
            Association of Research Libraries

      New Ways of Listening to Library Users Workshop
                      Washington, DC
                   November 4 - 5, 2005

Who I Am
• Jonathan D. Sousa (
• Technical Applications Development Manager
  – Responsible for the development of online
    applications for ARL’s Statistics and Measurement
  – LibQUAL+™, StatsQUAL™, DigiQUAL™, ARL
    Statistics Survey (new version), MINES for Libraries
     •   4 years at ARL
     •   8 years in application development
     •   Worked 2 years in an ARL library
     •   Mucking about online since Gopher was new and 14.4 Kbps
         was fast

What I Want to Cover Today
• The technological history of LibQUAL+™
   – Nicely representative of how technology and assessment in
     libraries have grown together
• Current assessment offerings using technology
   – StatsQUAL™ project, DigiQUAL™, MINES for Libraries
   – Tools to help with other types of analysis
• Future directions (with online demonstration)
• Whatever you want
   – I’m happy to digress

• But one thing you should know:

The LibQUAL+™ Online
Survey Process
• Web-based survey hosted at a single location
  but delivered all over the world
• Each participating institution receives a URL with
  a unique code embedded
• When individuals take the survey, they see a
  version tailored to their institution’s language
  and type and further customized by the survey
  liaison using our online administration tools
• Results are compiled and analyzed by the
  LibQUAL+™ team and made available for
  download at the end of the survey run

LibQUAL+™ – The Technological
• Texas A&M – Pre-2000:
  – Simple piece of rough ColdFusion code and a flat-file
  – Experimental
  – Few thoughts of scaling the project
  – Innovative but localized

Pilot Program with ARL – 2000-2001:

• Some new techniques were attempted
• Numerous obstacles were encountered in moving
  beyond one institution
• Production of the results was a laborious process,
  particularly in 2001
• An attempt at gathering data on library users across
  institutions with a new protocol using a relatively new
• The full power of LibQUAL+™ was starting to appear in
  its standardized core: cross-institutional analysis

LibQUAL+™ – A Shift in
Development – 2002
• The project development moved to ARL
• Code base was entirely rewritten
• Database was shifted to a scalable relational
• An interface for institution liaisons was added
• Administrative pages were created

LibQUAL+™ – Refinement and
Growth – 2003-2005
• The site was rewritten again in 2003
   –   Improve the interface
   –   Simplify the process
   –   Increase customization options
   –   Add additional languages and institution types
• Automation of results production
• What the public sees is now the tip of the iceberg: nearly
  all administration of the project takes place through
  restricted pages on the website
• Survey instrument refined: the core is now 22 questions,
  3 dimensions

Making LibQUAL+™ Mundane
• Faced with many people now coming to LibQUAL+™
  without first being familiar with ARL
• Different types of institutions coming onboard, many with
  very limited staff and few hours to devote to assessment
• How LibQUAL+™ has responded:
   – Simplified all steps as much as we are able, while still allowing
   – Accept credit cards and consortial payment arrangements
   – Direct download of results instead of mailing them around the
     world: faster and it keeps the price of the survey low
   – Excel format of results for those without access to SPSS
   – Available all year long—2 sessions to accommodate the
     Southern Hemisphere
   – Moved to professional hosting facility for assured uptime

LibQUAL™ – Rapid Growth

                350                                                                                                 160,000

                                                                              308                                   140,000


                250                                                                 113,480


                200                                                                           204

 Number of                                                                                                                    Number of
                                                 78,863                                                             80,000
 Institutions                                             164                                                                 Responses



                                                                                                                                          Number of Institutions
                50                                                                                                                        Number of Responses
                                   20,416   43                                                                      20,000

                      4,407   13
                 0                                                                                                  0
                          2000          2001          2002                 2003           2004            2005

                                                             Survey Year

Rapid Growth in System
• 9 Languages                      • 14 Types of Institutions
                                           – Academic Health Sciences
  –   Afrikaans                            – Academic Law
                                           – Academic Military
  –   American English                     – College or University
  –   British English                      – Community College
                                           – Electronic
  –   Dutch                                – European Business
                                           – Family History
  –   Dutch English                        – Hospital
  –   French (Canada)                      – Natural Resources Library
                                           – Public
  –   French (Europe)                      – Research Centers (FFRDC)
  –   Swedish                              – State
  –   Swedish English                      – University/TAFE

LibQUAL+™ as a Mirror of
Assessment in Libraries
• Started in isolation
• Expanded but suffered growing pains
• Grant funded or recipient of a one-time allocation of
• Began to prove its worth
• Became part of library’s and institution’s decision making
• Shifted from a single event to an ongoing process (a
  culture of assessment)
• Now frequently treated as a necessary part of the
  library’s mission as demonstrated by employees and
  committees dedicated to assessment

How LibQUAL+™ Technology
Concerns Have Changed
• 2 years ago the things I talked about:
    – Network issues
        • Either things are getting better or we are all more familiar with how the
          Internet works… and breaks
    – Caching
        • Software on the networks and on computers is finally doing its job without
          getting in the way
        • Less concern about bandwidth and network speed and fewer people looking
          to caching as a solution
    – Getting accurate email lists
        • Many more institutions now see keeping an up-to-date email list to be critical
          to their functioning
        • Now we need to worry about keeping email out of aggressive spam filters
    – Browser compatibility
        • This one hasn’t changed: there are still holdouts who continue to use
          browsers from before 1997

Online Tools for Assessment
• DigiQUAL™
   – Short survey for digital libraries using the 2-column SERVQUAL
   – Math Forum, MERLOT, Utopia, etc.
• MINES for Libraries
   – Survey on the purpose of use of electronic resources
   – Run at individual libraries without a centralized server
   – Mandatory for certain time periods
• ARL Statistics Survey
   – A long-running survey of collections, expenditures, staffing, and
     service activities at ARL institutions
   – The collection tool was completely rewritten this year while the
     questions remained the same

Other Technological Tools
• Usability studies
  – Camtasia
  – Captivate
• Web log analysis
  – Urchin
  – WebTrends

Future Projects
• StatsQUAL™
  – Unify the various assessment tools under one
    system with a single login
  – Share datasets between surveys
• Interactive Statistics
  – Beta version demonstration

What We’re Focusing On
•   Data integration
•   Dynamic analysis
•   Web standards
•   New assessment solutions

Online Resources
• StatsQUAL™ Website:

• LibQUAL+™ Website:

• Publications:

• Events and Training:

• LibQUAL+™ Bibliography:

• LibQUAL+™ Procedures Manual:

LibQUAL+™ Contacts
• Martha Kyrillidou
   – Director, Statistics & Measurement Program

• Amy Hoseth
   – LibQUAL+™ Communications Coordinator

• Richard Groves
   – Research Assistant - Statistics & Measurement

• Jonathan D. Sousa
   – Technical Applications Development Manager


To top