Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

New Ways of Listening to Library Users Workshop LibQUAL

VIEWS: 2 PAGES: 20

									Technological Tools for Assessment
                   Jonathan D. Sousa
       Technical Applications Development Manager
            Association of Research Libraries


      New Ways of Listening to Library Users Workshop
                      Washington, DC
                   November 4 - 5, 2005

                        old.libqual.org
Who I Am
• Jonathan D. Sousa (jonathan@arl.org)
• Technical Applications Development Manager
  – Responsible for the development of online
    applications for ARL’s Statistics and Measurement
    Program
  – LibQUAL+™, StatsQUAL™, DigiQUAL™, ARL
    Statistics Survey (new version), MINES for Libraries
     •   4 years at ARL
     •   8 years in application development
     •   Worked 2 years in an ARL library
     •   Mucking about online since Gopher was new and 14.4 Kbps
         was fast


                            old.libqual.org
What I Want to Cover Today
• The technological history of LibQUAL+™
   – Nicely representative of how technology and assessment in
     libraries have grown together
• Current assessment offerings using technology
   – StatsQUAL™ project, DigiQUAL™, MINES for Libraries
   – Tools to help with other types of analysis
• Future directions (with online demonstration)
• Whatever you want
   – I’m happy to digress


• But one thing you should know:


                            old.libqual.org
old.libqual.org
The LibQUAL+™ Online
Survey Process
• Web-based survey hosted at a single location
  but delivered all over the world
• Each participating institution receives a URL with
  a unique code embedded
• When individuals take the survey, they see a
  version tailored to their institution’s language
  and type and further customized by the survey
  liaison using our online administration tools
• Results are compiled and analyzed by the
  LibQUAL+™ team and made available for
  download at the end of the survey run


                      old.libqual.org
LibQUAL+™ – The Technological
Beginnings
• Texas A&M – Pre-2000:
  – Simple piece of rough ColdFusion code and a flat-file
    database
  – Experimental
  – Few thoughts of scaling the project
  – Innovative but localized




                        old.libqual.org
Pilot Program with ARL – 2000-2001:

• Some new techniques were attempted
• Numerous obstacles were encountered in moving
  beyond one institution
• Production of the results was a laborious process,
  particularly in 2001
• An attempt at gathering data on library users across
  institutions with a new protocol using a relatively new
  medium
• The full power of LibQUAL+™ was starting to appear in
  its standardized core: cross-institutional analysis


                         old.libqual.org
LibQUAL+™ – A Shift in
Development – 2002
• The project development moved to ARL
• Code base was entirely rewritten
• Database was shifted to a scalable relational
  structure
• An interface for institution liaisons was added
• Administrative pages were created




                      old.libqual.org
LibQUAL+™ – Refinement and
Growth – 2003-2005
• The site was rewritten again in 2003
   –   Improve the interface
   –   Simplify the process
   –   Increase customization options
   –   Add additional languages and institution types
• Automation of results production
• What the public sees is now the tip of the iceberg: nearly
  all administration of the project takes place through
  restricted pages on the website
• Survey instrument refined: the core is now 22 questions,
  3 dimensions

                               old.libqual.org
Making LibQUAL+™ Mundane
• Faced with many people now coming to LibQUAL+™
  without first being familiar with ARL
• Different types of institutions coming onboard, many with
  very limited staff and few hours to devote to assessment
• How LibQUAL+™ has responded:
   – Simplified all steps as much as we are able, while still allowing
     flexibility
   – Accept credit cards and consortial payment arrangements
   – Direct download of results instead of mailing them around the
     world: faster and it keeps the price of the survey low
   – Excel format of results for those without access to SPSS
   – Available all year long—2 sessions to accommodate the
     Southern Hemisphere
   – Moved to professional hosting facility for assured uptime


                              old.libqual.org
LibQUAL™ – Rapid Growth

                350                                                                                                 160,000


                                                                                                    145,470
                                                                              308                                   140,000
                300

                                                                  128,958

                                                                                                                    120,000
                                                                                                              255
                250                                                                 113,480


                                                                                                                    100,000

                200                                                                           204

 Number of                                                                                                                    Number of
                                                 78,863                                                             80,000
 Institutions                                             164                                                                 Responses
                150

                                                                                                                    60,000


                100
                                                                                                                    40,000

                                                                                                                                          Number of Institutions
                50                                                                                                                        Number of Responses
                                   20,416   43                                                                      20,000


                      4,407   13
                 0                                                                                                  0
                          2000          2001          2002                 2003           2004            2005

                                                             Survey Year




                                                                     old.libqual.org
Rapid Growth in System
Complexity
• 9 Languages                      • 14 Types of Institutions
                                           – Academic Health Sciences
  –   Afrikaans                            – Academic Law
                                           – Academic Military
  –   American English                     – College or University
  –   British English                      – Community College
                                           – Electronic
  –   Dutch                                – European Business
                                           – Family History
  –   Dutch English                        – Hospital
  –   French (Canada)                      – Natural Resources Library
                                           – Public
  –   French (Europe)                      – Research Centers (FFRDC)
                                             Libraries
  –   Swedish                              – State
  –   Swedish English                      – University/TAFE


                         old.libqual.org
LibQUAL+™ as a Mirror of
Assessment in Libraries
• Started in isolation
• Expanded but suffered growing pains
• Grant funded or recipient of a one-time allocation of
  funds
• Began to prove its worth
• Became part of library’s and institution’s decision making
  processes
• Shifted from a single event to an ongoing process (a
  culture of assessment)
• Now frequently treated as a necessary part of the
  library’s mission as demonstrated by employees and
  committees dedicated to assessment

                          old.libqual.org
How LibQUAL+™ Technology
Concerns Have Changed
• 2 years ago the things I talked about:
    – Network issues
        • Either things are getting better or we are all more familiar with how the
          Internet works… and breaks
    – Caching
        • Software on the networks and on computers is finally doing its job without
          getting in the way
        • Less concern about bandwidth and network speed and fewer people looking
          to caching as a solution
    – Getting accurate email lists
        • Many more institutions now see keeping an up-to-date email list to be critical
          to their functioning
        • Now we need to worry about keeping email out of aggressive spam filters
    – Browser compatibility
        • This one hasn’t changed: there are still holdouts who continue to use
          browsers from before 1997




                                     old.libqual.org
Online Tools for Assessment
• DigiQUAL™
   – Short survey for digital libraries using the 2-column SERVQUAL
     format
   – Math Forum, MERLOT, Utopia, etc.
• MINES for Libraries
   – Survey on the purpose of use of electronic resources
   – Run at individual libraries without a centralized server
   – Mandatory for certain time periods
• ARL Statistics Survey
   – A long-running survey of collections, expenditures, staffing, and
     service activities at ARL institutions
   – The collection tool was completely rewritten this year while the
     questions remained the same


                               old.libqual.org
Other Technological Tools
• Usability studies
  – Camtasia
     • http://www.techsmith.com/
  – Captivate
     • http://www.macromedia.com/
• Web log analysis
  – Urchin
     • http://www.urchin.com/
  – WebTrends
     • http://www.webtrends.com/


                           old.libqual.org
Future Projects
• StatsQUAL™
  – Unify the various assessment tools under one
    system with a single login
  – Share datasets between surveys
• Interactive Statistics
  – Beta version demonstration




                     old.libqual.org
What We’re Focusing On
•   Data integration
•   Dynamic analysis
•   Web standards
•   New assessment solutions




                   old.libqual.org
Online Resources
• StatsQUAL™ Website:
       http://www.statsqual.org

• LibQUAL+™ Website:
       http://www.libqual.org

• Publications:
       http://www.libqual.org/publications

• Events and Training:
       http://www.libqual.org/events

• LibQUAL+™ Bibliography:
       http://www.coe.tamu.edu/~bthompson/servqbib

• LibQUAL+™ Procedures Manual:
       http://www.libqual.org/Manual/index.cfm


                                    old.libqual.org
LibQUAL+™ Contacts
• Martha Kyrillidou
   – Director, Statistics & Measurement Program
   – martha@arl.org

• Amy Hoseth
   – LibQUAL+™ Communications Coordinator
   – amyh@arl.org

• Richard Groves
   – Research Assistant - Statistics & Measurement
   – richard@arl.org

• Jonathan D. Sousa
   – Technical Applications Development Manager
   – jonathan@arl.org




                                  old.libqual.org

								
To top