Evidence-based librarianship - a measured approach to library success

Library Connect Bangkok, Thailand. 1 August 2007 Kuala Lumpur, Malaysia. 2 August 2007 Evidence-Based Librarianship – a measured approach to library success Lisa Cotter The University of Newcastle Gosford Hospital Library Andrew Spencer Macquarie University Library This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License Outline Evidence-based Librarianship (EBL) Library Assessment (LA) Relationship between EBL and LA Examples 2 • • • • • Evidence-based librarianship (EBL) Library assessment (LA) & metrics Relationship between EBL and LA Examples from practice How you can get started Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Evidence-Based Library and Information Practice (EBLIP) Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples 3 “EBLIP seeks to improve library and information services and practice by bringing together the best available evidence and insights derived from working experience, moderated by user needs and preferences. … it thus attempts to integrate user-reported, practitioner-observed and research-derived evidence as an explicit basis for decision-making” (Booth 2006a) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Evidence-Based Practice Process ASK a question ACQUIRE the evidence 4 ASSESS impact and performance (& report findings) APPLY the evidence APPRAISE the evidence Ask an Answerable Question Evidence-based librarianship a question Acquire 5 ASK Ask Appraise Apply Assess Library Assessment Relationship between EBL and LA Examples S P I C E setting perspective intervention comparison University library staff & students Instant messaging Email, virtual reference evaluation Level of use; client satisfaction; impact on roster “In a university library, is instant messaging, email or virtual reference software most effective in providing an online reference service to staff and students of the university in terms of level of use, customer satisfaction, and impact on library staff rosters” Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Find the Evidence Evidence-based librarianship Ask 6 Evidence-based librarianship: ACQUIRE Acquire the evidence Appraise Apply Assess Library Assessment Relationship between EBL and LA Examples “attempts to integrate user-reported, practitioner-observed and research-derived evidence as an explicit basis for decision-making.” (Booth 2006a) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Appraise the Evidence Evidence-based librarianship Ask Acquire 7 APPRAISE Appraise the evidence Apply Assess Library Assessment Relationship between EBL and LA Examples “An even more daunting question is how to critically read the existing research to determine whether it is valuable in a specific situation. Not all research is good, and how do we sort out the good from the bad?” (Koufogiannakis 2006) – reliability – applicability – validity Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Apply the Evidence Evidence-based librarianship Ask Acquire 8 Appraise the evidence Assess Library Assessment Relationship between EBL and LA Examples APPLY Apply "When using research to help with a question, look for high quality studies, but do not be too quick to dismiss everything as irrelevant. Try to take what does apply from the research and use it to resolve the problem at hand." (Koufogiannakis and Crumley 2004 ) – – – – directly conditionally by derivation for enlightenment Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Assess the Outcome and Process Evidence-based librarianship Ask Acquire 9 Appraise Apply Assess impact & performance Library Assessment Relationship between EBL and LA Examples ASSESS • Evaluate the impact of your intervention • Evaluate your performance as an evidence-based practitioner • Reflect and evaluate continuously • Consider evaluation/assessment methods at the start of the EBL process Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Is EBL anything new ? Evidence-based librarianship Ask Acquire 10 The great debate: be it resolved that Evidence Based Librarianship is a bunch of hooey (Dennis et al 2007) Appraise Apply Assess impact & performance Library Assessment Relationship between EBL and LA Examples ASSESS “What EBL does is help you think each step through. If a step is missed, some decisions could be made that wastes time in the long run. By following and understanding these steps, librarians can be assured that they are using the best practices to find a high quality solution. This is not “hooey” but a tool that can help us be better librarians.” Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Library Assessment Evidence-based librarianship 11 Library Assessment Relationship between EBL and LA Examples “A critical tool for understanding library customers and offering services, spaces, collections, and tools that best meet their needs. Without good assessment, libraries could lose touch with users’ desires and needs and even become irrelevant.” (Nardini, 2001) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Culture of Assessment Evidence-based librarianship 12 Library Assessment Relationship between EBL and LA Examples “…an environment where decisions are based on facts, research and analysis, and services are planned and delivered in order to maximise positive outcomes and impacts for the library clients” (Lakos, 1998) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. EBL and Library Assessment Evidence-based librarianship Library Assessment 13 • Evidence-Based Librarianship – Demonstrate effective and efficient library practice – Visionary… what could we be doing? EBL and LA Relationship Examples • Library Assessment – Justify existence and maintenance of library services – Current activity… what are we doing? (Booth 2006b) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Examples from Practice Evidence-based librarianship Library Assessment Relationship between EBL and LA 14 Examples • • • • • Strategic planning Collection development Usability testing Demonstrating value Information literacy programs Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Strategic Planning Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples 15 • Client Satisfaction Survey (Rodski) – Gap between what clients find important and how well they think we’re performing – What needs to change Strategic Planning Collection Development Usability Testing Demonstrating Value Information Literacy • EBL model – Solutions to help narrow the gap – How to change Using EBL practices in strategic planning (Mitchell 2007) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Collection Development Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning 16 Establishing a model for evidence based collection management (Koufogiannakis 2007) Collection Development Usability Testing Demonstrating Value Information Literacy • professional judgement and faculty feedback traditionally inform collection decisions • University of Alberta model seeks to be evidence and assessment based • Core and Innovative information needs Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Collection Development Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning 17 Collection Development Usability Testing Demonstrating Value Information Literacy (Koufogiannakis 2007) Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Usability Testing Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning Collection Development 18 Adding SPICE to our library intranet site: a recipe to enhance usability (Cotter et al 2005, 2006) • Use of research-derived, client-reported, and librarian observed evidence – Research Based Web Design & Usability Guidelines (Koyani et al 2003) – Raward Library Usability Analysis Tool (Raward 2002) – Observation sessions – Interviews – Other libraries’ case studies Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Usability Testing Demonstrating Value Information Literacy Demonstrating Value Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning Collection Development Usability Testing 19 Assessment, evaluation, and value: current trends (Dudden 2007) • Measure all aspects of management – Needs, inputs, quality processes, outputs, quality processes, outputs, quality service, outcomes and impacts – Use benchmarking, needs assessment, quality improvement, outcome measurements and library standards Demonstrating Service Evaluation Value Information Literacy Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Demonstrating Value Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning Collection Development Usability Testing (Dudden 2007) 20 • Goal of assessment is to improve economy, efficiency and effectiveness – If done systematically, results of local assessments, combined with published studies, can show the value of library services • To communicate value, librarians must – Understand goals and values of the organisation and the library – Understand the user – Establish a culture of assessment – Use a variety of measures – Use outcomes to tell your story Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Demonstrating Service Evaluation Value Information Literacy Information Literacy Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning Collection Development Usability Testing Demonstrating Value 21 Information skills training in health libraries: are we any nearer the evidence (Brettle 2007) • • • • does training improve searching skills? what are effective methods of training? does training effect patient care? what is the availability/use of objective and validated outcome measures? Information Literacy Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Information Literacy – the evidence (Brettle 2007) Evidence-based librarianship Library Assessment Relationship between EBL and LA Examples Strategic Planning Collection Development Usability Testing Demonstrating Value 22 • limited evidence that training improves search skills • no evidence about: – what is the most effective type of training – impact of training on patient care • objective and validated measures of evaluating training are essential to determine if training is effective HOWEVER: • More recent research is vibrant and shows some effectiveness in terms of improvement in skills, knowledge and confidence post training Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Information Literacy So, is EBL a bunch of hooey ? Evidence-based Librarianship (EBL) Library Assessment (LA) Relationship between EBL and LA Examples 23 You decide! What does the evidence reveal? Give it a go  Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Tools and Resources Evidence-based librarianship Library Assessment Relationship between EBL and LA 24 TOOLKIT (Cotter & Lewis 2006) Libraries Using Evidence - eblip.net.au http://www.eblip.net.au TEXT BOOKS Booth, A., & Brice, A. (Eds.). (2004). Evidence-based practice for information professionals : a handbook. London: Facet. Connor, E. (Ed.). (2007). Evidence-based librarianship: case studies and active learning exercises. Oxford: Chandos. KEY ARTICLES (about EBL and Assessment) Booth, A. (2006). Counting what counts: performance measurement and evidence-based practice. Performance Measurement and Metrics, 7(2), 63-74. Ryan, P. (2006). EBL and library assessment: two solitudes? Evidence Based Library and Information Practice, 1(4), 77-80. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/136/17 Examples Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. Tools and Resources Evidence-based librarianship Library Assessment Relationship between EBL and LA 25 JOURNAL (open access) Evidence Based Library and Information Practice http://ejournals.library.ualberta.ca/index.php/EBLIP/issue/archive Examples EVIDENCE SUMMARIES Grouped by domain, from the EBLIP journal http://toolkit.eblip.net.au/evidencesummaries.html DATABASE (open access) Library, Information Science & Technology Abstracts (LISTA) http://www.libraryresearch.com/ CONFERENCES 3rd International EBL Conference http://conferences.alia.org.au/ebl2005/ 4th International EBLIP Conference http://www.eblip4.unc.edu/ Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. References Booth, A. (2006a). Counting what counts: performance measurement and evidence-based practice. Performance Measurement and Metrics, 7(2), 63-74. Booth, A. (2006b). Comment on "EBL and Library Assessment". libraryassessment.info. http://libraryassessment.info/?p=71 Brettle, A. (2007, 6-11 May). Information skills training in health libraries: are we any nearer the evidence? Paper presented at the 4th International Evidence Based Library and Information Practice Conference, Durham, North Carolina, US. http://www.eblip4.unc.edu/papers/Brettle.pdf Cotter, L., Harije, L., Lewis, S., & Tonnison, I. (2005, 16-19 October). Adding SPICE to our library intranet site: a recipe to enhance usability. Paper presented at the 3rd International Evidence Based Librarianship Conference, Brisbane, Australia. http://conferences.alia.org.au/ebl2005/CotterTonnisonLewisHarije_pps.pdf Cotter, L., Harije, L., Lewis, S., & Tonnison, I. (2006). Adding SPICE to a library intranet site: a recipe to enhance usability. Evidence Based Library and Information Practice, 1(1), 3-25. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/11/0 26 Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. References Cotter, L., & Lewis, S. (2006). Libraries Using Evidence - eblip.net.au. Evidence Based Library and Information Practice, 1(3), 98-100. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/102/154 Dennis, C., Duda, D., McGowan, J., & White, L. (2007). The great debate: be it resolved that Evidence Based Librarianship is a bunch of hooey. Evidence Based Library and Information Practice, 2(2), 116-119. Dudden, R. (2007, 6-11 May). Assessment, evaluation, and value: current trends. Paper presented at the 4th International Evidence Based Library and Information Practice Conference, Durham, North Carolina, US. Koufogiannakis, D. (2005). Small steps forward through critical appraisal. Evidence Based Library and Information Practice, 1(1), 81-82. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/26/64 Koufogiannakis, D. (2007, 6-11 May). Establishing a model for evidence based collection management. Paper presented at the 4th International Evidence Based Library and Information Practice Conference, Durham, North Carolina, US. http://www.eblip4.unc.edu/papers/Koufogiannakis.pdf 27 Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. References 28 Koufogiannakis, D., & Crumley, E. (2004). Applying evidence to your everyday practice. In A. Booth & A. Brice (Eds.), Evidence-Based Practice for Information Professionals (pp. 119126). London: Facet. Koyani, S. et al. (2003). Research-based web design & usability guidelines. Washington, D.C., United States Department of Health and Human Services. libraryassessment.info [community discussion blog] http://libraryassessment.info Lakos, A. (1998, 22 April). Building a culture of assessment in academic libraries - obstacles and possibilities. Paper presented at the Living the Future II Conference, Tucson, Arizona. http://personal.anderson.ucla.edu/amos.lakos/Present/Arizona98/Ariztext98.html Lewis, S., & Cotter, L. (2007). Have the most relevant and answerable research questions facing librarians changed between 2001 and 2006? Evidence Based Library and Information Practice, 2(1), 107-120. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/98/243 Mitchell, E. (2007, 6-11 May). Using EBL practices in strategic planning. Poster presented at the 4th International Evidence Based Library and Information Practice Conference, Durham, North Carolina, US. Nardini, H. G. (2001). Building a culture of assessment. ARL Bimonthly Report, 218, 11. http://www.arl.org/bm~doc/assess.pdf Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. References 29 Raward, R. A. (2002). A study of best practice design guidelines and the development of a usability analysis tool for the evaluation of Australian academic library web sites. Unpublished thesis (M.A.), University of Canberra, Canberra. Ryan, P. (2006, 14-17 June). Good libraries use evidence: cultivating a culture of assessment and evidence based decision making in academic libraries. Paper presented at the Canadian Library Association 61st Annual Conference, Ottawa, Ontario. http://www.cla.ca/resources/cla2006/Presentations/CLA_EBL.pdf Ryan, P. (2006). EBL and library assessment: two solitudes? Evidence Based Library and Information Practice, 1(4), 77-80. http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/136/177 Elsevier Library Connect Seminars. Bangkok & Kuala Lumpur. August 2007. DISCUSSION Evidence-Based Librarianship - a measured approach to library success Elsevier Library Connect Seminars Bangkok, Thailand. 1 August 2007 Kuala Lumpur, Malaysia. 2 August 2007 Lisa Cotter Lisa.Cotter@newcastle.edu.au School Librarian, Nursing and Midwifery The University of Newcastle Gosford Hospital Library Andrew Spencer Andrew.Spencer@library.mq.edu.au Liaison Librarian Macquarie University Library CRICOS Provider 00109J | www.newcastle.edu.au This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License