RDS Acceptance Test Plan

ZLOT Project Phase 2 System Implementation of the LOT Acceptance Test Plan for the Resource Discovery Service Software This report is submitted in partial fulfillment of contract # 306-03-8322. William E. Moen, Ph.D. Kathleen R. Murray, Ph.D. Irene Lopatovska, M.I.S Texas Center for Digital Knowledge University of North Texas Denton, Texas September 3, 2003 The following research assistants contributed to this report: Barbara Schultz-Jones & Scott J. Simon William E. Moen, Ph.D. Principal Investigator Texas Center for Digital Knowledge University of North Texas ACCEPTANCE TEST PLAN Contents 1. 2. 3. INTRODUCTION ................................................................................................................................... 4 METHODOLOGY................................................................................................................................... 4 ACCEPTANCE TEST FORM ................................................................................................................ 6 APPENDIX A: Z39.50 SEARCH QUERIES ................................................................................................ 32 APPENDIX B: TEXSHARE SEARCH QUERIES ........................................................................................ 35 APPENDIX C: MARC RECORD ................................................................................................................. 39 ZLOT Project Page 2 September 3, 2003 ACCEPTANCE TEST PLAN Figures and Tables Table 1. RDS Technical Requirements by Priority Level .............................................................................. 4 Figure 1. Z39.50 Distributed Search Model .................................................................................................. 5 ZLOT Project Page 3 September 3, 2003 ACCEPTANCE TEST PLAN 1. Introduction In June 2002, the ZLOT project issued a report that identified the technical requirements for the Library of Texas (LOT) Resource Discovery Service (RDS). The report entitled Project Deliverable C: Functional 1 Requirements for the Library of Texas detailed and prioritized a number of features and requirements that a broad range of Texas librarians, state library agency staff, and others anticipate from an actual implementation of the LOT Resource Discovery Service. These functional requirements were later translated into technical requirements and included in the Library of Texas Resource Discovery Service Software Request for Proposal (RFP 306-03-8100). Based on the technical requirements detailed in the RFP, this acceptance test plan has been developed to engage in a systematic acceptance test of the Index Data Resource Discovery Service (RDS) application software. The acceptance test process incorporates four distinct phases (CSTE, 2003): 1. Define the acceptance test criteria 2. Develop an acceptance test plan 3. Execute the acceptance test plan 4. Develop an acceptance decision based on the results of acceptance testing The methodology for the acceptance testing is based in part on the Proof of Concept methodology developed by the ZLOT Project in consultation with Dr. John Bertot in phase I of the ZLOT Project and on 2 the Technical Evaluation Plan for RFP Responses developed in phase II by the ZLOT Project. The methodology specified is in the tradition of functional or black box testing (Software Testing Answers FAQs, 2003) and is not based on any knowledge of internal design or code. It is based primarily on technical requirements and functionality. Prior to the acceptance test, the plan will be pre-tested. Once the acceptance test has been completed, an analysis and report of the results will be developed. 2. Methodology Overall, there are 65 technical requirements for the RDS software that will be tested. Table 1 identifies the number of requirements by priority level and interface (Search and Retrieval Interface (SRI), Search Interface (SI), Retrieval Interface (RI)). For example there are 12 priority 1 technical requirements common to both the search and retrieval interfaces and an additional 13 priority 1 requirements specific to the search interface. Priority Level 1 - Must Have Technical Requirements TRS-SRI-A TRS-SI-A TRS-RI-A TRS-SRI-B TRS-SI-B TRS-RI-B TRS-SRI-C Total: Table 1. RDS Technical Requirements by Priority Level Testing of the RDS software will be in the environment represented by Figure 1, which depicts a Z39.50 distributed search model for library catalogs. In this model library patrons enter search requests from a 1 2 Number of Requirements 12 13 7 5 20 7 1 65 2 - Should Have 3 - Desirable http://www.unt.edu/zlot/ProjectDocs.htm http://www.unt.edu/zlot/project_docs/index.htm ZLOT Project Page 4 September 3, 2003 ACCEPTANCE TEST PLAN remote location [1] and the RDS software relays the search to multiple catalogs (or other LOT resources such as TexShare databases) [2]. Results from searches of the multiple catalogs are presented to the user in an easy-to-understand manner [3]. The TSLAC will host the web-accessible RDS software, which includes a Z39.50 client that will communicate directly with the Z-servers located in Texas libraries. From a web browser, RDS software evaluators will access the LOT RDS and issue appropriate search requests to evaluate the each of the technical requirements. Figure 1. Z39.50 Distributed Search Model Three ZLOT Project evaluators at the Texas Center for Digital Knowledge laboratory will conduct the acceptance test. In preparation for the testing, evaluators will read vendor documentation and pre-test the test plan. The Acceptance Test Form included in Section 3 of this document provides specific directions to evaluators and will be used to document the test results. The form includes a set of recommended search terms to standardize the search queries. When testing Bath-compliant searches, appropriate search terms are specified in Appendices A and B. ZLOT Project Page 5 September 3, 2003 ACCEPTANCE TEST PLAN 3. Acceptance Test Form Evaluators: ____________________________________ ____________________________________ ____________________________________ Test Start Date/Time: Test Completion Time: Computer Type: Operating System: Browser and Version: Resolution: Connection Type: General Directions 1. Log on to RDS search and retrieval interface and fill out initial required information in the acceptance test form. 2. Answer questions related to technical requirements in order listed in the acceptance test form. 3. When necessary, perform search queries using the general search terms provided at the beginning of the acceptance test form, and analyze results in order to answer the question(s). 4. If „somewhat‟ is chosen as the answer to a specific technical requirement, comments are required. 5. In certain cases, notes are provided when deviations from the general format are necessary. These include Z39.50, TexShare, Broadcast searching, and Texas Library Directory Database (TLDD) related technical requirements. Specific search terms are provided in the appendix for many of these technical requirements; the notes indicate when these search terms should be performed. General Search Terms Search Type Subject Author Keyword Title Search String 1 Lymes Disease Mark Twain/Samuel Clemmons Mosquito Dr. Spock‟s Baby and Child Care Search String 2 Mexican-American War; U.S. Mexican War Charles R. McClure Library information networks To Kill a Mockingbird __________________/________________ __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ ZLOT Project Page 6 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-SRI-A1.1 Technical Requirement Description and Test Directions Directions: Verify if the interface presents uniform icons, colors, function buttons, and other methods to create a common look and feel when the user is selecting resources to search, conducting searches on one or several resources, viewing brief or full records returned, and seeing the actual information object when it is available electronically. The user interface must provide a common look and feel for all functions available through the interface. Requirement Met? Yes No Somewhat Comments TRS-SRI-A1.2 DIRECTIONS: CONTACT TSLAC AND VERIFY IF USABILITY TESTING DOCUMENTATION WAS PROVIDED BY VENDOR. Usability testing must be documented to demonstrate a consistent user experience is available from the user interface. Yes No Somewhat TRS-SRI-A2 Directions: TSLAC should acquire login to the administration interface from the vendor and provide it to ZLOT evaluators for testing this technical requirement.3 The search and retrieval interface must allow easy, effective, and efficient administration and configuration to: Directions: Login to the administrative interface and verify if the following functions are available. 1) Add and delete target resources 2) Add and delete descriptions of resources Directions: Use non-RDS target catalog to test this requirement. 3) Place resources within existing categories, such as “subject” without affecting the common look and feel of the user interface. Yes No Somewhat Yes No Somewhat Yes No Somewhat TRS-SRI-A3.1 Directions: Verify if the interface provides a link to the online catalog‟s web interface and if so, how it does it (e.g. opens a new window, etc). Yes No Somewhat 3 Requirements pertaining to testing the administrative interface are highlighted in yellow. ZLOT Project Page 7 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions The search and retrieval interface must provide an effective mechanism for the user to select and use the native interface of a target online catalog system. The search and retrieval interface must provide a link to the catalog‟s web interface. Requirement Met? Comments TRS-SRI-A3.2 Directions: Verify if the interface provides links to the non-catalog target resources. The search and retrieval interface must provide an effective mechanism for the user to select and use the native interface of non-catalog target resources such as TexShare databases and the TRAIL system. The search and retrieval interface must provide a link to the non-catalog resource‟s web interface. Directions: Dial up to a non-UNT ISP, login with username and password, select native interface, and discover if password authorization carries over to the native interface without requesting the user to login a second time. 2) Access control and authorization in effect through the search and retrieval interface must be carried through when the user chooses the native interface. Yes No Somewhat Yes No Somewhat TRS-SRI-A4 Directions: Dial up to a non-UNT ISP to test username and password authentication. Access the interface through the UNT ISP to test IP authentication. The search and retrieval interface application must support an access control and authorization mechanism to provide users with access to restricted target resources (e.g., licensed databases). The application should be able to recognize the IP of the user‟s computer and implement a mechanism to accept usernames and passwords as assigned by a library organization. Yes No Somewhat TRS-SRI-A5 Directions: This requirement will not be tested by the ZLOT. The search and retrieval interface application must present customized views of distribute target resources dependent on attributes of the user, as determined at the time of logging into the application with a username and password. The application will use information provided by a local library that wants to present its patrons with access to locally owned, restricted access resources, along with the resources available through the LOT. Yes No Somewhat TRS-SRI-A6 Directions: Dial up to a non-UNT ISP and use username and password to login to the interface. Go to a different RDS target interface and verify it prompts for additional authentication. 1) The search and retrieval interface application must use the initial user login information for access control management and authorization throughout the user‟s engagement with the LOT resource discovery service. Yes No Somewhat ZLOT Project Page 8 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions Directions: Login using username and password, select a restricted native interface, and verify if authorization information is retained in native interface. 2) Authorization information must be retained if the user chooses to move to a restricted target resource‟s native interface. Directions: Login using username and password, wait 15 minutes without interacting with the RDS, and verify if reauthorization is required. Evaluators might also check if after 15 minutes of not using the interface the Book bags are cleared, the warning window pops up to indicate timeout, etc. 3) A timeout after a period of inactivity by the user with the search and retrieval interface application (e.g., 15 minutes) will require the user to log in again if the user wants to access target resources that require authorization. Directions: Dial up to a non-UNT ISP and use username and password to login to the interface. Verify if the user is prompted to store username and password cookie, check cookie properties, logoff from the interface and login back to check if the cookie is working. 4) Users must have the option to store password information locally in order to bypass login procedures during subsequent sessions. Requirement Met? Yes No Somewhat Comments Yes No Somewhat Yes No Somewhat TRS-SRI-A7 Directions: This requirement is tested in WA6. The search and retrieval interface application must collect user transactions and provide reports of these transactions that can be analyzed using common web log transaction software and/or statistical analysis packages such as SPSS or SAS. Since the application may be used with or without users being authorized (e.g., a user only searches publicly accessible and non-restricted target resources), the application must be able to collect usage data for both categories of users. Specific data must be collected that allows an analysis of the following:  Usage when a user is accessing the application from a known local library IP  Usage when a user is accessing the application outside of a local library but has provided username and password for authorization  Usage when a user is accessing the application but provides no authorization information  Number of searches by type of resource  Number of sessions and visit  Units and records examined by type of resource  The application must use the definitions of these metrics as defined in NISO Z39.7-2002 Information Services and Use. Yes No Somewhat Yes No Somewhat Yes No Somewhat Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat ZLOT Project Page 9 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-SRI-A8.1 Technical Requirement Description and Test Directions Directions: Verify if vendor provided adequate technical documentation for the RDS maintenance personnel (online help through the administrative interface, print out documents, etc). The vendor must provide complete and adequate written documentation covering all aspects of the search and retrieval interface application. Requirement Met? Yes No Somewhat Comments TRS-SRI-A8.2 Directions: Verify if links to online help are available for any and all functionality. The search and retrieval interface application must provide context sensitive online help for all functions available through the application. Yes No Somewhat TRS-SRI-A9 Directions: Verify if vendor provided the written statement of vendor‟s conformance to the user interface standards. The search and retrieval interface application, and specifically the user interface, must conform to state, federal, and other guidelines governing accessibility. 1) The vendor must confirm that the user interface and functionality available in the application meets or exceeds the requirements of the Web Accessibility Initiative guidelines promulgated by the World Wide Web Consortium (W3C) available at: http://www.w3.org/WAI/ 2) In addition, specific Texas requirements must be met, as defined in Standards Review and Recommendation Publication, SRRPUB11, World Wide Web Design Standards and Coding Guidelines Web Accessibility & Usability, available at: http://www.dir.state.tx.us/standards/srrpub11-accessibility.htm Yes No Somewhat Yes No Somewhat TRS-SI-A1.1 Directions: Verify that a simple search box is available and working. The user interface of the search and retrieval interface application must present the user with a simple, general keyword search functionality (taking as a model the web search engines‟ simple text box for typing in search terms). Yes No Somewhat TRS-SI-A1.2 Directions: Perform the following searches against a target catalog that has a Z39.50 interface. Analyze z-server and z-client log files to verify Bath compliance. Search terms for testing this requirement are provided in Appendix A. Yes No Somewhat ZLOT Project Page 10 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0 requirements for searching. Specifically, the client must be able to issue a general keyword search following the specifications in the Z39.50 profiles. Directions: Perform any keyword search against a non-Z39.50 catalog (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific Texas library online catalog systems that do not support a Z39.50 interface but are included as LOT target resources, the search and retrieval interface application must be able to issue this general keyword search to the selected online catalog systems through their web interface. Requirement Met? Comments Yes No Somewhat TRS-SI-A1.3 Directions: Perform any keyword search against a TexShare database that includes a Z39.50 interface. Analyze z-server and z-client log files to verify Bath compliance. Search terms for testing this requirement are provided in Appendix B. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target non-catalog target resources (e.g., the TexShare databases and the TRAIL system) that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0 requirements for searching. Specifically, the client must be able to issue a general keyword search following the specifications in the Z39.50 profiles. Directions: Perform keyword search against a non-Z39.50 database (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific non-catalog target resources that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application must be able to issue this general keyword search to the selected target resources through their web interface. Yes No Somewhat Yes No Somewhat TRS-SI-A2.1 Directions: Verify if the interface allows the following types of searches. The user interface of the search and retrieval interface application must present the user with basic keyword search functionality for the following types of searches:   Author Keyword Title Keyword Yes No Somewhat Yes No Somewhat ZLOT Project Page 11 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement  TRS-SI-A2.2 Technical Requirement Description and Test Directions Subject Keyword Directions: Perform the following searches and analyze log files against BATH profile to check for conformance. Please note that when two terms are used to perform a keyword search, the Bath compliant search should fail. Evaluators should consult log files to see how a 2+ words search was issued (e.g. Boolean AND, OR, or Phrase). Use search terms from Appendix A. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0 requirements for searching.    Author Keyword Title Keyword Subject Keyword Requirement Met? Yes No Somewhat Comments Yes No Somewhat Yes No Somewhat Yes No Somewhat Directions: Perform searches against a non-Z39.50 catalog (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific Texas library online catalog systems that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application must be able to issue these three types of keyword searches to the selected online catalog systems through their web interfaces.    TRS-SI-A2.3 Author Keyword Title Keyword Subject Keyword Yes No Somewhat Yes No Somewhat Yes No Somewhat Directions: Perform the following searches against TexShare databases. Analyze log files to verify Bath compliance. Search terms for testing this requirement are provided in Appendix B. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target non-catalog target resources (e.g., the TexShare databases and the TRAIL system) that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0 requirements for searching.   Author Keyword Title Keyword Yes No Somewhat Yes No Somewhat ZLOT Project Page 12 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement  Technical Requirement Description and Test Directions Subject Keyword Requirement Met? Yes No Somewhat Comments Directions: Perform keyword searches against a TexShare database that does not include a Z39.50 interface. Analyze log files to verify that the RDS is sending plausible search strings. Use search terms from Appendix B. 2) For a limited set of specific non-catalog target resources that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application must be able to issue these four types of keyword searches to the selected target resources through their web interfaces.     TRS-SI-A3.1 Author Keyword Title Keyword Subject Keyword General Keyword Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat Directions: Select one target with the Z39.50 interface and one target without Z39.50 interface. Send concurrent search queries, get results, and analyze log files. For the target with the Z39.50 interface analyze log files to check for BATH profile conformance. For the target without the Z39.50 interface analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix B. The search and retrieval interface application must be able to issue the same query (or semantically equivalent queries) concurrently to two or more online catalog systems (broadcast searching), where the online catalog systems may or may not have a Z39.50 interface to the underlying information retrieval system. Broadcast searching using the types of searches defined in TRS-SI-A1.1 and TRS-SI-A2.1 must be supported.     Author Keyword Title Keyword Subject Keyword General Keyword Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat TRS-SI-A3.2 Directions: Select two catalogs, send a broadcast search and verify that the user is presented with the appearance of a union catalog at the time of search. 1) The search and retrieval interface application must allow the user to select two or more online catalog systems to search concurrently. Yes No Somewhat ZLOT Project Page 13 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions Directions: Select two administratively defined sets of targets, send a broadcast search and verify that the user is presented with the appearance of a union catalog at the time of search. 2) The application must also support administratively defined sets of online catalog systems to search concurrently. Directions: This requirement might not be tested since the initial interface release might not support dynamic sets based on the TLDD data elements, but use predefined geographical areas. If dynamic sets are available select a dynamic set defined by geographical area. Send a broadcast search and verify that the user is presented with the appearance of a union catalog at the time of search. 3) The application must also support dynamic sets of online catalog systems to search concurrently (e.g., dynamically defined based on user attributes or other criteria such a within a geographic area). Directions: Select one TexShare database with the Z39.50 interface and one TexShare database without Z39.50 interface. Send concurrent search queries, get results, and analyze log files. For the target with the Z39.50 interface analyze log files to check for BATH profile conformance. For the target without the Z39.50 interface analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix B. The search and retrieval interface application must be able to issue the same query (or semantically equivalent queries) concurrently to two or more non-catalog target resources consisting of TexShare databases (broadcast searching), where the target resources may or may not have a Z39.50 interface to the underlying information retrieval system. Broadcast searching using the types of searches defined in TRS-SI-A1.1 and TRS-SI-A2.1 must be supported.     Author Keyword Title Keyword Subject Keyword General Keyword Requirement Met? Yes No Somewhat Comments Yes No Somewhat TRS-SI-A4.1 Yes No Somewhat Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat TRS-SI-A4.2 Directions: Verify if two or more TexShare targets could be selected for concurrent searches. 1) The search and retrieval interface application must allow the user to select two or more TexShare databases to search concurrently. Directions: Login to the administrative interface and verify if subject groupings are presented and include multiple TexShare databases. Yes No Somewhat Yes No Somewhat ZLOT Project Page 14 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions 2) The application must also support administratively defined sets of TexShare to search concurrently (e.g., based on subject coverage). Requirement Met? Comments TRS-SI-A5 Directions: Verify if the interface presents buttons, links, drop down boxes or other options for selecting geographical parameters for choosing online catalog systems to search concurrently. 1) The search and retrieval interface application must allow the user to choose a set of online catalog systems to search concurrently based on geographical parameters determined by the user. Directions: Verify if the interface uses dynamic look-up or static definitions to present information about individual libraries. Page properties or other sources should be consulted. 2) The application must be able to communicate with a database that contains geographic or location information about individual libraries. The database would be maintained outside of the application, and this could be the same database discussed in TRS-SRI-B2 and other requirements and specifications. Yes No Somewhat Yes No Somewhat TRS-SI-A6.1 Directions: Login using a username and password that has an associated Z39.50 target in the RDS; verify that the option to search only the home library is available. The search and retrieval interface application must allow the user to easily select his/her home library to search through the LOT resource discovery interface (not the home library‟s native interface). This assumes that the user‟s home library is listed as one of the LOT target resources. Yes No Somewhat TRS-SI-A6.2 Directions: This requirement is tested under TRS-RI-A4. The search and retrieval interface application must allow a user to do a follow-up search for any resources discovered in other library online catalog systems to see if it is held by the user‟s home library. Yes No Somewhat TRS-RI-A1.1 Directions: Verify if the interface presents options to specify to retrieve specific data elements listed in TRS-RI-A1.2 and TRS-RI-A1.3 (below). Verify if the option to switch between brief and full record views is available. The search and retrieval interface application must make available to the user specific data elements returned by the target resource. Multiple views (e.g., brief and full) of the returned records can be provided, but the application must be able to present, by user request, required data elements in a consistent manner when retrieved from the target resource. Yes No Somewhat ZLOT Project Page 15 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-RI-A1.2 Technical Requirement Description and Test Directions Directions: Select and search target online catalog, verify if the following data elements are presented in the RDS. The search and retrieval interface application, when interacting with online catalog systems, must make available the following data elements to present to the user:         Author Title Subject Format of resource Language of resource Date of publication Type of material (i.e., digital) Standard identifier Requirement Met? Comments Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat TRS-RI-A1.3 Directions: Select and search non-catalog target, verify if the following data elements are presented in the RDS. The search and retrieval interface application, when interacting with non-catalog target resources, must make available the following data elements to present to the user:         Author or semantically comparable data Title or semantically comparable data Subject or semantically comparable data Format of resource Language of resource Date of publication Type of material (i.e., digital) Standard identifier Yes No Somewhat Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat TRS-RI-A2 Directions: This requirement is not implemented and therefore not tested. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0), and Functional Area B that addresses the interchange of bibliographic holdings information. Directions: Perform a search specified in Appendix C. Compare information displayed in the RDS against the MARC record information available through the specified target (see Appendix C). Yes No Somewhat ZLOT Project Page 16 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement • • • • Technical Requirement Description and Test Directions 2) The application must be able to request and process the following information: Physical location of item Owning institution Circulation status Circulation policy Requirement Met? Comments Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat Directions: Verify if the interface provides a link, a button or other method to indicate whether the interlibrary loan is available or not, whether the document can be delivered or not, whether the document can be ordered on-line or not. 3) This information should be presented to the user in an effective way to assist the user in making a decision whether to initiate a request for the item. The search and retrieval interface application must provide functionality for the user to initiate an:    TRS-RI-A3 Interlibrary loan, Document delivery, Order from an online provider Yes No Somewhat Yes No Somewhat Yes No Somewhat Yes No Somewhat Directions: Perform a search specified in Appendix C. Compare information displayed in the RDS against the MARC record information available through the specified target (see Appendix C). 1) The search and retrieval interface application must be able to identify all retrieved records that indicate a described resource is available in digital form (e.g., through an indication in the MARC 856, a data element in the record from a TexShare database that the item is available in full-text, etc.). Directions: Verify if the interface provides icons, links or other methods to indicate that the item is available in digital form. 2) The application must clearly identify, through use of an icon or other technique that an item is available to be delivered to the user‟s computer in digital form. Directions: Verify if a hyperlink to the digital information object is available. 3) The application must also provide a hyperlink to the digital information object. Yes No Somewhat Yes No Somewhat TRS-RI-A4 Directions: Assuming that the user‟s home library is included in every search, this requirement will not be tested. Yes No Somewhat ZLOT Project Page 17 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions 1) The search and retrieval interface application must be able to check to see if a discovered resource is available from a user‟s home library. This implies that the user provides information to associate him/her with a home library (IP address, username/password, or user selected home library). 2) The application must indicate which of the items discovered are held by the home library. Requirement Met? Comments Yes No Somewhat TRS-RI-A5 Directions: Verify if the interface provides icons, links or other methods to indicate the following options for managing search results. The search and retrieval interface application must provide the user with the following options for managing search results (either results in the form of metadata records or the digital information object described): 1) Save the results (all or selected) to the user‟s local computer 2) Email the results (all or selected) to an email address supplied by the user 3) Print the results (all or selected) to a device associated with the user‟s local computer. Yes No Somewhat Yes No Somewhat Yes No Somewhat TRS-SRI-B2 Directions: Prior to testing this requirement consult with the TLDD to identify what information about the selected for testing library should be presented through the RDS interface. Login to the interface using selected library username and password and verify if the information about the library displayed through the interface is consistent with the TLDD information. The search and retrieval interface application should have the capability of presenting a custom user interface, under certain circumstances, where the interface presents the user with information about a local Texas library. If the user provides username and password that allows association with a local library, the application will provide the following in subsequent interfaces for the duration of the session (information about the local library will be stored in a relational database management system existing alongside the search and retrieval interface application and accessible from the application. Maintenance of information in the database is outside the scope of the search and retrieval interface application): 1) Name of local library 2) Hyperlink to local library home page 3) Basic contact information such as phone number and physical address Yes No Somewhat Yes No Somewhat Yes No Somewhat Yes No Somewhat TRS-SRI-B4 Directions: View the interface page source to verify that the interface connects to the TLDD to retrieve information about the library and that this information is not hard coded in the interface. ZLOT Project Page 18 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions The search and retrieval interface application must be able to query a database of information about Texas libraries to drive specific features of the user interface (see TRS-SRI-B1). (All data in the database associated with a single library can be considered a library‟s profile. Maintenance of the data in the database is outside the scope of the search and retrieval interface application. The search and retrieval interface application should assume the existence of a relational database and have the necessary capabilities for interacting with the database to pull information for use in the application.) Requirement Met? Comments TRS-SRI-B5 Directions: Verify if the list of target resources and links to brief or extensive information about a target resource are available. The search and retrieval interface application should present, upon user request, descriptive information about all target resources available through the LOT resource discovery service. Three approaches for describing target resources should be available through the application: 1) A simple list of all target resources available through the application Directions: Login to the administrative interface, input target description, and view inputted description through the end user interface. 2) Brief description of each target resource that is stored as part of the application Directions: Assuming that information about a resource comes from the TLDD, go to the TLDD, change resource description information and see if the changes appear in the RDS interface. 3) More extensive information about a resource stored outside of the application (e.g., in a database or on a website) Yes No Somewhat Yes No Somewhat Yes No Somewhat TRS-SRI-B6 Directions: Check with vendor how to ensure that the interface has a dynamic link to the information about the resources (link to TSLAC‟s TD-Net database, etc). Confirm that the RDS offers an option to view the list of journal titles in TexShare databases. 1) The search and retrieval interface application should be able to offer detailed information about the resources in each TexShare database when requested by the user. Directions: Select TexShare database and verify that the detailed information about the resource is available. If it is available, verify that it appears in the RDS. Yes No Somewhat Yes No Somewhat ZLOT Project Page 19 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions 2) The application should be able to interact with or link to descriptions of the databases stored separately from the application. Requirement Met? Comments TRS-SRI-B7 Directions: Verify if a link, a button or other option to select RDS languages is available. 1) The search and retrieval interface application could present a user interface in languages other than English. (Excluded from this feature are automatic translation of queries from non-English languages into the English and automatic translation of retrieved records or resources into the non-English language chosen by the user for the interface.) 2) Help and context-sensitive help would be presented in the language chosen by the user. Yes No Somewhat Yes No Somewhat TRS-SI-B1.1 Directions: Perform the following searches against a target catalog that has a Z39.50 interface. Analyze z-server and z-client log files to verify Bath compliance. Search terms for testing this requirement are provided in Appendix A. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client must support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0, 1, and 2 requirements for searching that cover the following search types or search functionality: • Boolean searches • Correct truncation for keyword and phrase searches • Exact-match searches • First-words in field searches • First-characters-in-field searches • ISBN and ISSN searches Directions: Perform searches against a non-Z39.50 database (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific Texas library online catalog systems that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application should be able to issue these search types and search functionality to the selected online catalog systems through their web interface. • • Boolean searches Correct truncation for keyword and phrase searches Yes No Somewhat Yes No Somewhat Yes No Somewhat Yes Yes Yes Yes Yes Yes Yes No No No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat Somewhat ZLOT Project Page 20 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement • • • • TRS-SI-B1.2 Technical Requirement Description and Test Directions Exact-match searches First-words in field searches First-characters-in-field searches ISBN and ISSN searches Requirement Met? Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat Comments Directions: This requirement is tested as part of TRS-RI-A3. The search and retrieval interface application should be able to process results retrieved from online catalog systems to filter for MARC records that have 856 fields which indicate the resource described is electronically available (e.g., full text) Yes No Somewhat TRS-SI-B1.3 Directions: Consult the vendor‟s documentation and verify if the search history function is presented in the interface (button, link, etc). The search and retrieval interface application should be able to provide the user with a search history. Yes No Somewhat TRS-SI-B1.4 Directions: Perform the following searches against a TexShare database that includes a Z39.50 interface; analyze vendor‟s log files against BATH profile to check for conformance. Use search terms provided in Appendix B. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target non-catalog target resources (e.g., the TexShare databases and the TRAIL system) that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client should support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0, 1, and 2 requirements for searching that cover the following search types or search functionality: • Boolean searches • Exact match searches • Phrase searching • ISSN searches Directions: Perform searches against a non-Z39.50 database (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific non-catalog target resources that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application should be able to issue these search types and search Functionality to the selected target resources through their web interface. • Boolean searches Yes No Somewhat Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat Yes No Somewhat ZLOT Project Page 21 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement • • • TRS-SI-B1.5 Technical Requirement Description and Test Directions Exact match searches Phrase searching ISSN searches Requirement Met? Yes No Somewhat Yes No Somewhat Yes No Somewhat Yes No Somewhat Comments Directions: Verify if an icon, a link or other option is available to limit results to those available only in electronic form or full text. The search and retrieval interface application should be able to issue a query to non-catalog target resources to limit results to items only available in electronic form or full-text. TRS-SI-B2.1 Directions: Perform the following searches against a target catalog that has a Z39.50 interface. Analyze z-server and z-client log files to verify Bath compliance. Search terms for testing this requirement are provided in Appendix A. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client should support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Levels 0, 1, and 2 requirements for searching where the query contains the following qualifying criteria. The interface application will make this feature visible only when it is appropriate for the selected target resources. • • • • • Format Language of resource described by a record Date of publication of the resource Date range Type of material Yes No Somewhat Yes Yes Yes Yes Yes No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Directions: Perform the following qualifying searches against a non-Z39.50 catalog by linking to the native interface (such as the UT online catalog). Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix A. 2) For a limited set of specific Texas library online catalog systems that do not support a Z39.50 interface, but are included as LOT target resources, the search and retrieval interface application should be able to issue these qualifying search criteria to the selected online catalog systems through their web interface. The interface application will make this feature visible only when it is appropriate for the selected target resources. • Format • Language of resource described by a record • Date of publication of the resource • Date range Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat ZLOT Project Page 22 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement • TRS-SI-B2.2 Technical Requirement Description and Test Directions Type of material Requirement Met? Yes No Somewhat Comments Directions: Perform searches containing the following qualifying criteria against a TexShare database that includes a Z39.50 interface. Analyze log files against BATH profile to check for conformance. Use search terms provided in Appendix B. 1) The search and retrieval interface application must include a Z39.50 client that can communicate with target non-catalog target resources (e.g., the TexShare databases and the TRAIL system) that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client should support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 0, 1, and 2 requirements for searching where the query contains the following qualifying criteria: • • • • • Format Language of resource described by a record Date of publication of the resource Date range Type of material Yes Yes Yes Yes Yes No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Directions: Perform qualifying searches against a TexShare database that does not include a Z39.50 interface. Analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix B. 2) For a limited set of specific non-catalog target resources that do not support a Z39.50 interface but are included as LOT target resources, the search and retrieval interface application should be able to issue these qualifying search criteria to the selected target resources through their web interface. • • • • • TRS-SI-B3.1 Format Language of resource described by a record Date of publication of the resource Date range Type of material Yes Yes Yes Yes Yes No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Directions: Select one target with the Z39.50 interface and one target without a Z39.50 interface. Send concurrent search queries, get results, and analyze log files. For the target with the Z39.50 interface analyze log files to check for BATH profile conformance. For the target without the Z39.50 interface analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix B. ZLOT Project Page 23 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions The search and retrieval interface application should be able to issue the same query (or semantically equivalent queries) concurrently to two or more LOT target resources (broadcast searching), where the target resources may or may not have a Z39.50 interface to the underlying information retrieval system. Broadcast searching using the types of searches defined in TRS-SI-A1.1 and TRS-SI-A2.1, and other technical requirements and specifications should be supported.     Author Keyword Title Keyword Subject Keyword General Keyword Requirement Met? Comments Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat TRS-SI-B3.2 Directions: This requirement will be tested as part of the TRS-SI-B3.1 1) The search and retrieval interface application should allow the user to select two or more LOT target resources to search concurrently. Directions: Functionality available through the administrative interface is tested under TRS-SRI-A2. 2) The application should also support administratively defined sets of LOT target resources to search concurrently (e.g., based on subject coverage or other criteria). Yes No Somewhat Yes No Somewhat TRS-SI-B4.1 Directions: Select one target with the Z39.50 interface, one target without the Z39.50 interface, and one or more web search engines. Send concurrent search queries, get results, and analyze log files. For the target with the Z39.50 interface analyze log files to check for BATH profile conformance. For the target without the Z39.50 interface analyze log files to verify if the RDS is sending plausible search strings. Use search terms provided in Appendix B. The search and retrieval interface application should be able to issue the same query (or semantically equivalent queries) concurrently to two or more LOT target resources and to one or more web search engines (broadcast searching), where the target resources and the web search engines may or may not have a Z39.50 interface to the underlying information retrieval system. Broadcast searching using the types of searches defined in TRS-SI-A1.1 and TRS-SI-A2.1, and other technical requirements and specifications should be supported.     Author Keyword Title Keyword Subject Keyword General Keyword Yes No Somewhat Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat ZLOT Project Page 24 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-SI-B4.2 Technical Requirement Description and Test Directions Directions: This requirement is tested as part of the TRS-SI-B4.1. 1) The search and retrieval interface application should allow the user to select two or more LOT target resources and one or more web search engines to search concurrently. Directions: Login to the administrative interface and verify that LOT target resources and web search engines can be grouped for concurrent searches. 2) The application should also support administratively defined sets of LOT target resources and web search engines to search concurrently (e.g., based on subject coverage or other criteria). Requirement Met? Yes No Somewhat Comments Yes No Somewhat TRS-SI-B5.1 Directions: Login to the administrative interface and verify that the sets of LOT targets can be defined based on subject coverage. 1) The search and retrieval interface application should present, through the user interface, administratively defined sets of LOT target resources based on subject coverage of the resources. Directions: Through the end user interface evaluators should be able to select a subject and view administratively defined sets of targets on a selected subject. 2) Users can select a topic or subject area and be presented with a defined set of appropriate LOT target resources to which searches can be issued. Yes No Somewhat Yes No Somewhat TRS-SI-B5.2 Directions: Login to the administrative interface, verify that the subject categories can be modified and LOT targets grouped by subject categories. The administrative module of the search and retrieval interface should allow the administrator to name and organize the subject categories, and to associate appropriate LOT target resource with one or more of the subject categories. Yes No Somewhat TRS-SI-B6 Directions: Verify that the „type of library‟ option (button, icon, etc) is available for selecting online catalog systems. 1) The search and retrieval interface application should present a list of LOT target online catalog systems grouped according to type of library. Directions: Verify if several library types could be selected to search. 2) Users will be able to select one or more from the list of libraries to search. Yes No Somewhat Yes No Somewhat ZLOT Project Page 25 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions Directions: Login to the administrative interface and verify if targets can be grouped by public and academic library types. A default set of public libraries and a default set of academic libraries should be able to be defined by the administrator or a local library for use by patrons associated with that library (see TRS-SRI-A5). Requirement Met? Yes No Somewhat Comments TRS-SI-B7 Directions: Perform search, get results, save results, re-run canned search using URL or other interface option. The search and retrieval interface application should allow administrative configuration to bring together selected resources based on a range of criteria. This can be implemented through a pre-tested default search the results of which are cached for efficient access by a user. Yes No Somewhat TRS-SI-B8 Directions: Functionality available through the administrative interface is tested under TRS-SRI-A2. Verify if the default group of online catalog systems can be modified through the user interface. The search and retrieval interface application should present a pre-selected set of online catalog systems that serves as a default set of targets for searching by the user. The user‟s home library selects the target resources. When a user is associated with a local library by the application, this default set of online catalogs is the target for searching. Users must be able to refine (select and/deselect) this default group of online catalog systems to search. Yes No Somewhat TRS-SI-B9.1 Directions: Verify if the interface presents an option to retrieve resources that are available in a digital form. Evaluators should be able to locate a button or other mechanism to enable filtering out materials based on their availability (e.g. Need Now button.) The search and retrieval interface application should provide, at a minimum, a mechanism through which the user can request a search that returns descriptive records of information objects that can be delivered to the user‟s computer (i.e., resources that are in digital form and can be delivered directly to the desktop). Yes No Somewhat TRS-SI-B9.2 Directions: Check the URL and verify if its format is consistent with the Open URL structure. The search and retrieval interface application should be extensible to use Open URL linking mechanisms to assist users in getting to available resources from a citation. Yes No Somewhat ZLOT Project Page 26 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-SI-B10 Technical Requirement Description and Test Directions Directions: Identify Z39.50 target with valid IP/domain name and add it through the administrative interface. Identify active non-Z39.50 target that is not included in the RDS and add it through the administrative interface. 1) The search and retrieval interface application should provide administrative configuration functionality that allows new target resources to be easily added for user searching. Directions: Perform searches against TRAIL, Z39.50 target catalog, and TexShare database and verify that the retrieved data are presented consistently. 2) The application should be extensible to handle a range of metadata formats in multiple syntaxes, yet be able to present to the user a consistent view of the search results no matter what native metadata scheme of the target resource. Requirement Met? Yes No Somewhat Comments Yes No Somewhat TRS-SI-B11 Directions: Perform the following searches against a target catalog that has a Z39.50 interface. Analyze z-server and z-client log files to verify Bath compliance. Use search terms that are provided in Appendix A. The search and retrieval interface application should include a Z39.50 client that can communicate with target online catalog systems that have a Z39.50 interface to the underlying information retrieval system. The Z39.50 client should support the Bath Profile, Release (2.0) and the U.S. National Profile (ANSI/NISO Z39.89), Functional Area A, Level 1 requirements for use of the Scan service.     Author Keyword Title Keyword Subject Keyword General Keyword Yes No Somewhat Yes Yes Yes Yes No No No No Somewhat Somewhat Somewhat Somewhat TRS-RI-B1 Directions: Verify with the TSLAC that the vendor submitted usability testing documentation pertaining to this requirement. 1) The search and retrieval interface application should process retrieved data that indicate type and format of described information resources. The application should be able to group together the discovered resources according to type and format, and clearly label the groupings.   By type By format Yes No Somewhat Yes No Somewhat ZLOT Project Page 27 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions Directions: Perform searches in the RDS and native interfaces and verify if the type and format data are consistently presented in both interfaces. View MARC record information of the retrieved record to verify if the type and format data are available. If type and format data are available, verify that the RDS interface groups and displays resources by the type and format. 2) The vendor should indicate the extent of usability testing that informs the design of this functionality and presentation.   By type By format Requirement Met? Comments Yes No Somewhat Yes No Somewhat TRS-RI-B2 Directions: Verify if the interface presents icons, buttons, links or other methods to group retrieved results by the following criteria. Verify that consistent graphics are used to indicate format and type of resource. 1) The search and retrieval interface application should present the user with the following ways to group the retrieval results. Use of consistent icons to communicate format and type of material of the described resources is recommended. The interface application will make this feature visible only when it is appropriate for the selected target resources. • • • • • By format (e.g., book or video) By type of material (digital or analog) By date of publication By language of described resource Presented consistently Yes Yes Yes Yes Yes No No No No No Somewhat Somewhat Somewhat Somewhat Somewhat Directions: Select several online catalog systems, perform a search, verify if options are available to group results by physical distance from the user (by clicking on a button, drop down menu listing distances, link, etc). 2) Results from searches of online catalog systems should provide a mechanism to allow users to group results by libraries and those libraries physical distance from the user. The interface application will make this feature visible only when it is appropriate for the selected target resources. Directions: Verify that the retrieved results appear according to the user grouping criteria. Yes No Somewhat Yes No Somewhat ZLOT Project Page 28 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions 3) When the user selects one of the grouping criteria, the application should process retrieved data and present the results accordingly. The interface application will make this feature visible only when it is appropriate for the selected target resources. Requirement Met? Comments TRS-RI-B3 Directions: Perform a search of a TexShare database; make sure that the retrieved results are not available in electronic format. Verify if the RDS presents a link or a button to enable the follow-on journal title and/or ISSN search of online catalog systems to discover which libraries hold the journal. The search and retrieval interface application should support a follow-on search for discovered items, where the items are published in a journal and where the items are not available in digital form to the user. The application should allow the user to select a journal title and have the application query a selected set of online catalog systems to discover which libraries hold the journal. The follow-on search could include an ISSN or the journal title. Yes No Somewhat TRS-RI-B4.1 Directions: This requirement is tested under TRS-RI-B4.2. The search and retrieval interface application for the resource discovery service should provide an intersection with the LOT resource access/document delivery service. This service should provide the user with options for acquiring the resource. Yes No Somewhat TRS-RI-B4.2 Directions: Identify the resource to initiate document delivery or interlibrary loan. Click on the link, button, icon or other tool to facilitate interlibrary loan request; see if it launches the email browser with the message to the specified library (e.g. UNT ZInterop collection). Insert name and email of evaluator Irene Lopatovska as ILL UNT contact. Compile and send the message, if possible .cc evaluator; if not, contact the specified library to request a copy of the interlibrary loan request. 1) If a discovered resource is available from a library, the application should include a mechanism to initiate an interlibrary loan or document delivery request. The search and retrieval interface application can identify an appropriate email address for the home library from the library database described in other technical requirements and specifications. Directions: Analyze the text of the interlibrary loan email message to verify that it includes bibliographic information. 2) The form of this request is an email message to the user‟s home library, where the email message automatically includes the bibliographic information from the retrieved record. Yes No Somewhat Yes No Somewhat ZLOT Project Page 29 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement Technical Requirement Description and Test Directions Directions: Verify that the interlibrary loan request form includes fields for user‟s personal information. 3) And provides a form for the user to complete personal information to be used by the home library for processing the interlibrary loan request. Requirement Met? Yes No Somewhat Comments TRS-RI-B4.3 Directions: Locate and click on the buttons or links to one or more online bookstores. 1) The search and retrieval interface application should offer the user a link to one or more online bookstores and when one is selected send the bookstore system a search for the resource desired by the user. 2) The user is then taken to the native interface of the online bookstore for subsequent interaction to order the item. Directions: Click on the link to online bookstore and make sure that it opens a new browser window. 3) The search and retrieval interface application should open another browser window when taking the user to the online bookstore. Directions: Verify if a button, link or icon is available to find more information objects that are in some way similar to the retrieved one. 1) The search and retrieval interface should provide a mechanism that allows the user to select a record that has been retrieved and ask the application to find more information objects that are similar. Directions: Verify if options for selecting similar types for the follow-up searches are presented 2) The application can assume that the user is interested in finding items that are similar in content, although it should be prepared to solicit from the user other types of similarity (subject and format, etc.). Directions: Perform follow-up searches for similar records and verify if the retrieved results are similar to the initially retrieved records. 3) The application should be able to construct a robust query that it sends to appropriate target resources to identify additional resources that are most similar to the one selected by the user as the base resource. Yes No Somewhat Yes No Somewhat Yes No Somewhat TRS-RI-B5 Yes No Somewhat Yes No Somewhat Yes No Somewhat ZLOT Project Page 30 September 3, 2003 ACCEPTANCE TEST PLAN Technical Requirement TRS-SRI-C1 Technical Requirement Description and Test Directions Directions: Verify if the interface provides an option to 1) define user profile; 2) select target resources not included in the profile; 3) prompt the user to store the profile information. The search and retrieval application should provide a limited personalization service to users. 1) Through the user interface, the user will be able to define a profile that includes at minimum one or more sets of target resources that are searched concurrently. 2) The user should be able to select other target resources available through the LOT, even when they are not defined as a preferred set to search. 3) Users must have the option to permit or refuse the storage of this profile information on their local computer. Requirement Met? Comments Yes No Somewhat Yes No Somewhat Yes No Somewhat ZLOT Project Page 31 September 3, 2003 ACCEPTANCE TEST PLAN Appendix A: Z39.50 Search Queries Following are the specific searches and search terms sent in the queries for this evaluation. 1. Any Keyword (TRS-SI-A1.2) Evaluators will send 1 search using one of the specified search terms. Search Number 1 2 2. Type of Search Any Search -- Keyword Search Term mobile nepal Author, Title, Subject Keyword (TRS-SI-A2.2) Evaluators will send 1 search per type of search using one of the specified search terms. Evaluators will send 2 search terms simultaneously to test if the Bath compliant search will fail. Search Number 3 4 5 6 7 8 3. Boolean (TRS-SI-B1.1) Type of Search Author Search – Keyword Title Search – Keyword Subject Search – Keyword Search Term aleksandrovich elena astronomical nepal dragons mobile Evaluators will send 1 search using the specified search terms. Search Number 9 10 11 12 13 14 15 16 17 18 19 20 4. Type of Search Any Search – Keyword Any Search – Keyword Any Search – Keyword Author Search – Keyword Author Search – Keyword Author Search – Keyword Title Search – Keyword Title Search – Keyword Title Search – Keyword Subject Search – Keyword Subject Search – Keyword Subject Search – Keyword Boolean Operator OR AND NOT OR AND NOT OR AND NOT OR AND NOT Search Term #1 root nepal mobile aleksandrovich jessica garrison nepal astronomical treasures mobile semiconductors dragons Search Term #2 whale everest homes alexandra stirling hyslop festivals almanac museum robots congresses monsters Truncation (TRS-SI-B1.1) Evaluators will send 1 search using one of the specified search terms. ZLOT Project Page 32 September 3, 2003 ACCEPTANCE TEST PLAN Search Number 21 22 23 24 25 26 27 28 Type of Search Any Search – Keyword with Right Truncation Search Term mobile nepal aleksandrovich elena astronomical nepal dragons mobile Author Search – Keyword with Right Truncation Title Search – Keyword with Right Truncation Subject Search – Keyword with Right Truncation 5. Exact Match (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 29 30 31 6. Type of Search Author -Exact Match Title –Exact Match Subject –Exact Match Search Term henry mintzberg the fifth discipline knowledge transfer First Words in Field (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 32 33 34 7. Type of Search Author -First Words in Field Title -First Words in Field Subject -First Words in Field Search Term henry the fifth knowledge First Characters in Field (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 35 36 37 8. Type of Search Author -First Characters in Field Title -First Characters in Field Subject -First Characters in Field Search Term hen the fif knowl ISBN & ISSN (TRS-SI-B1.1) ZLOT Project Page 33 September 3, 2003 ACCEPTANCE TEST PLAN Evaluators will send 1 search using the specified search terms. Search Number 38 39 Type of Search ISBN ISSN Search Term 0824-1961 0363-0277 ZLOT Project Page 34 September 3, 2003 ACCEPTANCE TEST PLAN Appendix B: TexShare Search Queries Following are the specific searches and search terms sent in the queries for this evaluation. 1. Any Keyword (TRS-SI-A1.3) For each general keyword search, the ZLOT RFP evaluator will send 1 search using one of the specified search terms. Search Number 1 2 3 4 5 2. Type of Search Any Search -- Keyword Search Term dojo encoding glossy gremlin transom Author, Title, Subject Keyword (TRS-SI-A2.3) For each of the four types of keyword searches, the participant will send 1 search using the specified search terms for each type of search. Since the general keyword search was performed in the previous step, there are 3 remaining types of keyword searches left to be performed. Search Number 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 3. Boolean (TRS-SI-B1.1) Type of Search Author Search – Keyword Search Term dojo kantner modigliani penzias picasso bellagio biltmore glorification inane sump bizarre decoys dragons supernatural needlepoint Title Search – Keyword Subject Search – Keyword Evaluators will send 1 search using the specified search terms. Search Number 1 2 3 4 Type of Search Any Search – Keyword Any Search – Keyword Any Search – Keyword Any Search – Keyword Boolean Operator OR OR AND AND Search Term #1 encoding glossy search awards Search Term #2 sump transom mission legacy ZLOT Project Page 35 September 3, 2003 ACCEPTANCE TEST PLAN Search Number 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 4. Type of Search Any Search – Keyword Any Search – Keyword Author Search – Keyword Author Search – Keyword Author Search – Keyword Author Search – Keyword Author Search – Keyword Author Search – Keyword Title Search – Keyword Title Search – Keyword Title Search – Keyword Title Search – Keyword Title Search – Keyword Title Search – Keyword Subject Search – Keyword Subject Search – Keyword Subject Search – Keyword Subject Search – Keyword Subject Search – Keyword Subject Search – Keyword Boolean Operator NOT NOT OR OR AND AND NOT NOT OR OR AND AND NOT NOT OR OR AND AND NOT NOT Search Term #1 tiara prim dojo picasso sterling brown lautrec picasso conceited whidbey united museum ndebele rustlers decoys bizarre semiconductors varieties dragons supernatural Search Term #2 crown proper kantner lorde smith duguid toulouse rubin inane bremerton turkey gifts africa cattle dragons needlepoint industry plants monsters literature Truncation (TRS-SI-B1.1) For each of the following keyword truncation searches, the participant will send 1 search using the specified search terms for each type of search. Search term is right truncated after last letter in word. Search Number 1 2 3 4 5 6 7 8 Type of Search Any Search – Keyword with Right Truncation Author Search – Keyword with Right Truncation Title Search – Keyword with Right Truncation Subject Search – Keyword with Right Truncation Search Term dojo gremlin dojo kantner biltmore sump dragons supernatural 5. Exact Match (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 1 2 Type of Search Exact Match - Author Exact Match - Author Search Term henry mintzberg rosabeth kantner ZLOT Project Page 36 September 3, 2003 ACCEPTANCE TEST PLAN 3 4 5 6 6. Exact Match – Title Exact Match - Title Exact Match – Subject Exact Match - Subject the fifth discipline the social life of information knowledge transfer elder care First Words in Field (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 1 2 3 4 5 6 7. Type of Search Author -First Words in Field Author -First Words in Field Title -First Words in Field Title -First Words in Field Subject -First Words in Field Subject -First Words in Field Search Term henry rosabeth the fifth the social life knowledge elder First Characters in Field (TRS-SI-B1.1) Evaluators will send 1 search using the specified search terms. Search Number 1 2 3 4 5 6 8. ISSN (TRS-SI-B1.1) Type of Search Author -First Characters in Field Author -First Characters in Field Title -First Characters in Field Title -First Characters in Field Subject -First Characters in Field Subject -First Characters in Field Search Term hen rosa the fif the soc knowl eld Evaluators will send 1 search using the specified search terms. Search Number 1 2 9. Type of Search ISBN ISSN Search Term 0824-1961 0363-0277 Any, Author, Title, Subject Keyword (TRS-SI-A4.1) Each of the keyword searches must be issued concurrently to the TexShare databases. Evaluators will send 1 search using the specified search terms. Search Number 1 2 3 4 Type of Search Any Search -- Keyword Search Term dojo encoding glossy gremlin ZLOT Project Page 37 September 3, 2003 ACCEPTANCE TEST PLAN Search Number 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Type of Search Search Term transom Author Search – Keyword dojo kantner modigliani penzias picasso bellagio biltmore glorification inane sump bizarre decoys dragons supernatural needlepoint Title Search – Keyword Subject Search – Keyword 10. Any, Author, Title, Subject Keyword (TRS-SI-B3.1) Each of the keyword searches must be issued concurrently to the two TexShare databases and the Bath compliant Z-Interop search targets. Evaluators will send 1 search using the specified search terms. Search Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Type of Search Any Search -- Keyword Search Term dojo encoding glossy gremlin transom dojo kantner modigliani penzias picasso bellagio biltmore glorification inane sump bizarre decoys dragons supernatural needlepoint Author Search – Keyword Title Search – Keyword Subject Search – Keyword ZLOT Project Page 38 September 3, 2003 ACCEPTANCE TEST PLAN Appendix C: MARC record The following appendix includes a copy of a MARC record retrieved from Sirsi‟s Austin Collage catalog. Library Name Austin College Library Z-Address Abell.austincollege. edu Port 2200 Database Name Unicorn WebCat http://abell.austincollege.edu/uhtbin/cgi sirsi/KVi6U7jlaQ/269660011/60/53/X Author Search = McClure 090 100 245 260 300 651 630 630 926 $aDS 107$b.T467 1 $aThomson, William McClure,$d1806-1894. 14$aThe Land and the Book;$bor Biblical illustrations drawn from the manners and customs, the scenes and scenery of the Holy Land.$cBy W. M. Thomson. $aNew York,$bHarper$c[1858] $a2 v.$billus.$c23 cm. 0$aPalestine$xDescription and travel.$9sh 85097163-6 00$aBible$xCriticism, interpretation, etc.$9sh 85013617-1 00$aBible$xGeography.$9sh 85013642-1 $aABELL$bSTACKS$cDS107.T467 V.1$dBOOK$f1 ZLOT Project Page 39 September 3, 2003 ACCEPTANCE TEST PLAN References US National Profile (ANSI/NISO Z39.89). Available: http://www.niso.org/standards/balloting.html Bath Profile, Release 2. Available: http://www.nlc-bnc.ca/bath/bath-draft2-2-e.html CSTE. Body of Knowledge: Acceptance Testing. (2003). Available: http://www.softwarecertifications.cam/cstebok/cstebok10.htm Software Testing Answers FAQs. (2003). Available: http://www.geocities.com/pammal_surshbabu/software_testing/qa7.html ZLOT Project Page 40 September 3, 2003