Docstoc

qa-focus_1_

Document Sample
qa-focus_1_ Powered By Docstoc
					    Quality Assurance (QA) For
    Digital Library Projects
                                                              This talk will describe
                                                              the work of the JISC-
    Brian Kelly                                               funded QA Focus
    UKOLN                                                     team and outlines
    B.Kelly@ukoln.ac.uk                                       approaches to QA
                                                              which can be
                                                              deployed in Digital
                                                              Library projects

    http://www.ukoln.ac.uk/qa-focus/

    QA Focus – Supporting JISC's Digital Library Programmes
1
    Contents
       • Introduction
       • Background:
            • The need for QA                          QA Focus
       • Work activities:
            • Web                                      Accessibility
            • Providing Advice
       • Future Work
            • Metadata                                 Software Development
            • Service Deployment                       Toolkit
       • A QA Methodology
       • Relevance To You
       • Conclusions
    QA Focus – Supporting JISC's Digital Library Programmes
2
    About UK Web Focus / UKOLN
    UK Web Focus:
     • Funded by JISC since Nov 1996 to advice UK HE
       (and the FE) on standards and best practices for the
       Web
     • Provided by Brian Kelly
     • Based at UKOLN, a national centre of expertise in
       digital information management
     • Located at the University of Bath
     • Project manager for JISC's QA Focus project


    Note: Since 1 August 2003 UK Web Focus has been jointly
    funded by JISC and Resource and now also advises the
    cultural heritage sector in England and Wales
    QA Focus – Supporting JISC's Digital Library Programmes
3
    Introduction   Aims Of Today’s Talk
                     • To describe the role of quality assurance in
                       digital library programmes
                     • To describe the approach taken by the QA
                       Focus project
                     • To summarise QA Focus activities to date
                     • To describe future activities
                     • To explore potential for collaboration with
                       projects & services represented here

                    Note that although this talk is based on a JISC-funded
                    project the content is applicable to everyone involved in
                   QA Focusdevelopment activities – including Webmasters
                    Web – Supporting JISC's Digital Library Programmes
4
    Introduction   An Introduction to QA
                   What is Quality?
                        “Quality is the ability of your product to be able to
                        satisfy your users”

                   What is Quality Assurance?
                        “Quality assurance is the process that demonstrates
                        your product is able to satisfy your users”




                   QA Focus – Supporting JISC's Digital Library Programmes
5
    Introduction   An Introduction to QA
                   Why Is QA Needed Now?
                   In the past JISC had standards-based philosophy but:
                      • Approach of "let a 1,000 flowers bloom"
                      • Diversification as no killer application then
                        (it could have been Gopher!)
                   Things are different now:
                     • The Web is the killer application
                     • There is now a need for interoperability (not just
                       self-contained independent Web sites)
                     • There is a need for publicly-funded services to be
                       seen to provide value for money

                   QA Focus – Supporting JISC's Digital Library Programmes
6
    Introduction   Approach Taken
                   Two possible approaches to ensuring
                   compliance with standards and best practices:
                     Enforce
                     • Inspect all project’s work
                     • Strict auditing, with penalties for no-compliance
                     Encourage
                     • Training for project staff
                     • Developmental, explaining reasons for compliance,
                       documenting examples of best practices and
                       providing advice on implementation and monitoring
                          The latter approach is preferable,
                          especially in a HE context
                   QA Focus – Supporting JISC's Digital Library Programmes
7
    QA Focus   QA Focus
               QA Focus:
                • Funded by JISC
                • Currently provided by UKOLN and AHDS
                • Supports JISC's 5/99, FAIR & X4L programmes
                • Staff:
                          Brian Kelly, Project manager
                          (Marieke Guy, QA Focus officer, UKOLN)
                          Hamish James, AHDS
                          Gareth Knight, QA Focus officer, AHDS
                 • Areas covered:
                        Web                                Digitisation
                        Metadata                           Software development
                        Service deployment
               QA Focus – Supporting JISC's Digital Library Programmes
8
    QA For Web
                 QA For Web Sites
                 The issues:
                  • The Web is the main delivery mechanism for projects
                      and services
                  • An increasing awareness of the importance of:
                           Accessibility
                           Use of new devices (PDAs, WAP, e-books, …)
                           Repurposing of Web content (e.g. archiving)
                  • Technologies such as XSLT will support repurposing
                      of valid XML resources
                 But:
                  • Invalid HTML is the norm
                  • Many authoring tools produce poor HTML
                  • Authors aren’t Digital Library Programmes
                 QA Focus – Supporting JISC's aware of the problems
9
     QA For Web   Guidelines
                  We often say:
                     • Open standards are important
                     • HTML, XML, XHTML, CSS, … are important
                  but fail to explain why and how
                  JISC’s QA Focus is addressing such concerns by:
                     • Documenting example of best practices in which
                        projects can share their implementation successes
                        (and difficulties they experienced)
                     • Provide brief advice in specific aspects of the
                        standards and best practices
                     • Surveying its communities to highlight best
                        practices and areas in which improvements can be
                        made
                     • Demonstrating use of testing tools and procedures
                  QA Focus – Supporting JISC's Digital Library Programmes
10
     QA For Web
                  Standards & Best Practices

                  Standards For Web:
                    • Use compliant HTML / XHTML
                    • Use CSS
                  Best Practices For Web:
                    • Implement WAI accessibility guidelines
                    • Ensure Web resources are suitable for reuse and
                      repurposing
                    • Where proprietary formats need to be used, flag
                      them and use in most open way



                  QA Focus – Supporting JISC's Digital Library Programmes
11
     QA For Web   Surveying The Community
                  Surveys of JISC 5/99 (and FAIR & X4L) project Web
                  sites have been carried out in order to:
                      • Obtain a profile for the community
                      • Identify examples of best practices
                      • Identify areas in which further advice is needed
                  Surveys included:
                      • HTML & CSS compliance           Accessibility
                      • 404 error pages                 HTTP headers
                      • Repurposing resources



                  QA Focus – Supporting JISC's Digital Library Programmes
12
     QA For Web   Survey Philosophy
                  The surveys made use of freely-available Web-based
                  tools:
                       • Methodology is open
                       • No software needs to be installed locally (apart from Web
                         browser)
                       • Findings can be reproduced
                       • Latest results can be obtained by clicking on link to testing
                         service
                  The surveys typically examined project entry points and
                  not entire Web site as:
                       • This page has the highest profile
                       • The aim is to validate a methodology which can be deployed
                         by projects themselves, not to test every page on behalf of
                         the projects

                  QA Focus – Supporting JISC's Digital Library Programmes
13
     Advice
              Documentation: Case Studies

                                                                        Case Studies are
                                                                        being commissioned
                                                                        These are:
                                                                         • Written by
                                                                           projects
                                                                           themselves
                                                                         • Describe the
                                                                           solution adopted
                                                                           to a particular
                                                                           problem
                                                                         • Include details of
                                                                           lessons learnt –
                                                                           not just a press
                                                                           release!

              QA Focus – Supporting JISC's Digital Library Programmes
14
     QA For Web   Survey Findings
                  Initial set of findings available from
                  <http://www.ukoln.ac.uk/qa-focus/surveys/web-10-2002/>




                  QA Focus – Supporting JISC's Digital Library Programmes
15
     QA For Web   Providing Motivation
                  We have found evidence of failure to comply with HTML
                  standards
                  There is a need to explain why compliance is important
                  (and avoid the “it’s OK in my browser” argument) and to
                  provide motivation for projects to update their tools,
                  authoring procedures, etc.
                  A further set of surveys look at repurposing of the
                  project Web sites:
                       • Availability of Web sites in the Internet Archive
                       • Ease of making Web sites available on a PDA
                       • Transformation of embedded metadata



                  QA Focus – Supporting JISC's Digital Library Programmes
16
     QA For Web   Repurposing Resources
                  We examined the Web sites to see if they were
                  available in the Internet Archive and could be
                  transformed into a format for viewing on a PDA

                                                                            AA small number of
                                                                              small number
                                                                             Web sites could
                                                                            of Web sites not
                                                                             be transformed.
                                                                            were not in the
                                                                             Analysis of HTTP
                                                                            Internet Archive
                                                                             headers indicated
                                                                            due to the due to
                                                                             that this was
                                                                            robots.txt file.
                                                                             incorrect HTTP
                                                                             headers.
                                                                            We will need to
                                                                             We will advice
                                                                            provide need to in
                                                                             provide advice in
                                                                            this area.
                                                                             this area.
                  QA Focus – Supporting JISC's Digital Library Programmes
17
     QA For Web   Surveys For FAIR And X4L Programmes
                                             http://www.ukoln.ac.uk/qa-focus/
                  QA Focus now               surveys/fair/web-self-assessment/
                  supports JISC's
                  FAIR and X4L
                  programmes
                  The Web site
                  surveys now
                  provides a self-
                  assessment
                  interface



                  We hope that projects will (a) check and, if necessary, fix
                  their home page; (b) use such tools to check/fix other key
                   QA Focus and (c) address underlying
                  pages – Supporting JISC's Digital Library Programmescauses of problems
18
     QA For Web   A Caveat
                  There are a number of limitations to the work we have
                  carried out so far:
                    • Project Web sites have different purposes
                      (information about the project; communications with
                      project partners; project deliverables themselves;
                      etc.)
                    • Projects have different levels of funding, resources,
                      expertise, etc.
                    • Projects are at different stages of development (and
                      some have finished)
                  The surveys are intended to demonstrate a methodology
                  which projects can use for themselves

                  QA Focus – Supporting JISC's Digital Library Programmes
19
     Advice   Providing Advice
              We have:
                 • Surveyed project Web sites and identified areas of
                   lack of compliance with standards & best practices
                 • Shown examples of the potential importance of
                   compliance for repurposing resources
              In addition we need to provide:
                 • Brief focussed advice on the standards
                 • Information on how to monitor compliance
                 • Case studies on solutions deployed by projects
                   themselves
                 • Guidance on dealing with implementation difficulties
                   and what to do when strict compliance
                   is difficult to achieve
              QA Focus – Supporting JISC's Digital Library Programmes
20
     Advice
              Documentation: Advice

                                                                        Advisory briefing
                                                                        documents are being
                                                                        produced
                                                                        These are:
                                                                         • Brief, focussed
                                                                           documents
                                                                         • Informed by
                                                                           findings of the
                                                                           surveys



              QA Focus – Supporting JISC's Digital Library Programmes
21
     Tools   Putting it Together
             Based on surveys we find project Web sites which have
             invalid HTML and CSS or other examples of failure to
             implement best practices
             So we:
                • Explain why projects should follow best practices
                • Provide case studies on approaches by projects
             But how do we ensure projects :
                • Check Web sites for themselves
                • Have systematic checking procedures
             There is a need to describe:
                • Tools & architectures for projects for check
                  compliance with standards & best practices
                • Systematic QA procedures
             QA Focus – Supporting JISC's Digital Library Programmes
22
     Tools   Example - Link Checking Tools
             We should all be familiar
             with link checking tools.
             But:
              • Do such tools check all
                 types of links (e.g. What
                 about links to external
                 style sheet files,
                 JavaScript files, etc.)?
              • Are such tools
                 integrated with normal
                 working practices?


             QA Focus – Supporting JISC's Digital Library Programmes
23
          Describing Tools and Approaches
                                                              As an example of approach
                                                              which provides easier access
                                                              to testing tools, see the
                                                              ,tools approach deployed
                                                              on UKOLN Web site.
                                                              This approach:
                                                                • Applies to all resources on
                                                                  Web site
                                                                • Covers HTML and CSS
                                                                  validation and various
                                                                  other tests
                                                                • Some recursive apps
     Approach is documented at                                    (,rvalidate)
     <http://www.ukoln.ac.uk/qa-focus/                          • Easily implemented with
                                                                  single line redirect
     documents/papers/iadis-2003/>
          QA Focus – Supporting JISC's Digital Library Programmes
24
     Next Steps   Next Steps
                  Extended Coverage
                   We are addressing other areas besides the Web (and
                   digitisation) including:
                      Metadata                                  Software development
                      Deployment into service
                  Moving On From Automated Testing
                   The initial work made use of automated testing tools:
                      Can be used remotely                       Objective
                      Applicable across all projects
                   We have started work on QA procedures in areas
                   which are not suitable for automated checking


                  QA Focus – Supporting JISC's Digital Library Programmes
25
     Next Steps   Self Assessment Toolkit
                  Further Deliverables
                  We will be developing a self-assessment toolkit for
                  projects to use, by individual projects or across project
                  clusters
                  The toolkit will consist of:
                     • Examples of QA procedures
                     • Documented examples of use of testing tools
                     • Self-assessment questionnaires
                     • Advice on standards and best practices
                     • Case studies
                     • FAQs
                     • …
                  QA Focus – Supporting JISC's Digital Library Programmes
26
     Next Steps   Self Assessment Toolkit




                  QA Focus – Supporting JISC's Digital Library Programmes
27
     Next Steps   Next Steps – Metadata
                  Thoughts on QA for metadata:
                  Syntax              Check syntax on embedded DC
                  Cataloguing Rules Ensure projects have appropriate
                                      cataloguing rules
                  Interoperability    Ensure metadata can
                                      interoperate with third parties
                  Fitness for purpose Ensure metadata is appropriate
                                      for its purpose
                  Change control      Ensure architecture for managing
                                      metadata can cope with change
                                      control (cf Exploit Case Study)



                  QA Focus – Supporting JISC's Digital Library Programmes
28
     Next Steps   Next Steps – Software
                  Thoughts on QA for software:
                  Specification    Process for developing systems spec,
                                   agreeing spec with stakeholders, etc.
                  Methodology      Documentation of systems
                                   development methodology (cf UML
                                   case study)
                  Testing          Documentation of testing
                                   methodology
                  Deployment       Awareness of deployment challenges



                   See for example <http://www.ukoln.ac.uk/qa-
                   focus/documents/briefings/#software>
                  QA Focus – Supporting JISC's Digital Library Programmes
29
      Next Steps   Next Steps – Service Deployment
                   Thoughts on QA for service deployment:
                   Development Architecture
                                Early public documentation of
                                development architecture
                   Service Architecture
                                Awareness of likely service architecture
                                and liaison with service
                   Security     Awareness of security concerns
                   Licensing    Licensing issues
     Scenarios
     Software developed using cool open software environment – but
     JISC Service has no knowledge of environment.
     Resources digitised, but copyright clearance not documented.
30
     ServiceQA Focus – Supporting JISC's Digital Library Programmes concerns.
             refuses to take delivery due to liability
     Next Steps   QA Focus And You
                  What relevance has QA Focus for you?
                    • You can write case studies and advisory
                      documents:
                        Seen to be working with JISC
                        We can promote your approaches (and
                         therefore your work)
                        This will help instigate best practices within
                         your group




                  QA Focus – Supporting JISC's Digital Library Programmes
31
      Next Steps   QA Focus And You
                   What relevance has QA Focus for you?
                    • You can deploy QA practices for your projects, etc:
                         You should benefit
                         You can provide feedback to QA Focus
                    • You can develop QA procedures in innovative areas
                      (e.g. OAI, thesaurii, ePrints, etc.) which QA Focus
                      can use:
                         You gain kudos, publications, etc.
                         We can help embed your practices across the
                          community
     e.g. see joint UKOLN/CDLR paper on "Deployment Of Quality Assurance
     Procedures For Digital Library Programmes" at
     <http://www.ukoln.ac.uk/qa-focus/documents/papers/iadis-2003/>
             QA Focus – Supporting JISC's Digital Library Programmes
32
     Next Steps   Beyond Advice – Implementing QA
                  QA Focus is providing a wide range of advice
                  for digital library programmes
                  However there is also a need for projects
                  themselves to be proactive in implementing
                  QA procedures
                  What approaches can projects take, bearing in
                  mind issues such as resources, existing
                  procedures, etc.?




                  QA Focus – Supporting JISC's Digital Library Programmes
33
     QA Focus Methodology   The QA Focus Methodology
                            The QA Focus methodology which we are
                            advising projects (and JISC) to deploy:
                              • Based on well-established QA principles
                              • Seeks to be lightweight and achievable
                              • Seeks to provide benefits to the projects
                                themselves, as well as (downstream)
                                interoperability benefits to JISC programmes
                            Consists of:
                              •   Documented policies
                              •   Procedures for monitoring compliance with policies
                              •   Audit trails
                              •   Sharing and collaboration
                            QA Focus – Supporting JISC's Digital Library Programmes
34
     QA Focus Methodology   Applying QA – Standards
                            Policy:     Open standards used where possible.
                                        Proprietary formats with published spec and
                                        cross-platforms viewers and authoring tools
                                        may be acceptable
                            Monitoring: Project manager responsible
                            Documentation:
                                        Document on standards (cf QA Focus doc)
                            Exceptions: If open standards not available, immature
                                        or costly to implement proprietary standards
                                        may be used.
                                        The project manager should document
                                        such decisions: why proprietary format was
                                        used, migration strategies to open formats
                                        in future, architecture for
                                        migration and indication of costs
                            QA Focus – Supporting JISC's Digital Library Programmes
35
     QA Focus Methodology   Applying QA – Web Standards
                            Policy:       The Web site will use XHTML 1.0 and
                                          CSS 2.0 standards
                            Architecture: The Web site will be based on XHTML
                                          templates and SSI
                            Monitoring:   New and updated pages validated using
                                          ,validate and ,cssvalidate. Every
                                          month ,rvalidate will be used
                            Exceptions:           HTML derived automatically (e.g. Save As
                                                  HTML in PowerPoint) need not comply
                                                  with standards. The files will be stored in
                                                  a standard directory to enable such files
                                                  to be excluded from checks

                            QA Focus – Supporting JISC's Digital Library Programmes
36
     QA Focus Methodology   Applying QA – Proprietary Formats
                            Policy:               Proprietary formats may be deployed on
                                                  Web site in certain documented
                                                  circumstances including MS Office files,
                                                  PDF and Flash
                                                  MS Office files should also be stored in
                                                  most open HTML format.
                                                  PDF files should include link to online PDF
                                                  converter
                                                  Flash can be used for specific applications
                                                  in which it is needed.
                            Monitoring:           Project manager




                            QA Focus – Supporting JISC's Digital Library Programmes
37
     QA Focus Methodology   Applying QA – Accessibility (1)
                            Policy:       The Web site will strive to attain WAI A
                                          guidelines.
                                          Consistent accessibility shortcuts will be
                                          used.
                                          An accessibility policy will be published.
                            Architecture: The Web site will be based on XHTML
                                          templates which comply with WAI A.
                            Monitoring: New and updated pages will be validated
                                          using ,bobby. A monthly batch checker
                                          will be used and audit reports published (to
                                          enable any trends to be spotted).
                            Exceptions: A list of permitted exceptions will be
                                          provided.
                            Note that University of Edinburgh Library
                                                       accessibility policy
                            have a– useful WebDigital Library Programmes
                            QA Focus Supporting JISC's
38
     Applying QA – Accessibility (2)
     Policy:       The organisation has no
                   accessibility policy
                   Authors are free to implement their
                   own accessibility shortcuts (if at all)
     Architecture: No centralised policy covering
                   authoring tools or architecture will
                   be provided
     Monitoring: No monitoring will be carried out



     QA Focus – Supporting JISC's Digital Library Programmes
39
     QA Focus Methodology   Applying QA – Usability (1)
                            Policy:       A structured usability test will be applied
                                          using at least 5 users for new Web sites
                                          Tailored 404 error pages will be provided
                                          Use of <link rel="home" …> to
                                          support navigation across Web site
                                          Use of <link rel="next" …> to
                                          support navigation in structured areas of
                                          Web site
                            Architecture: The <link rel="next" …> elements
                                          should be implemented by automated
                                          scripts
                            Monitoring:   Need to check the next, previous goes to
                                          correct page.
                            Exceptions: …
                            QA Focus – Supporting JISC's Digital Library Programmes
40
     QA Focus Methodology   Applying QA – Usability (2)
                            Policy:               Broken links to be kept to a minimum
                            Monitoring:           The ,checklink command applied to
                                                  new and updated resources and
                                                  ,checklink run monthly
                                                  The Xenu link checker will be run monthly
                                                  and a report published, in order for any
                                                  trends to be monitored
                            Exceptions:           Don't fix broken links in published
                                                  documents or derived files.
                                                  Need to formulate policy on large
                                                  numbers of broken links
                                                  (cf JISC Web site)

                            QA Focus – Supporting JISC's Digital Library Programmes
41
     Recommendations
     QA Focus recommendations are likely to include:
      • Project proposals should describe approaches to
         selection of standards, technical architecture & QA
      • Once approved initial project report should document:
           Standards chosen (e.g. see "ESDS Web
             Standards Policy" case study and "Matrix for
             Selection of Standards" briefing document)
           Justification for any deviations from open
             standards (e.g. see NOF-TAS FAQ)
           Technical architecture
           Intended service deployment
             environment
     QA Focus – Supporting JISC's Digital Library Programmes
42
     Reality Check
     Is this really needed? The pages are fine in my
     browser.
     But:
        • What about repurposing resources (XHTML  RSS)?
        • What about new devices (using wireless PDA in
          student's union)?
        • What about accessibility?
        • What about tomorrow's maintenance nightmare?
        • What about the exploitation of XML resources
          (quicker, richer, …)?
        • What about the vision of universal access, and
          removal of platform and application dependencies?
     QA Focus – Supporting JISC's Digital Library Programmes
43
     Getting There
     I want to improve the quality of my Web services. But
     I've got a large Web site, investment in current systems,
     inertia, … So:
        • Deploy QA procedures in new developments
        • Use tools such as W3C's Log Validator to prioritise
          key areas - <http://www.w3.org/QA/Tools/>
       • A CMS can help
       • Consider use of Tidy on server (cf. LSE approach)
       • Ensure output from backend scripts is compliance
       • Revisit MS Office  Web tools
       • Plan migration strategy to XHTML 2 world
         (compliant XHTML 1.0  XSLT  XHTML 2.0 = )
     QA• Remember you already have QA procedures
       Focus – Supporting JISC's Digital Library Programmes
44
     Conclusions
     We have described:
       • The approaches to QA which have been taken by
         the QA Focus project
       • Findings of a number of surveys which have
         highlighted common problems
       • An example of an approach to providing access
         to testing tools using a URI-interface
       • A systematic methodology for QA
       • Recommendations for future programmes




     QA Focus – Supporting JISC's Digital Library Programmes
45
     Questions
     Any questions?




     QA Focus – Supporting JISC's Digital Library Programmes
46

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:8/5/2011
language:English
pages:46