Slides available at
Quality Assurance
For Your Web Site
Brian Kelly Email
UKOLN B.Kelly@ukoln.ac.uk
University of Bath URL
Bath http://www.ukoln.ac.uk/
UKOLN is supported by:
A centre of expertise in digital information management www.ukoln.ac.uk
About Me / QA Focus
Brian Kelly:
• UK Web Focus: adviser on Web standards and
best practices
• Funded by JISC (and MLA 1 Aug 2003)
• Web developer since 1993
• Based at UKOLN
QA Focus:
• Project funded by JISC to support JISC's digital
library programmes
• Has developed a quality assurance methodology
and range of support materials
• Provide by UKOLN and AHDS
• Project manager is Brian Kelly
A centre of expertise in digital information management www.ukoln.ac.uk
2
What Can Go Wrong? What Can Go Wrong?
Accessibility problems
Usability problems
Corrupted display
Incorrect content Broken links
Problems Broken forms
Out-of-date content
Broken scripts
Ambiguous content
Why Do Things
Incomprehensible content Go Wrong?
Device problems The problems
Interoperability problems may be due to:
• People
• Technologies
A centre of expertise in digital information management
• Systems
www.ukoln.ac.uk
3
What Can Go Wrong? Finding Errors
How do we spot such errors?
• Automated tools
• Manual checking
• User feedback
• Failure of systems to work correctly
• Failure of systems to be interoperable
• …
But:
• How systematic are we in checking?
• Do users really give us feedback?
• Do we know when are systems are non-
interoperable?
A centre of expertise in digital information management www.ukoln.ac.uk
4
What Can Go Wrong? Fixing Errors
If we spot errors or errors are reported what are our
approaches to correcting the errors:
• Fix them straight away
• Scope the extent of problems and make plans for
fixing problems
• Do nothing – there are too many errors to fix
• Do nothing – it's somebody else's responsibility
• Do nothing – it's a problem with the CMS, the
user's browser, …
• Something else
A centre of expertise in digital information management www.ukoln.ac.uk
5
What Can Go Wrong? Why Do Things Go Wrong?
Things can go wrong for several reasons:
• Failure to understand the bigger picture:
• Importance of open standards
• Limitations of open standards
• Use of an inappropriate for the deployment of
solutions
• Failure to check compliance with standards
• Failure to appreciate limitations of testing tools
• Failure to understand what should be tested
A centre of expertise in digital information management www.ukoln.ac.uk
6
Quality Assurance QA Focus Approach
QA Focus approach to these issues:
• Advice on reasons for use of open standards
• Advice on specific open standards
• Case studies describing approaches taken by
projects (including any problems experiences and
lessons learnt)
• Advice on approaches to testing
• Development of a quality assurance methodology
A centre of expertise in digital information management www.ukoln.ac.uk
7
Quality Assurance A QA Approach
Quality Control:
• Spotting errors and then fixing them
• CF production line processes (rejection of bottles,
cars, … which aren't up to scratch)
Quality Assurance:
• Having documented procedures
• Addressing the underlying causes of problems
• Fixing the workflow processes
• Addressing human issues (training, …)
• Introduced in the 1940s
A centre of expertise in digital information management www.ukoln.ac.uk
8
Quality Assurance We Need Policies
Quality Assurance requires documented policies:
• How do we know if something (non-trivial) is
broken if we haven't got a documented policy
• The policies should be realistic
In a Web context we need policies on:
• HTML compliance
• CSS
• Links
• Accessibility / usability
• Error reporting
• …
It is recognised that policies may need to be lightweight
centre of expertise onerous to management
Aand not too in digital informationdevelop. www.ukoln.ac.uk
9
Quality Assurance We Need Checking Procedures
Quality Assurance requires systematic procedures for
ensuring compliance with policies:
• Without this, our policies can be meaningless
'motherhood and apple pie' statements
The procedures:
• Should be systematic
• Should provide an audit trail
• Should result in action if deviations from policies
found
It is recognised that procedures may need to be
Alightweight and not too onerous to implement.
centre of expertise in digital information management www.ukoln.ac.uk
10
Quality Assurance A Template Policy
Area: Give the area covered by the policy
Standards / Best Practices: State the standards or
best practices which will be used
Justification: Give reasons for chosen standards /
best practices
Exceptions: State any permitted deviations
Implementation Architecture: If applicable, describe
the architecture used to implement the standards
Change Control: Describe the responsibilities for the
policy, its implementation and for making changes to
the policy.
A centre of expertise in digital information management www.ukoln.ac.uk
11
Quality Assurance A Template Procedure
Area: Give a link to the policy.
Procedure(s): Describe the procedure(s) used for
ensuring compliance with the policy.
Limitations: Describe any limitations in the
procedures.
Audit Trails: Describe any audit trails used to record
the findings of the procedures.
Correcting Errors : Describe the approaches for
correcting errors which may be found.
Change Control: Describe the responsibilities for the
procedures, its implementation and for making
changes to the procedures.
A centre of expertise in digital information management www.ukoln.ac.uk
12
Quality Assurance QA Areas
Areas in which QA Focus has been developing QA
policies and procedures and accompanying support
materials include:
• Web/access Digitisation
• Metadata Software
• Service Deployment Standards selection
A centre of expertise in digital information management www.ukoln.ac.uk
13
Quality Assurance QA For Web
QA for Web sites will cover areas such as:
• HTML compliance
• CSS compliance
• Functionality in Web browsers
• Link checking
• Accessibility checking
• Usability checking
• Accuracy of content
• …
A centre of expertise in digital information management www.ukoln.ac.uk
14
QA For Web (HTML/CSS) -
Quality Assurance
Policy
Area: Web standards (HTML, CSS).
Standards / Best Practices: Web site will comply with
XHTML 1.0 and CSS 2.0.
Justification: Compliance with HTML &CSS standards
with help to maximise access to Web site.
Exceptions: Files derived from MS Office apps need
not comply with HTML standard.
Implementation Architecture: PHP on Apache
platform used, which includes HTML fragments. Also
makes use of backend MS SQL Server database and
MT Blog.
Change Control: The project manager is responsible
for the policy, its implementation and for making
15
changes to the policy.
A centre of expertise in digital information management www.ukoln.ac.uk
QA For Web (HTML/CSS) -
Quality Assurance
Procedures
Area: Web standards (HTML, CSS).
Procedure(s): The ,validate tool should be used
when pages created/updated. The ,rvalidate tool
should be used at least quarterly. The W3C Web Log
validator should be used monthly.
Limitations: The ,rvalidate tool only validate up to
100 files. The W3C Web Log validator only validates
the 10 most popular pages.
Audit Trails: An audit trail will be kept of the output
from the monthly W3C Web Log validator output.
Correcting Errors : Errors spotted using ,validate and
,rvalidate should be updated immediately. A
record of pages fixed/not fixed should be kept for the
W3C of expertise in digital information output.
A centre Web Log validator management www.ukoln.ac.uk
16
Quality Assurance QA In Other Web Areas
CSS:
• Similar to HTML standards (see briefing doc)
Link-checking:
• Need systematic use of link-checkers.
• Need to ensure tools covers links other than
and e.g. links to CSS & JavaScript files.
• Need to have policy on fixing broken links.
Accessibility:
• Important to have QA to cover "reasonable
measures" clause in DDA.
• Will need automated and manual checks.
Usability:
• Related to accessibility.
• Will need automated and manual checks.
A centre of expertise in digital information management www.ukoln.ac.uk
17
Quality Assurance QA And Metadata
Metadata is the glue for interoperable services. It is
therefore important that we have QA to ensure that our
metadata is
• Accurate
• Represented in correct format
• Interoperable with other services
For further information see:
• An Introduction To Metadata (briefing 41)
• Metadata Deployment (briefing 42)
• Quality Assurance For Metadata (briefing 43)
See
A centre of expertise in digital information management www.ukoln.ac.uk
18
Quality Assurance QA And Software
Software may be:
• Used software to create, manage and deliver
resources on our Web site
• Purchased, developed locally or used as open
source
There is a need to:
• Ensure software is appropriate for its purpose
• Ensure we have resources needed to use /
develop software
• Ensure software outputs are compliance with
appropriate standards & guidelines
See
A centre of expertise in digital information management www.ukoln.ac.uk
19
Quality Assurance QA And Service Deployment
Project-funded work can help to develop content,
applications, etc. which will then be deployed in a
service environment.
There is a need to ensure that project deliverables:
• Can be deployed easily
• Are legal and unencumbered with IPR restrictions
• …
See
A centre of expertise in digital information management www.ukoln.ac.uk
20
Quality Assurance Deviation From Best Practices
QA is about “fitness for purpose” – not necessarily the
ideal solution.
The NOF-digitise Technical Advisory Service defined
a reporting process when non-optimal solutions (e.g.
proprietary formats like Flash) were to be deployed:
• Description of preferred open standard/best
practice
• Proposed solution
• Reason for choice of proposed solution
• Description of migration strategy
A NOF-TAS FAQ gives scenarios such as use of Flash
and use of externally-hosted Web services
A centre of expertise in digital information management www.ukoln.ac.uk
21
Quality Assurance Matrix For Standards Selection
The selection of formats to be used is not necessarily
easy. Open standards may be immature, costly to
deploy or fail to be widely deployed (cf. OSI networks)
We have developed a template matrix for selection:
Maturity of standard
Complexity
Availability of tools
Resource issues
Organisational culture
A centre of expertise in digital information management www.ukoln.ac.uk
22
Quality Assurance Useful Tools
A number of simple tools and techniques for checking
compliance have been documented:
,tools
• Append ,validate ,rvalidate ,checklink
etc. to any URL on UKOLN Web site
• Easy to implement – see ,tools
W3C QA Log Validator
• Periodic report on 10 most popular pages which
are non-compliant
• Means of prioritising pages to fix (and spotting
workflow problems and motivating page authors to
address problems)
• Simple Perl script
• See
A centre of expertise in digital information management www.ukoln.ac.uk
23
Quality Assurance QA In "Softer" Areas
There may be a temptation to address only the „hard„
areas with use of automated tools
It is equally important to address „softer‟ areas such as
accessibility, usability, content, functionality, etc.
(cf. DRC Accessibility Report)
How can QA be used in these areas:
• Still a need for policies
• Testing compliance cannot be done with
automated tools
• See Alice Grant‟s report on approaches to
evaluation – to be published on MLA Web site
shortly
• Sarah Agarwal's talk on usability testing
A centre of expertise in digital information management www.ukoln.ac.uk
24
Quality Assurance Embedding QA In Your Library
QA Focus resources:
• Developed for JISC digital library community
• Looking to extend remit to include MLA sector
• You can help by providing feedback on:
• Existing resources
• QA methodology
• What‟s missing
•…
Please complete feedback forms and return
A centre of expertise in digital information management www.ukoln.ac.uk
25
Conclusions Conclusions
To conclude:
• Web sites now provide mission-critical services
• Robustness and reliability are therefore crucial
• We could react to problems
• A better approach is use of well-established
quality assurance principles
• QA need not be onerous to introduce
• QA Focus have developed a methodology and
accompanying materials which are freely
available
A centre of expertise in digital information management www.ukoln.ac.uk
26