OPAL Online Recording Toolbox by pengxiuhui

VIEWS: 21 PAGES: 35

									         Centre for Ecology and Hydrology


         OPAL
         Online Recording Toolbox
         Requirements Specification




         Biodiverse IT
           th
         16 October 2008




Online Recording Toolbox Requirements – CEH   1 of 35
TABLE OF CONTENTS

Table of Contents ..................................................................................................................................... 2
Document History..................................................................................................................................... 4
Glossary .................................................................................................................................................... 5
Introduction .............................................................................................................................................. 6
Demonstration Site Requirements ........................................................................................................... 7
   High Level Requirements ...................................................................................................................... 7
   User Login and Management Requirements ........................................................................................ 7
   Data Entry Requirements ..................................................................................................................... 8
   Non-Occurrence Data Requirements ................................................................................................... 9
   Data Flow Requirements .................................................................................................................... 10
   Mapping Requirements ...................................................................................................................... 11
   Report Requirements ......................................................................................................................... 11
   Administration Requirements ............................................................................................................ 12
Technical Requirements ......................................................................................................................... 13
   Architectural Approach ...................................................................................................................... 13
       Friendly URLs .................................................................................................................................. 14
   Database Design ................................................................................................................................. 14
       Spatial Reference Attributes........................................................................................................... 15
       Specification of Custom Attributes ................................................................................................. 15
Core Module ........................................................................................................................................... 17
   Architecture ........................................................................................................................................ 17
   Lookup Services .................................................................................................................................. 17
       Custom Term Lists .......................................................................................................................... 18
       Lookup Services Functionality ........................................................................................................ 19
   Record Services................................................................................................................................... 20
   Validation Services ............................................................................................................................. 20
   Helper Services ................................................................................................................................... 21
   Security Services ................................................................................................................................. 21
   Report Services ................................................................................................................................... 22
   User Interface ..................................................................................................................................... 22
Site Module ............................................................................................................................................ 24
   Customisation and Configuration Approaches ................................................................................... 24
   Components ....................................................................................................................................... 25
       Composite Controls ........................................................................................................................ 25



Online Recording Toolbox Requirements – CEH                                      2 of 35
       Data Entry Controls ........................................................................................................................ 27
GeoServer ............................................................................................................................................... 32
Appendices ............................................................................................................................................. 33
   Appendix 1 – Vague Dates .................................................................................................................. 33
   Appendix 2 – Spatial Reference and Georeference Formats .............................................................. 34
       British National Grid Reference System ......................................................................................... 34
       Irish Grid System ............................................................................................................................. 34
       Latitude and Longitude WGS84 ...................................................................................................... 34
       Postcodes ....................................................................................................................................... 34
       Additional Systems ......................................................................................................................... 35




Online Recording Toolbox Requirements – CEH                                    3 of 35
DOCUMENT HISTORY

Version           Date                 Description
1.0               07/10/2008           Initial Draft
1.1               16/10/2008           Revised after comments from David Roy and Steve
                                       Wilkinson




Online Recording Toolbox Requirements – CEH   4 of 35
GLOSSARY

Term                    Definition
AJAX                    Asynchronous JavaScript And Xml. A scripting technique that allows web
                        pages to interact with the server and update parts of the page without
                        reloading the entire page content.
CMS                     Content Management System, a tool for managing the pages and content
                        present on a website.
CSS                     Cascading Style Sheets
Core Module             The software module which must be installed to provide services which
                        support one or more online recording sites. The Core Module is able to
                        support multiple websites (Site Modules).
Core Host               An organization hosting the Core Module software.
OGC                     The Open Geospatial Consortium is an organisation that leads the
                        development of standards for geospatial and location based services.
LRC                     Local Record Centre
NBN                     National Biodiversity Network
Site Module             The software module which must be installed in order to provide an online
                        recording website application. The Site Module consumes services
                        provided by the Core Module in order to provide the website functionality.
Site Host               An organization hosting an online recording site via the Site Module
                        software.
SQL                     Structured Query Language, a language that provides access to the data
                        and facilities of a database.
Validation              The process of checking that data entered is sensible and reasonable but
                        does not check the accuracy of the data.
Verification            The process of checking the accuracy of entered data.
WGS84                   World Geodetic System 1984, a worldwide datum used by the international
                        GPS system to provide latitude and longitude coordinates.
WFS                     Web Feature Service – a standard for implementing web services that
                        accept requests for geographic features. For example, a WFS server might
                        provide the distribution points (features) to be drawn on a distribution
                        map. WFS also allows updates to the features to be accepted though this
                        facility is not part of the specification of this project.
WMS                     Web Map Service – a standard for implementing web services that provide
                        map images from geospatial datasources. For example, a WMS server
                        might provide the background map images on which a distribution map is
                        drawn.
WKT                     Well Known Text is a simple text mark-up language used to represent the
                        geometry of objects on a map. The WKT format is a widely accepted
                        standard that is regulated by the Open Geospatial Consortium (OGC).
                        Examples include:
                                   POINT(-0.842 50.46)
                                   POLYGON((-0.842 50.46,-0.822 50.46,-0.822 50.26,-0.842 50.46))
                        The full Well Known Text specification is available at
                        http://www.opengeospatial.org/standards/sfa though the parts of the
                        specification required for the implementation of this project are likely to
                        only include geometry definitions.




Online Recording Toolbox Requirements – CEH      5 of 35
INTRODUCTION

There is currently rapidly rising interest across both LRC’s and National Schemes and Societies in using
the internet to both capture and deliver species information. In particular the recent successes of on-
line capture projects, for example BirdTrack (British Trust for Ornithology) and the Harlequin Ladybird
Survey (http://www.harlequin-survey.org/), have been testament to the potential of uptake and
improved efficiency of data capture through this route.

In parallel with this the NBN has invested heavily in developing a suite of web services to allow users
to create customised outputs from the NBN Gateway on their own websites. Adoption of these across
the community has been relatively limited primarily due to the fact that the implementation of these
requires a reasonable level of technical knowledge and skill.

The overall requirement is to facilitate the development of customised on-line data entry for species
occurrence records by provision of a suite of programming tools as well as a complete online
recording example website which is based on and demonstrates the tools. Participating organisations
are able to use the demonstration website with no customisation, customise the site by changing
templates, CSS and images or can develop a new site from scratch using the provided tools. To ensure
the successful take up of the project a key feature is ease of use of all aspects, including the
demonstration user interfaces for online record entry, the installation process for the example
website as well as the programming tools provided. In addition to providing online data entry
facilities, the scope of the project includes any feature that may encourage submission of further
data. Reporting and atlas functionality are provided for this purpose with the ability for an end user to
view their contributed records against the national distribution map of a species being a feature that
encourages the user to continue to enter their records. However, the project is focused on data
capture and does not aim to replicate the functionality of a fully fledged recording package. Therefore
the ability to easily transfer captured records into other recording packages is also a key requirement.




Online Recording Toolbox Requirements – CEH          6 of 35
DEMONSTRATION SITE REQUIREMENTS

A key part of the project is the development of a demonstration online recording website based on
the tools provided. This serves a number of purposes:

        To provide examples and a template from which new sites can be developed
        To provide a test framework for the tools and technologies developed as part of the project
        To provide a default site installation that can be used directly or with minimal customisation
         for organisations without the capability to develop their own site.

This section describes the requirements of the demonstration website, and in doing so defines the
minimum possibilities for applications developed using the toolkit.

HIGH LEVEL REQUIREMENTS
Description                                                                                       Required
Allow quick and accurate entry of occurrence records through an on-line interface.                Mandatory
Provide feedback mechanisms including reports and distribution maps in order to                   Mandatory
encourage submission of further records from users.
Allow captured data to be downloaded for incorporation into other biological recording            Mandatory
systems’ datasets.
Ensure the degree of configuration and basic functionality is sufficient to meet the              Mandatory
needs of the majority of species surveillance schemes, national schemes and local
record centres.
Allow the user to enter details of additional types of data required to support the               Mandatory
occurrence records, including locations, samples, sample types, people, taxa and
sex/stage terms.
Compose the user interface of re-useable components that allow easy customisation or              Mandatory
building of entirely new online recording websites within the same framework.
To allow further extensions to the tools by more experienced developers; this can then            Mandatory
be shared with other implementations of the tool kit.



USER LOGIN AND MANAGEMENT REQUIREMENTS
Description                                                                                       Required
Provide a username and password based login facility. Site visitors are directed to this          Mandatory
page when not logged in. Links are available to the user registration page, forgotten
password request facility and an option is available to remember the user’s login
credentials the next time they visit the site.
Provide a user registration page that captures the user’s name, username, password                Mandatory
and email address.
Provide optional new user registration verification through an email confirmation                 Mandatory
message with a link that must be followed to confirm the new user.
Provide optional new user registration verification through a CAPTCHA code entry                  Desirable
facility. For more information, see http://en.wikipedia.org/wiki/Captcha.
There are three user access levels, user, editor and site administrator.                          Mandatory
All users can access a profile page allowing them to update their own details.                    Mandatory
Site administrators can access a list of all users with a search facility and ability to filter   Mandatory
for users awaiting verification.
Site administrators can access the profile page for any user. When viewed as an                   Mandatory
administrator, there is an option to resend the username and password by email to the



Online Recording Toolbox Requirements – CEH            7 of 35
user owning the profile in the case that they have forgotten them.
Site administrators can ban any user by email including wildcards, username or IP            Mandatory
address. For example, it is possible to ban users at a domain by banning
*@domain.com.
Site administrators can delete any user. This is either:                                     Desirable

    a) A mark delete operation (the user record remains but the login is deactivated,
       similar to banning)
    b) A true delete operation (this forces all their records to be also deleted).



DATA ENTRY REQUIREMENTS
Description                                                                                  Required
A home page provides links to other site pages. The links present depend on the user’s       Mandatory
access level.
Data entry card 1. Provide a screen that allows data entry of a single ad-hoc species        Mandatory
occurrence. This has the following fields on it for the demonstration application:
         Reference Number – text box for user’s specified record reference number
         Species – auto-complete text box
         Date or Vague Date – date picker and text box
         Location – auto-complete text box (optional, but one of Location, Location
         description or Spatial Reference/Georeference must be supplied)
         Location description – text box
         Spatial Reference or Georeference – text box for validated entry with a Spatial
         Ref Picker control – a map control allowing the user to click on the map to
         identify the location of an occurrence more precisely.
         Recorder – auto-complete text box allowing multiple recorders to be specified
         from the list of existing names in the People list.
         Comment – text area (optional)
         Sex/Stage – auto-complete text box (optional)
         Abundance – text box (optional)
         Sample Method – auto-complete text box
         Photo – file upload text box (optional)
Data entry card 2. Provide a screen that allows entry of a list of species that were         Mandatory
recorded in the same sample with same data entry controls as data entry card 1, except
for the Species, Sex/Stage and Abundance text boxes. These are replaced by a table
containing 4 columns, a checkbox, a species name label, an abundance text box and a
sex-stage text box with auto-complete. There is a single row in the grid per species
available for the card. In addition, a text box with auto-complete is available and linked
to the entire species list which allows the user to add new species to the card so that
species not initially on the card can still be recorded.
For data entry card 2, the site administrator is able to define new cards from subsets of    Mandatory
the species list and the user is able to select which card to use when they access the
screen, and therefore which species are available in the list. For example the user may
be able to choose from a Woodland Birds or Heathland Birds data entry card.
Data entry card 3. Provide a screen that allows entry of a list of species that were         Optional
recorded in the same sample with same data entry controls as data entry card 1, except
for the Species, Sex/Stage and Abundance text boxes. In this case, they are replaced by
a single multi-line text area control. As the user types a species name, the entered text
is matched against the species list or a subset and a list of suggestions is displayed
beneath the text area. When the user has selected the species, they are able to type a
comma or other configurable separator and then type another species name, thus
building up a simple list of species in the sample.
Sex/stage values are controlled by a termlist in the Lookup Services module.                 Mandatory
Sample Method values are controlled by a termlist in the Lookup Services module.             Mandatory


Online Recording Toolbox Requirements – CEH          8 of 35
Support data entry of spatial references using British or Irish National Grid or Latitude       Mandatory
and Longitude WGS84 systems and also support georeferencing using Postcodes,
ignoring the last 2 digits of the postcode if it is not possible to do this without incurring
a license fee for the copyright on the postcode list.
Support data entry of spatial references using Universal Transverse Mercator                    Desirable
coordinates.
Provide an ability for the user to browse the data entry cards they have created and to         Mandatory
re-access the cards allowing data to be updated. Data is viewed using the same data
entry card it was entered in. The list of cards can be sorted by name, location, date or by
date of comment to bring cards with recent comments to the top.
Provide a list of all occurrence records which is paginated and can be filtered by taxon        Mandatory
group, date or location. This list provides links to the original card used to enter the
data.
When viewing a data entry card that was entered by a different user, only editors and           Mandatory
administrators have edit rights over the card unless the records have been downloaded
and the top copy is held elsewhere in which case the card is read only.



NON-OCCURRENCE DATA REQUIREMENTS
Description                                                                                     Required
Non-occurrence data exists in the following standard data entities:                             Mandatory
          Taxon
          TaxonGroup
          Sample
          SampleRecorder
          Location
          Person
Users are able to browse and search the available data in any of the standard data              Mandatory
entity master lists, any global subset of the master lists, or any subset of the list which
is local to the site. For example, a user might be offered the following options when
selecting a list to browse from:
Master Species List
          Vascular Plants list (global subset)
          Dorset Heathland Plants (local subset)
Master Locations List
          Dorset Heathland Sites (local subset)
When viewing the content of a list, the list is displayed in a Data Grid or Hierarchy           Mandatory
control (for species and locations).
When viewing the content of a list, the user can apply a simple text filter to the list.        Desirable
Administrators are able to add, modify and delete records from any local subset of the          Mandatory
above lists. Addition of new records can be achieved by identifying an entry from the
master list copy to include in the list subset. Alternatively, a completely new entry can
be created in the list subset. In this case, facilities are provided so that if the entry is
subsequently added to the master list copy, the administrator is able to link their local
entry to the master list entry.
Administrators are able to browse, search, add, modify or delete the available data in          Mandatory
any custom termlist created for this site.
In addition to the standard data entities, custom termlists contain controlled lists of         Mandatory
words that can be used to limit the values that can be entered for any attribute in the
system. For example, there are global Sex\Stages and Sample Methods termlists to
control the data values entered in their respective fields.
Like the standard data entities, local subsets of custom termlists can be created by            Mandatory
administrators at the site level.
Entirely new custom termlists can be created and subsetted by administrators at the



Online Recording Toolbox Requirements – CEH           9 of 35
site level.
Each of the standard data entities allows additional custom attributes to be declared by         Mandatory
the site administrator. A custom attribute is defined by selecting a name, a data type
and data entry control type, basic validation rules (minima and maxima or regular
expressions) and optionally selecting a custom termlist to control the list of possible
values.
When controlling the list of possible values for a custom attribute, the site                    Desirable
administrator can select from the following list of control methods:
              1. Value must be selected from the list, ID from list is stored in the
                    database attribute.
              2. Value must be selected from the list, but the term from the list is
                    stored in the database attribute.
              3. Value must be selected from the list, ID from list is stored in the
                    database attribute. User can type in new terms and they are
                    automatically added to the termlist. In this case, the administrator
                    defines the user roles which are able to add new terms.
              4. List is just suggested values, user is free to type in a new term. The
                    term is stored in the database attribute.
Custom attributes automatically appear on the relevant details pages for the associated          Mandatory
standard data entity. For example, if an altitude attribute is associated with the
Location entity, then when a Location is being edited the system automatically
generates an Altitude data entry control.
Administrators are able to use the data browsing screens to define subsets of each list          Mandatory
of data that are named and available from this site. For example they are able to create
a local species list from the master species list, or a local sites list from the master sites
list.
There is a facility to allow simple import of CSV or similar data into any site specific list    Mandatory
subset or custom list including occurrences and the standard entities.
When importing data, the user is able to map columns in the uploaded import file to              Mandatory.
the attributes of the entity being imported into.



DATA FLOW REQUIREMENTS
The following list of requirements describes the flow of data with validation and verification
processes.

Description                                                                                      Required
Occurrence records entered on the site are initially flagged as “in progress”. In progress       Mandatory
records are not visible to any other users but remain editable by the user owning the
record.
Once the user is happy with a record, they are able to flag it as “complete”. They then          Mandatory
become visible to other system users but remain editable by the user owning the
record.
The site administrator is able to configure whether basic data format validation occurs          Desirable
at the time of data entry or at the time a record is marked as complete.
Users are able to add comments to any record that they are able to see in the system. A          Mandatory
comment contains the following attributes:
          User
          Date
          Comment Text
          Type (comment, confirmation, considered incorrect, questionable).
The commenting process is considered analogous to commenting on a typical blog
system.
Comments can also be attached at the data entry card (sample) level.                             Desirable



Online Recording Toolbox Requirements – CEH            10 of 35
Any external process that verifies the data is able to add comments for each record           Mandatory
confirming it or marking it as incorrect. The comment text contains details of the
process run.
Site administrators are able to download selected occurrence records as simple flat files     Mandatory
for import into other systems.
When downloading a set of records, the administrator is able to select if the copy of the     Mandatory
record on the site remains the top copy, or if the top copy is being transferred to
another system. If the latter then the administrator provides details of a contact that is
stored against the record copy.
For records where the top copy has been transferred to another system, the user is no         Mandatory
longer able to edit the record but they are able to comment on the record.
If a record is commented when the top copy is held elsewhere, then a notification (e.g.       Desirable
by email) is sent to the contact identified when the record was downloaded.
Downloaded records include a GUID Globally Unique IDentifier) allowing the record to          Mandatory
be uniquely identified in future.
If additional record verification is carried out on the downloaded top copy (e.g. if a        Mandatory
record is confirmed by a National Scheme) then a simple CSV upload facility allows the
verification information to be flagged against the record as a comment. The CSV file
contains the record GUID, the comment text and type (confirmed or considered
incorrect).



MAPPING REQUIREMENTS
Description                                                                                   Required
Basic distribution map functionality is provided in order to encourage record                 Mandatory
submission. It is not a replacement for a full GIS system.
When the user browses to the details of any species, they are able to draw a                  Mandatory
distribution map of the species plotted on a Google Map.
When the user browses to the details of any species, they are able to draw a                  Desirable
distribution map of the species overlaid on a map generated by the NBN Gateway.
The results of any report that contains spatial information can be plotted on a               Mandatory
distribution map.
Points on a distribution map contain basic underlying record data in a popup, and             Desirable
provide a hyperlink to the details page for the underlying record.



REPORT REQUIREMENTS
Description                                                                                   Required
The user is able to browse data in the standard entities such as occurrences, taxa or         Mandatory
people. On selecting an item, a list of reports available for that item can be accessed.
The reports are in effect SQL queries with a parameter identified to allow them to be
called from the chosen record.
The results of a report are displayed in a reports output grid using the Data Grid control,   Mandatory
providing pagination, print and CSV download facilities.
Reports can be added to the system by the administrator. A report definition declares         Mandatory
which user roles are able to operate the report. Report definitions can declare optional
sort orders which the user selects from when running the report.
Reports that are available for the Person entity can be quickly accessed for the logged in    Desirable
user’s own data from their home page.


The following list of reports is provided with the initial installation of the toolkit:




Online Recording Toolbox Requirements – CEH             11 of 35
Report                      Description
Observations by Person      A report listing observations (species, date, location, spatial reference) for
                            a person is provided.
                            Each row provides a hyperlink to the data entry card page used to enter
                            the underlying record.
My Observations        of   For a selected species, lists all observations recorded by the logged in
Selected Species            user, ordered by date by default.
                            Each row provides a hyperlink to the data entry card page used to enter
                            the underlying record.
My Samples                  Spatial Reference, Location and Date for the list of samples entered by the
                            logged in user.
                            Each row provides a hyperlink to the details page for the underlying
                            sample.
Observations          for   Sort options include Species then Date or Date then Species.
Selected Location
My Observations       for   As the Observations for Selected Location but only displays observations
Selected Location           recorded by the logged in user.

ADMINISTRATION REQUIREMENTS
Description                                                                                    Required
Administrators are able to define the level of access that Guests have to the site, selected   Desirable
from the following:
          Guests have no site access.
          Guests can view data.
          Guests can view and comment on data, with an option to enforce CAPTCHA
          code entry when a guest posts a comment against a record.
          Guests can enter data
Data entered by guests defaults to being owned by the site administrator for subsequent
editing purposes.
Administrators are able to define if login is achieved by username or email.                   Desirable
Administrators are able to define which of the following control mechanisms are in place       Desirable
for verification of new user accounts:
          CAPTCHA codes
          Activation by confirmation email
          Require administrator to manually activate each new user
          Allow new users to post records, but not to mark them as complete so that they
          are not available to other system users until their account has been activated by
          the administrator.
Administrators are able to define the default zoom scale and centre point for distribution     Desirable.
maps and spatial ref picker controls. Alternatively the administrator is able to define that
maps default to being centred on the user’s home address if know.
Administrators are able to list in-progress records which are older than a user-specified      Desirable
date (e.g. 6 months). They have the option to amend the in-progress records and post
them so they are no longer in-progress, or to delete them.




Online Recording Toolbox Requirements – CEH         12 of 35
TECHNICAL REQUIREMENTS

ARCHITECTURAL APPROACH
A key consideration in the project is making it as simple and cost effective as possible to host online
recording websites. This requirement is in contrast with the complex, disk and processor intensive
nature of hosting a full online biological recording system due to the size of species lists and the
complex nature of the system. For this reason, the solution is split into two modules, a Core Module
and a Site Module. The Core Module is responsible for providing service interfaces that allow data to
be persisted and read, as well as validation, verification and other tasks required for a biological
recording website. The Site Module is responsible for presentation of web pages to the user,
capturing data from the user and submitting it to the Core Module for persistence and other site
specific tasks. Generally, the Site Module’s complexity is minimised making it easier to develop new
sites and easier and cheaper to host them. Note that the Site Module may be hosted on the same
facility as the Core Module allowing sites to be hosted for organisations that have no hosting capacity.
Alternatively, the Core Module can be hosted by the website provider allowing them to take complete
control of the entire system.

The Site Module itself is an extension to an Open Source Content Management System, providing site
administrators with the ability to extend and customise their site and content using the CMS’s built in
functionality. The choice of CMS has not been made yet and is pending architectural decisions by the
Natural History Museum’s own web development team, since there is a clear benefit to cooperation
between the 2 projects.

                                                       Service Provider



                                            Database




                                                                          Site Module
                                           Core Module
                                                                           (Optional)




                                  Server                    Server


         Website Providers, for
         example LRCs and
         Wildlife Trusts               Site Module               Site Module




         Data contributors,
         including general
         public



                                    FIGURE 1 - ARCHITECTURAL OVERVIEW

The Opal Online Recording application provides the following deliverables:




Online Recording Toolbox Requirements – CEH                13 of 35
        A core software module which provides services such as data storage, taxonomic and other
         lookup facilities and other web services as an API to online recording websites.
        A site software module which can be installed “out of the box” to provide a basic but
         functional online recording website, making use of services provided by the core module.
         The site software module’s code is provided so that the website can be fully customized or
         re-written from scratch.
        A suite of fully documented development components and tools that can be used to
         construct new online recording websites and that are used to provide the functionality
         within the standard site software module.
        Easy to use installation kits for each part of the system.

FRIENDLY URLS
Using “friendly URLs” to access each page of the site allow data item details pages to be accessed by a
reproducible and predictable URL. This facilitates external links and also the construction of links to
underlying pages from internal reports. For example, an occurrence record with internal ID=17 might
be accessed through a URL http://www.mysite.com/occurrences?id=17 or using a restful URL such as
URL http://www.mysite.com/occurrences/17.


DATABASE DESIGN
This document does not aim to finalise the database design but to clarify the requirements by
describing key entities and attributes in the model. All database records include basic metadata
attributes that identify the person who entered the record, entry date, the person who last changed
the record and the date of last change. The following entities are envisaged:

Entity                Description and Attributes
Occurrence            Identifies a single species occurrence record. Attributes include:
                           Taxon – identifies the recorded taxon from the Taxon entity.
                                (searchable)
                           Determiner - identifies the person who determined the taxon from the
                                Person entity. (optional, searchable)
                           Sex/Stage – a foreign key to an item selected from a custom list,
                                identifies the sex or stage of the observed species. (optional)
                           Abundance – free text, allows the count or other description of
                                abundance to be provided. (optional)
                           Photo – a path to the uploaded image if present. (optional)
                           Photo Caption – free text caption for the photo.
Sample                Identifies attributes common to a group of occurrence records that form the
                      same sample. Attributes include:
                           Date – a vague or precise date. (searchable)
                           Spatial Info – free text, represents the spatial reference or
                                georeference as entered. (optional, searchable)
                           Spatial Reference System – system used for the entered reference.
                                (optional)
                           Geometry – a spatial data type storing the point or polygon of the
                                sample as decimal latitude and longitude using the WGS84 datum.
                                (optional)
                           Location – link to the Location entity. (optional, searchable)
                           Location Name – free text description of the locality. (optional,
                                searchable)
                           Sample Method - a foreign key to an item selected from a custom list,
                                identifies the method of sampling (e.g. quadrat, pitfall trap, field
                                observation). ( searchable)



Online Recording Toolbox Requirements – CEH        14 of 35
SampleRecorder        Identifies a single person responsible for a sample. A sample has at least one
                      person associated with it through this entity. Attributes include:
                            Sample – link to the Sample entity
                            Person – link to the Person entity.
Taxon Group           Identifies a taxonomic group, used to add a simple descriptive label to each
                      taxon for clarification of names particularly where names clash. Attributes
                      include:
                            Name – name of the taxonomic group. (searchable)
Taxon                 Identifies a single species or other taxon. A taxon entry has the following
                      attributes.
                            Name (searchable)
                            Authority (optional)
                            Code (optional, searchable)
                            Taxonomic order code
                            Common name (optional, searchable)
                            Image (optional) – an uploaded image file
                            Taxon Group (optional) - link to an entry in the Taxon Group list.
                            Parent – link to a parent Taxon in the hierarchy
                            Description
                            Taxon Version Key (optional) - the 16 character unique identity
                                allowing the species to be uniquely identified on the NBN Gateway.
                      Note that in the special case of the Taxon entity, administrators cannot update
                      the name, authority and taxon version key for any taxa which have a taxon
                      version key (i.e. they were imported from the official NHM species lists). For
                      these taxa, there is an option to refresh the content using a CSV file import
                      (described in the section Non-Occurrence Data Requirements).
Person                Identifies a single person. Attributes include:
                            Firstname (searchable, optional)
                            Surname (searchable)
                            Email address - hidden unless user has set profile option to allow their
                                email address to be displayed (searchable, optional).
Location              Identifies a single site or other type of location (e.g. town or administrative
                      boundary). Attributes include:
                            Name – free text name of the site. (searchable)
                            Code – free text site code or reference number. (optional, searchable)
                            Spatial Reference – represents the spatial reference as entered
                                (searchable)
                            Spatial Reference System – system used for the entered reference.
                            Geometry – a spatial data type storing the point or polygon of the
                                location as decimal latitude and longitude using the WGS84 datum.
                            Parent site – an optional link to the parent location in the hierarchy.
                                (optional)


SPATIAL REFERENCE ATTRIBUTES
Spatial coordinates are stored in a spatially enabled database using the OGC GEOMETRY data type,
allowing points, lines and polygons (e.g. grid squares) to be stored. The coordinates stored use
WGS84 decimal latitude and longitude values. This architecture allows future expansion of the system
to include storage of boundaries associated with occurrence records though this is not a current
requirement.

SPECIFICATION OF CUSTOM ATTRIBUTES
As well as the standard attributes for each of the standard entities in the system, the site
administrator is able to configure additional custom attributes to store against each species record in



Online Recording Toolbox Requirements – CEH        15 of 35
the Occurrence, Location or Sample table, the primary data capture tables. These custom attributes
are available at the site level. A custom attribute has a name and data type specified by the
administrator of the site. In addition there is a flag that can be set to indicate multiple values are
allowed for this custom attribute. The following list describes the possible data types:

Data Type                   Description
Text                        Allows text values to be captured for an attribute. The allowable values
                            can be controlled by one of the following means:
                                   specifying a lookup list as the list of allowable values for a the
                                      attribute (see the section Lookup Services for more
                                      information).
                                   by providing a regular expression which defines the allowable
                                      format of the entered text. For example, the regular expression
                                      “[SACFOR]” allows only the characters S, A, C, F, O or R to be
                                      specified as a field value. For more information, see
                                      http://en.wikipedia.org/wiki/Regular_expression.
                            The attribute also defines the maximum allowable number of characters
                            in the text values.
Number                      Allows a whole number or floating point number to be captured as an
                            attribute value. Minimum and maximum allowable values can be
                            optionally specified.
Date                        Allows precise dates to be captured. Attributes of this type allow a
                            minimum and maximum value to be specified, which can include the
                            variable TODAY. For example, a date can be validated to not be allowed
                            in the future by specifying the maximum value as TODAY.
Vague Date                  For dates which may not be precisely known, a vague date can be
                            specified which is a text format that describes the date range, for
                            example “Summer 1995”. A vague date can be decoded into a start date
                            and end date for the range of possible values. For more information, see
                            Appendix 1 – Vague Dates.
                            Vague dates also allow minima and maxima to be specified in the same
                            fashion as dates.
Item from Lookup            Allows an attribute to be defined which only allows values from a lookup
                            list to be used. The administrator defines which lookup list or subset is
                            used when they create the attribute.
                            This differs from a text value as the database stores the ID of the lookup
                            list entry rather than the text. Subsequent updates to the lookup list
                            entry’s text will therefore affect the attributes which use that entry.
                            See the section entitled Lookup Services for more information.


For each table that supports custom attributes, a second database table is present with a single row
or multiple rows for each attribute value, for example (for illustration purposes only):

Field                       Description
LocationAttributeId         Unique identifier for the attribute value. Primary key of the table.
LocationId                  Identifies the Location record this value applies to. Foreign key to the
                            Location table.
AttributeId                 Identifies the attribute this value is for (e.g. Altitude). Foreign key to the
                            Attribute table.
Value                       Value stored for this attribute against this location.




Online Recording Toolbox Requirements – CEH         16 of 35
CORE MODULE

The Core Module provides a set of back-end services required to run an online recording website. In
addition a user interface is available solely for the purpose of managing the back end services. A
single Core Module is able to support multiple websites. The purpose of separating the OPAL online
recording application into the Core Module and Site Module is to segregate the processor, disk space
and technology intensive parts of the system from the web site user interface layer, enabling each
participating organisation to host their own website using low cost hosting services without having to
host their own Core Module. The Core Module itself is hosted centrally, potentially by the BRC, as well
as separately by organisations seeking to take control of the entire system and with the hosting
capacity to do so.

ARCHITECTURE
The core module consists of a spatially enabled database plus an application server - a suite of web
services which may or may not reside on the same physical machine as the database. There is also a
minimal user interface required for administration tasks against the Core Module. The software that
must be installed for the Core Module has a preference for being free and Open Source though
minimal license costs are acceptable as long as this does not

         a) preclude the fully customisable nature of the module and

         b) through cost, preclude the installation of the Core Module by larger organisations such as
         national schemes and major record centres that are likely to want to install the Core Module.

The preferred database is MySQL due to the ease of hosting and its low cost of hosting and
ownership. MySQL also supports spatial data types required to support online recording. A database
agnostic solution which supports PostGIS and/or SQL Server in addition to MySQL is preferable but
not at the cost of performance of the system. This may be achieved through use of a database
agnostic application framework.

There is also a preference for the application server web services to be written in PHP due to its wide
support, Open Source nature and low cost hosting options. However, alternative technologies can be
considered given the constraints defined relating to ability for larger organisations to be able to host
the Core Module at relatively low cost. The selection of any technical solution must take into account
the fact that this is an Open Source project and the ease with which new developers can join the
project is critical to its success, therefore widely available technologies are used at all times.

The Core Module can be divided into a common data access layer plus a number of services which
interface with the online recording websites.

LOOKUP SERVICES
The Core Module provides a suite of services for facilitating the use of controlled terminology and
other lookup information. A lookup operation can be performed against any of the standard data
entities (for example locations, species or people) or against a custom list of terms created to control
the values entered in a particular attribute.

Controls which use the lookup lists include but are not limited to drop-down selection boxes, list
boxes and text entry boxes featuring auto-completion.




Online Recording Toolbox Requirements – CEH         17 of 35
CUSTOM TERM LISTS
As well as the standard, complete lists of information provided for each of the data entities, the
lookup services provide access to customised lists available for each site copy. Lists can be customised
in the following ways:

    1) The administrator of a Core Module creates a subset of a standard data list and makes it
       available as a named list for all sites using the Core Module. For example, the Core Module
       administrator creates a Woodland Birds subset of the main species list.
    2) The administrator of a website creates a subset of a standard data list and makes it available
       as a named list only available to the specific site. For example, the site administrator creates
       a Heathland Birds subset of the main species list.
    3) The administrator of a website creates a subset of a named data list subset (as described in
       item 1) and makes it available as a named list only available to the specific site. For example,
       the site administrator creates a Small Woodland Birds subset of the Woodland Birds species
       list.
    4) The administrator of a website creates a new, custom list in order to control the entries on a
       custom attribute they use on their website. The site administrator creates and populates a
       new list of terms called Specimen Types.
    5) The administrator of a website creates a subset of a custom list in order to control the
       entries on a custom attribute they use on their website. The site administrator creates
       subsets of the Specimen Types custom list called Zoological Specimen Types and Botanical
       Specimen Types.

This is illustrated in the following diagram.




                                             Standard
                                               Lists

                                                                                     Public
                                                                                     Information


                               Subset        Subset        Subset




        Private      Private            Private       Private            Custom
        Subset       Subset             Subset        Subset             Custom
                                                                          Lists
                                                                           lists

                                                                                     Site specific
                                                                                     Information


                                                         Custom List   Custom List
                                                           Subset        Subset




                           FIGURE 2 - HIERARCHY OF LISTS, CUSTOM LISTS AND SUBSETS



Online Recording Toolbox Requirements – CEH               18 of 35
LOOKUP SERVICES FUNCTIONALITY
The Lookup Services provide the following, with fictitious examples given in italics for clarification:

        Ability to maintain a centralised master copy of a list controlled by the administrator of the
         Core Module. A master list of species and higher level taxa is provided on the Core Module.
        Allow a Site Module administrator to create their own named versions of common lists
         stored on the Core Module. A site’s own list version may contain additional terms and may
         also hide terms not required on the site. A site utilises its own subset of the master species
         list called Woodland Birds specifically tailored for the users of the site.
        Allow a Site Module administrator to create and manage lists and list content for use on their
         own site only. Terms and lists may be added, removed and edited though removal is only
         allowed when the term or list is not in use, or where a suitable replacement is nominated.
         The site creates a list of plumage terms in order to control the terminology used in records of
         birds.
        Allow controls on each site to be linked to the master copy of a common list, a named subset
         of a common list or a site specific list. When entering records in the site, a record card is
         available with data entry controls that allow any species from the master species list to be
         recorded. A second record card is available that constrains the species entered to the list of
         Woodland Birds. The Plumage entry control is constrained to the site specific Plumage Terms
         list.
        Allow a Site Module administrator to declare additional terms to add to any list. Terms added
         to common lists are available on the local site only. The core module provides a Sample
         Method common list containing common sampling method terms. This contains a term
         Quadrat. A bryophytes site administrator adds a term to this list “1*1m Quadrat” but this
         term is not available to other users of the sample methods list.
        Allow a site to be designed to enable dynamic adding of terms to lists during data entry. The
         bryophyte site’s data entry screen provides a text box with auto-complete facilities for the
         entry of the sample method. If the user enters a term not in the list, then this term is stored
         in the sample methods list.
        Allow any control on a site to be linked to a named list, by providing either:
               o the complete contents of the list,
               o a paginated subset of the list (for example page 2 of 10 given a configurable number
                    of entries per page),
               o a list filtered to terms or attributes flagged as searchable that start with or contain
                    given search text,
               o a list filtered by attribute, allowing a basic query to be constructed using at least
                    EQUAL, NOT EQUAL, CONTAINS, STARTS WITH, AND and OR logical operations. For
                    example, list people where the Surname STARTS WITH “Br” AND Initials NOT EQUAL
                    “J.”. An alternative approach is to allow the filter to be defined using an SQL
                    statement.

         The list content is returned in a form suitable for use in AJAX driven controls as well as
         controls populated at page load time. The list can be sorted by the overall term or any of the
         individual attributes of the list entry, e.g. the taxon list is sortable alphabetically or by
         taxonomic sort code. The previously mentioned Sample Method text box searches on the
         Sample Method list for terms that start with the characters entered by the user. A user
         entering “Quad” is presented with Quadrat as a term for auto-completion.

        Allow the contents of any lookup list to be populated by uploading a CSV file and a providing
         parameters to the service defining a set of column mappings.


Online Recording Toolbox Requirements – CEH           19 of 35
RECORD SERVICES
This service provides read and write access to the list of records for each of the standard entities,
custom lists or the user’s profile. The following actions are available:

        Read record – takes a record ID as a parameter and makes the record’s attributes available
         for population into an HTML form consisting of data entry controls.
        Update record – accepts a submission of record attributes from an HTML form, applies the
         validation rules associated with each attribute and its data type and updates the record.
        Delete – deletes a single identified record.
        Read dataset – accepts an SQL statement which selects attributes from the standard entities
         and/or custom term lists and retrieves a dataset that contains the results.
        Spatial Query – operates in the same fashion as the Read Dataset function for the
         Occurrences and Locations data, but in addition accepts a point or polygon (as a WKT string
         using WGS84 latitude and longitude) and retrieves a list of all occurrences or locations that
         match the query. When requesting the data the developer can select from the following
         query options:
              o Any overlap is accepted
              o The returned record must fall inside the query boundary
              o The returned record must fully contain the query boundary.

VALIDATION SERVICES
The Core Module provides a web service API for validation of data entered on the online recording
websites. The Validation Services themselves do not define when the validation occurs; rather the Site
Module defines when validation is triggered:

        using client-side JavaScript at the point of data entry
        on the server at the point of acceptance of a posted record
        after a number of records have been entered in a session as a batch process.

Validation services are available which fulfil the functions defined in the following table:

Service                     Description
Vague date validation       Checks an entered date string and confirms that it is recognisable as a
                            valid vague date string.
Spatial        reference    Checks an entered spatial reference string and confirms it is recognisable
validation                  as a valid spatial reference.
                            The minimal requirement is for a spatial reference to be acceptable as an
                            British or Irish National Grid Reference (including DINTY tetrads and
                            quadrants) or a WGS84 decimal latitude and longitude. The supported list
                            of spatial references systems is extensible as described in Appendix 2 –
                            Spatial Reference and Georeference Formats.
Validate list entry         Checks an entered string and confirms that it can be matched to a single
                            entry on a named lookup list, returning the ID of the entry, the full
                            matched term and the preferred term for the entry if available. Note this
                            may be achieved by calling the Lookup Service to retrieve entries from a
                            list filtered by search text.
Validate number             Checks an entered string and confirms that it can be parsed as a number.
Validate attribute          Checks an entered string against the parameters of an attribute (minimum
                            and maximum, regular expression).




Online Recording Toolbox Requirements – CEH          20 of 35
HELPER SERVICES
In a similar fashion to the Validation Services, the Core Module provides a web service API that
declares a number of helper methods required by online recording websites.

Service                     Description
Georeference                Georeferencing is the process of attaching geographic data to the
                            available data for a record. Examples include the identification of latitude
                            and longitude coordinates from an available post code or place name.
                            The georeferencing method takes the following parameters:
                                  a text string which describes a post code, place name or spatial
                                     reference (using grid references or any other spatial reference
                                     system supported).
                            The value returned by the method describes the data in terms of WGS84
                            decimal latitude and longitude values. This may be a single point, a
                            polygon or grid square and may also include accuracy information (e.g.
                            within 2km of 50.238, -1.024). The results of a georeferencing request
                            may also include multiple entries where the georeferenced text cannot be
                            resolved to a single value.
                            The results returned by this method use the Well Known Text (WKT)
                            format.
                            The implementation of the georeferencing facility is likely to be based on
                            an existing georeference API such as the Google Maps API or Yahoo
                            GeoPlanet.
Reverse georeference        This method is provided in order to identify a list of possible locality
                            matches for a given geographic coordinate. The reverse georeferencing
                            methods take the following parameters
                                  a text string describing either spatial reference
                                  the spatial reference system, possibly by identifying its EPSG
                                     number. For more information, see http://www.epsg-
                                     registry.org/.
                                  Alternatively, an already converted WKT value which describes a
                                     single point or polygon using WGS84 decimal latitude and
                                     longitude.
                            The return result is a list of locations which contain the supplied
                            parameter. This is achieved by running a spatial query against the
                            boundaries of the list of locations.

SECURITY SERVICES
This module is described here to clarify its functionality. If the functionality is available within the
standard facilities of the selected CMS tool then this is part of the Site Module. If the functionality is
not available within the selected CMS, then the User Authentication becomes an additional service
based component of the Core Module.

Description                                                                                   Required
Allow registration details of a new user to be posted into the system.                        Mandatory
Allow a user’s profile details to be read and updated.                                        Mandatory
Send activation emails to newly registered users as appropriate.                              Mandatory
Provide a method which accepts the input from the Login control and authenticates the         Mandatory
user onto the site.
Provide a method which emails the username and password to a user when a forgotten            Mandatory
password reminder is requested.
Provide a default set of 3 user access levels, user, editor and administrator.                Mandatory




Online Recording Toolbox Requirements – CEH          21 of 35
Allow additional levels of user access to be created by site administrators.                 Desirable
Allow a matrix of tasks against user access levels to be created which defines which         Mandatory
tasks can be performed by each user access level. Tasks include access to each
individual page on the website.
Provide methods which allow the online recording website code to easily determine            Mandatory
whether the logged in user has access to a certain function.



REPORT SERVICES
The Report Services part of the core module provide methods for managing and accessing the list of
reports stored on the system. The following methods are provided.

Description                                                                                  Required
Allow a list of globally accessible reports to be obtained.                                  Mandatory
Allow a list of site specific reports to be obtained.                                        Desirable
Allow each list of reports to be filtered to only include reports that apply to one of the   Mandatory
standard entities.
Allow a new globally accessible report to be added to the system. A report is declared as    Mandatory
an SQL statement with a definition of parameters, which entity it is associated with as
well as metadata including title and description.
Allow a new site-specific report to be added to the system.                                  Desirable
Provide access to the output of any report in a form that is suitable for displaying in a    Mandatory
Data Grid control.
Allow existing report details to be modified.                                                Mandatory
Allow existing report definitions to be removed from the system.                             Mandatory



Reports are defined using SQL queries with associated parameter information and metadata. Report
SQL queries are able to use the following special values which are substituted by the system when the
report is run:

Value                          Description
CURRENT_USER                   Id of the logged in user. For example, allows a report to be
                               automatically filtered to only records recorded by the logged in user.
SITE_URL                       Root URL of the site. Allows a report output to provide HTML which
                               links to the underlying record’s full details page.
PREFERRED_SPECIES_NAME         Substituted with the name of the attribute that contains the scientific
                               name or common name of a taxon depending on the user’s profile
                               option Display Common Names.



USER INTERFACE
The Core Module exposes a user interface with the following screens and facilities:

Description                                                                                  Required
The Core Module replicates the login, forgotten password, user list and maintenance          Mandatory
facilities from the default site installation described in the section Error! Reference
source not found.Error! Reference source not found.. In this case, they are used to
provide for login and user management facilities within the Core Module.
The Core Module includes a user interface that allows the list of global term lists or the   Mandatory
list of custom term lists specific to any site operating on the Core Module to be viewed
and selected. For each term list, it is possible to add, remove and modify entries on the



Online Recording Toolbox Requirements – CEH         22 of 35
list. It is also possible to add, remove and rename lists.
For a selected list, any number of subsets can be defined and items from the list added
to the subset. For more information on available functionality, see the section entitled
Custom Term Lists.
There is a list of sites which operate against this Core Module, including site name,            Mandatory
description and owner details.
A facility for adding, removing and modifying registered sites on the Core Module is             Mandatory
provided.
The list of sites includes the following information:                                            Desirable
        Number of occurrence records
        Recent activity (new records this week or month, number of users currently
             connected)
The user interface on the Core Module has the same requirements for access to non-               Mandatory
occurrence records as the demonstration site, described in the section Non-Occurrence
Data Requirements, with the following amendments:
        Provides full control over the master list of each standard entity and custom
             term list.
        Before working with custom term lists or subsets of lists specific to any site, it is
             necessary to select from a list of the sites running against the core module.
Note that this includes functionality allowing CSV files to be imported into any list or
subset.




Online Recording Toolbox Requirements – CEH           23 of 35
SITE MODULE

The Site Module is a collection of web pages that provide online recording functionality for the web
users and utilise the services exposed by the Core Module in order to achieve that. Although an “out
of the box” installation of the Site Module is available which provides a simple but complete online
recording solution, the Site Module is composed of an aggregation of components and web pages
allowing any level of customisation to be performed.


                                   Site Module                                                Core Module
                                                                                Service

                         Web pages
                                                                 Data reads

                          \/\/\/\/\/\/\                                         Service
Re-useable
                          \/\/\/\/\/\/\
web page controls                                             Data writes
                          \/\/\/\/\/\/\



                                                                                Service
                                                              Other service
                                                                  calls

                                                                                Service



                                FIGURE 3 - SITE MODULE WEB PAGES AND CONTROLS




CUSTOMISATION AND CONFIGURATION APPROACHES
The following list defines the different approaches to customisation for the online recording websites.

        Default Site Configuration

        The default site template provided has configuration facilities allowing the site administrator
        to control which types of data entry facility are available, the content of lookup lists used for
        data entry including species and site lists, the level of access given to non-registered users as
        well as other identified site options.

        Images, XHTML and CSS

        The design of the web pages displayed on the user’s web browser is controlled by XHTML
        markup, CSS stylesheets and images. The most basic level of customisation is to modify the
        images and CSS provided with the default site installation to allow the site appearance to be
        rebranded. The XHTML output by the site is controlled by templates within the CMS system
        used by the site module. Modification of the CMS site templates provides further options for
        customising the appearance of the site.




Online Recording Toolbox Requirements – CEH           24 of 35
        CMS content management

        The management tool provided with the Content Management System provides site
        management tools that allow addition, modification and removal of pages and content on
        the website. In addition, the CMS management tool allows

        Developing new sites

        Because the Site Module is architected as a set of pluggable tools and components combined
        within a set of web pages, development of entirely new sites is possible which re-use the
        appropriate tools and components to facilitate rapid development.

COMPONENTS
The following list of components is present in the toolbox available from which an online recording
site is constructed. Note that depending on the CMS solution selected to host the site, some of these
components may take the form of standard controls or pages provided within the CMS’ functionality
rather than components written specifically for the OPAL online recording project.

Where possible, all components use templates to allow the output XHTML to be fully configured. The
design of the toolbox is such that it is a simple task for a developer to create a page which loads an
associated record from one of the database entities and link each attribute (custom or standard) to a
control on the page allowing viewing and data entry.

COMPOSITE CONTROLS

 L OGIN C ONTROL
The Login Control is a panel which provides a standardisation of the following controls:

       A label and text entry box for the username. The site administrator is able to configure this
        to accept the user’s email address as an alternative.
       A label and text entry box for the password, with obscured characters.
       A login button which authenticates the user on the site when they log in or displays a
        message if the login is unsuccessful.
       An optional link to a page allowing the user to register on the site. Note that on sites where
        registration is by invite only, this link is removed.
       An optional “Request Forgotten Password” link to a page allowing the user to request their
        forgotten password.
       An optional checkbox that allows the user to select the option to remember them the next
        time they visit the website, so there is no need to log in again.

 U SER R EGISTRATION C ONTROL
The User Registration Control is a panel which provides a set of controls for capturing information
required to register a new user. This includes the following text entry controls:

       User name
       Firstname and surname
       Email address
       Password, with a password confirmation entry control.
       OpenID URL with a help page link, described below.




Online Recording Toolbox Requirements – CEH         25 of 35
        Make data publicly available – a checkbox that, when checked, defines that the user accepts
         all data entered onto the system is being placed in the public domain.
        Permission confirmation – a checkbox that, when checked, defines that the user has
         permission to submit all the data they enter onto the system. Default is unchecked.
        Hide email address – default checked. When unchecked allows the user’s email address to be
         displayed to other users.
        CAPTCHA – displays a graphic image containing a word or a series of characters that is
         obscured so that it is difficult to machine read but readable to humans. The user is required
         to type in the correct sequence of characters in order to successfully register. This control is
         hidden if the option to authenticate new users using CAPTCHA is turned off.
        Submit button.

The email address format is validated to confirm it is of the correct structure. In addition, if the option
“Require new users to activate by email” is set, an email is sent to the address requiring that a link is
clicked on to activate the new account. The user is not able to login until their account is activated.

The user registration control may be extended in some implementations to capture additional user
profile information.

OpenID support is provided for users which have an OpenID account. OpenID is an open framework
for digital identities. Users create a single identity at their chosen OpenID provider which provides
them with a globally unique URI. When they register with a site that supports OpenID such as this
online recording site, they provide their URI which allows the site to obtain their standard user profile
information from the provider. This greatly simplifies the registration process and allows a single
digital identity to be used across sites. For more information, see http://openid.net.

 F ORGOTTEN P ASSWORD C ONTROL
The Forgotten Password Control is a panel which is placed on a page referred to by the Request
Forgotten Password link in the Login control. This provides a text entry box allowing the user to enter
their email address and a submit button. On clicking submit, the email address is checked against the
list of known users. If valid, an email is sent to the address detailing the password. Otherwise a
message is displayed stating that the email address could not be recognised and the user can retype
their email address to try again.

 U SER P ROFILE C ONTROL
The User Profile Control is a panel which contains the basic controls required for modifying a user’s
profile. This includes at least the following elements:

        Email address
        Password – allows a new password to be specified
        Password confirmation
        Avatar – uploadable image shown beside user’s comments
        Location – text box allowing the user to optionally specify their locality
        Interests – text box allowing the user to optionally specify their interests
        Website URL – text box allowing the user to optionally specify their web address, displayed
         by the user’s comments
        Website title – text box allowing the user to specify the caption of the hyperlink to the user’s
         website displayed next to the user’s comments.
        Display Common Names – checkbox allowing the user to set their preference for displaying
         common names over scientific names for species.



Online Recording Toolbox Requirements – CEH          26 of 35
        Hide email address – when unchecked allows the user’s email address to be displayed to
         other users.
        Home location – a text box allowing entry of a postcode or spatial reference and a spatial ref
         picker control that allows the user to identify the location of their home. This is used to
         define the default centre point when displaying a spatial ref picker or Google distribution
         map, and also allows the developer to use this information when creating records. For
         example, a checkbox placed on a data entry screen for “At Home” could allow a quick
         method of defining that an observation was recorded in the user’s garden.

DATA ENTRY CONTROLS
The controls in this section describe screen elements that are associated with attributes (custom or
standard) of the record that is being displayed on the current page. The controls all allow the
developer to specify which attribute(s) they are linked to, and automate loading and of the attribute
value and saving it back to the database.

All controls that are used to enter data into an attribute of a data entity, or a custom attribute, apply
the validation rules associated with the attribute at the point of data entry using JavaScript if
JavaScript is enabled on the client browser. In all cases, a second layer of validation occurs on the
server to ensure that data remains valid when posted from a browser without JavaScript support,
where JavaScript is disabled, or where the data is posted from a non-standard client interface which
does not use these controls.

An additional point is that when a custom attribute is flagged as allowing multiple values, then the
control’s behaviour is modified to accept multiple values. This may be achieved by adding buttons to
add and delete “rows”, or new controls on the screen, or by allowing the user to type multiple values
into a box with a list separator between each value.

 D ATA L ABEL
A static text control that simply displays the value of a database attribute.

 T EXT B OX
A text entry control which is associated with an attribute or custom attribute. The text box accepts
data of the appropriate format for the data type it is linked to, using client side validation where
possible but always backed up by server side validation. For example a text box linked to a
georeference data type validates that the entered text matches one of the supported spatial
reference or postcode formats. For more information on supported formats see




Online Recording Toolbox Requirements – CEH          27 of 35
Appendix 2 – Spatial Reference and Georeference Formats.

 A UTO - COMPLETE T EXT B OX
This is a variant of the Text Box control which allows a lookup list to be associated with it. When
typing into the control, AJAX is used to identify possible matches which are displayed in a drop down
list beneath the control. If the user selects an item from the list then the item’s caption is populated
into the text box. The intention is to provide an rapid means of using controlled terminology lists and
encourage the user to select existing entries rather than enter new ones. This leads to better data
quality and less variations or typographic errors of the same term.

 U PLOAD T EXT B OX
A text box with an associated Upload button that displays a file browser and allows the user to select
a file for upload to the site. The control can be configured to allow uploading only files of a certain file
type or types.

The uploaded file is placed into a document library folder location on the server. In addition, for
recognised image file types, resized versions of the image are created. The site administrator is able
to configure how many resized images are created and which sizes they are to be created as. For
example, a photo gallery upload accepts an original jpg file at 3000*2000 pixels called “seahorse.jpg”.
The image is stored in the document library location at its original size as IMG_id.jpg; a 300*200
version is created and stored as IMG_id_medium.jpg and a 128*128 version is created and stored as
IMG_id_thumbnail.jpg.

 D ATE P ICKER C ONTROL
A text box control allowing the entry of a date or vague date. A drop down button beside the control
provides a monthly calendar allowing the user to page backwards and forwards between the months
to select a date. Selecting a day posts the date into the text box.

 D ROP D OWN BOX
A drop-down box containing a list of options loaded from a list provided by the Lookup Services
module.

 L IST BOX
A fixed list box containing a list of options loaded from a list provided by the Lookup Services module.
There is an option to specify that the list box allows multiple item selection.

 T ICK L IST
Outputs a list of items loaded from a list provided by Lookup Services, with a checkbox beside each
one allowing the user to select one or more items.

 D ATA G RID
The Data Grid control provides a paginated grid of entries loaded from a lookup list provided by the
Lookup Services or Record Services part of the Core Module, with a configurable target URL when the
row is selected. The list of columns in the grid is configurable and can display any attribute of the
lookup entry or record, or the complete preferred caption of the lookup entry. The rows listed in the
grid are defined by the filter applied to the lookup list or record list. The grid includes a pagination
footer which provides information on the current visible page and allows navigation between pages
of data.


Online Recording Toolbox Requirements – CEH           28 of 35
The Data Grid has a default template which defines the output as an XHTML table. However, the site
administrator is able to specify custom templates which allow the output to be modified, for example
as <div> or <span> elements.

When displayed on a data entry page, the developer is able to define which columns are editable and
replaced by data entry controls. This is suitable for creating simple grids for data entry. The developer
is also able to enable the addition of new rows and deletion of existing rows in the grid (and therefore
to the associated list)

Example uses of the Data Grid control are to list the users of the system for administrators to manage
the accounts, or for a user to list the records they have entered.

 H IERARCHY C ONTROL
The hierarchy control provides an alternative to the Data Grid for browsing information in either a list
of Taxa or Locations both of which have a hierarchical structure. The hierarchy is dynamically
populated with only the top level of the hierarchy being loaded initially, and sub-levels are loaded
from the database only when required using AJAX calls to the Core Module.

The data displayed and caption structure for each hierarchy node is configurable when setting up the
control on a page.

 L OOKED U P I TEM D ETAILS
A XHTML <div> element which contains labels and data for each of the attributes and values for an
item looked up by the user using one of the other controls on the page that allows selection of an
entry from a list,, for example a drop down box or auto-complete text box. When an item is selected
in the other control, the Looked Up Item Details control is refreshed using AJAX to display the
selected item’s attribute values. The display of this control is configurable using CSS.

An example of this is to provide a panel which displays a species’ group and thumbnail image when
the user selects a species using an auto-complete text box.

 G OOGLE M APS C ONTROLS
The standard Google Maps API is encapsulated into 2 variations provided as re-useable controls. The
first is a Spatial Ref Picker allowing the the user to click at a location on the map to provide a spatial
reference for use during data entry. The second is a Distribution Map control that allows species
distributions to be displayed.

The following points apply to both variations of the Google Maps controls:

        The web site administrator is responsible for obtaining a Google Maps API key and ensuring
         that the terms and conditions of Google’s license are met.
        Allows standard Google Maps pan and zoom behaviour and allows controls to be added to
         the map window, for example to define the background layer (Map, Satellite or Terrain), pan
         and zoom controls.
        The default zoom and centrepoint is a configurable option for the administrator.
        Allow polygon layers to be added to the map. At the minimum, this allows KML files of site
         boundaries to be created and linked to maps. A further option is to allow WMS and WFS
         layers to be added to the map.

 S PATIAL R EF P ICKER
The Spatial Ref Picker control has the following features:



Online Recording Toolbox Requirements – CEH          29 of 35
       Allows a text box to be defined which acts as the recipient of the spatial reference.
       Allows the system used for the spatial reference captured to be defined (e.g. Ordnance
        Survey Grid Reference or Decimal Latitude and Longitude WGS84).
       When the user clicks on the map, the spatial reference is passed back to the linked text box
        without causing a page refresh.
       Where grid references are captured, the precision of the grid reference reflects the accuracy
        of the clicked point implied by the zoom level of the map. Likewise, the accuracy of latitude
        and longitude or other returned coordinates is rounded according to the zoom level.
       Can be panned and zoomed to a given point on a map by external controls. For example, a
        text box linked to the Georeference Support Service is able to pan and zoom the Spatial Ref
        Picker control to the location identified by the Georeference. In the following examples, the
        user first enters the name of a general location into a Georeferenced text box control, then
        the Spatial Ref Picker is panned and zoomed to show the location.




        The user then clicks on the map to put a marker down at the exact location of the record:




 D ISTRIBUTION M AP C ONTROL
This control is displayed on a page accessed when requesting a distribution map of a selected species,
or when requested after viewing a report containing spatial data.

Each occurrence of the species or row of the report is drawn on the map as an icon (for coordinate or
POINT based observations such as latitude and longitude) or a polygon (for grid square observations
such as British National Grid References). Polygon transparency can be configured by the developer.
Clicking on an icon or polygon displays a popup information window displaying the underlying report
attributes or a panel of information about the occurrence with content that is configurable by the site




Online Recording Toolbox Requirements – CEH        30 of 35
administrator. In addition, the popup window contains a hyperlink to the occurrence or other record
that underlies the spatial object on the map that was clicked on.

 R EPORT G RID WITH CSV DOWNLOAD
The Report Grid is a control that can be linked to the output of any report or other query. It
automates the following:

       Outputs XHTML according to the developer’s templates
       Creation of column titles
       Output of rows, with alternate rows having a different CSS class therefore allowing different
        styling.
       Pagination with a page navigator displayed beneath the table.
       Sort by column title clicked on.
       CSV download link
       Printable version (removes the pagination to display the entire table in a new window)
       If a spatial column is present (e.g. spatial reference) then there is automatically a Display on
        Map link which loads the data onto the Map page if one is present.




Online Recording Toolbox Requirements – CEH        31 of 35
GEOSERVER

GeoServer is an Open Source solution which exposes spatial datasets using OGC standard webservices
including WFS (Web Feature Specification) and WMS (Web Map Service). It is an optional but natural
extension to the online recording project Core Module simplifying the connection of external
geospatial tools to the stored occurrence data. OGC standards are widely supported by web service
clients including, but not limited to, Google Earth, Google Maps, OpenLayers and other GI S systems.

Although not explicitly part of the project, the configuration and customisation documentation
includes a step by step guide to setting up GeoServer, connecting it to the occurrences held in the
database, and setting up an example web page which allows species distributions to be drawn using
the WFS service provided by GeoServer.


                                                                                     Google Maps



                                                                                      Google Earth

       MySQL                    GeoServer
     Occurrences
                                                               Web                    OpenLayers



                                                                                            etc



FIGURE 4 - GEOSERVER OVERVIEW




Online Recording Toolbox Requirements – CEH       32 of 35
APPENDICES

APPENDIX 1 – VAGUE DATES
In biological recording, there is a frequent requirement to allow capture of records where the date of
observation or determination is uncertain. Many very valuable records are sourced from literature
with a limited amount of precise information, so it is essential that the recording application allows
imprecise dates to be entered and analysed.

The online recording system shares the concept of Vague Dates with the Recorder biological
recording package. This allows a textual representation of an exact or vague date to be entered. The
system decodes this date text into a start and end date of the range of possible dates, for example the
start and end of the possible year. Because all dates are then stored in a common format of start and
end date, analysis of the data remains possible in a consistent fashion.

The following table illustrates examples of each type of vague date that is accepted, and provides the
start and end date values that are stored in the database for each:


Date Type                     Example Text                        Start Date        End Date

Exact date                    23 Mar 1987                         23 Mar 1987 23 Mar 1987

Date range                    23 Mar 1987 – 30 Mar 1987 23 Mar 1987 30 Mar 1987

Month in year                 Mar 1987                            01 Mar 1987 31 Mar 1987

Month range                   Mar 1987 – May 1987                 01 Mar 1987 31 May 1987

Season in year                Summer 1987                         01 Jun 1987       31 Aug 1987

Year                          1987                                01 Jan 1987       31 Dec 1987

Year range                    1981-1983                           01 Jan 1981       31 Dec 1983

From year                     1980-                               01 Jan 1980       Null

To year                       -1979                               Null              31 Dec 1979

Month                         July                                1 Jul 9999        31 Jul 9999

Season                        Autumn                              1 Sep 9999        30 Nov 9999

Unknown date                  Unknown                             Null              Null

Century                       18c                                 01 Jan 1700       31 Dec 1799

Century range                 18c-19c                             01 Jan 1700       31 Dec 1899

From start of century 18c-                                        01 Jan 1700       Null

To end of century             -18c                                Null              31 Dec 1799




Online Recording Toolbox Requirements – CEH        33 of 35
APPENDIX 2 – SPATIAL REFERENCE AND GEOREFERENCE FORMATS
A spatial reference system is a means of representing a geographic coordinate or grid square as a
human readable text string. There are many spatial reference systems in use worldwide and several
are in frequent use within the British Isles. In order for data to be comparable and analysable despite
being entered in different systems it is necessary to translate the geographic information into a
common system. The common system used for this project is decimal latitude and longitude using the
WGS84 datum since this is a worldwide system made popular through its use in the GPS network.

As well as spatial references, the data entry allows for other types of text to be entered that can be
translated (georeferenced) to its geographical coordinates. An example supported in the initial
implementation is the British postcode system. Therefore, in order for a spatial reference or
georeference system to be supported a translation from the system to its equivalent latitude
longitude WGS84 coordinates is required. There are a number of Open Source projects in existence
that provide for coordinate transformation between different systems, for example JSCoord
(http://www.jstott.me.uk/jscoord/) or Proj4 and Proj4JS (http://proj4js.org/) and it is expected that
this project will utilise one of these for coordinate transformation.

BRITISH NATIONAL G RID REFERENCE SYSTEM
A British National Grid Reference is given by specifying a two letter code for the 100km square, then
giving an even number of digits representing the easting and northing of a point within the 100km
square. Optionally, the position within a 10km square can be given using a single additional letter (a
Tetrad or DINTY reference), or using 2 additional letters (a quadrant reference).

Note that because the British and Irish National Grid System are based on OSGB36 rather than the
WGS84 datum, after converting to latitude and longitude it is necessary to further perform a datum
shift to correct for the differences between the two.

For further information, see http://en.wikipedia.org/wiki/British_national_grid_reference_system.

A British National Grid Reference is translated to a grid square with 4 latitude and longitude points
stored in the database for each corner, facilitating spatial queries on the data.

IRISH GRID SYSTEM
The Irish Grid System uses the same notation as a British National Grid Reference except that there
are only 20 100km squares, so the 100km square is identified by a single character prefix rather than
2.

For further information, see http://en.wikipedia.org/wiki/Irish_grid_reference_system.

LATITUDE AND LONGITUDE WGS84
Valid latitude and longitude coordinates are specified by a decimal number of degrees latitude,
optionally followed by N (default) or S, then a comma, then a decimal number of degrees longitude,
optionally followed by an E (default) or W.

POSTCODES
Entry of a postcode provides the system with a centroid for that postcode as well as accuracy
information reflecting the size of the postcode region.




Online Recording Toolbox Requirements – CEH        34 of 35
Note that since the copyright for the UK postcode database is held by Royal Mail, the precision
available through non-paying methods is limited to approximately 2km, where the last 2 characters of
the postcode are not significant.

ADDITIONAL SYSTEMS
In addition to the above supported systems, the spatial reference translation utilities in the online
recording project are written in an extensible fashion in order to allow easy addition of further
systems in future. Support for UTM (Universal Transverse Mercator) system and British & Irish
National Grid Eastings & Northings are likely enhancements, as well as supporting the common
systems and notations in use in Europe.




Online Recording Toolbox Requirements – CEH       35 of 35

								
To top