skelton by ashrafp


									                    Implementing Metadata Standards for Recordkeeping:
              Challenges and Developments in the Australian Government Sector

There are several challenges facing Australian Government organisations in the development and
implementation of metadata standards for recordkeeping. One suspects that many of these
challenges are little different from those faced by any metadata community represented here today.

This session will identify some of these challenges, before moving on to cover two current
recordkeeping metadata standard developments in the Australian Government sector, and how these
are attempting to address the challenges raised. It will focus not so much on the content of these
standards, as on the work being undertaken by the National Archives, and its partners, to assist in
the implementation of these standards across the Australian Government.

I will begin with a couple of definitions, before making some brief comments about recordkeeping,
and the role played by the National Archives in the Australian Government recordkeeping sector. I
will also provide some information about the broader Australian recordkeeping metadata
environment, as this constitutes the framework within which current recordkeeping metadata
standards are being developed.

Records and Recordkeeping
According to AS/ISO 15489.1-2002 Records Management (Part 1: General), records are
“information created, received, and maintained as evidence and information by an organisation or
person, in pursuance of legal obligations or the transaction of business.”

Recordkeeping metadata is defined in AS/ISO 23081.1-2004 Information and documentation –
Records management processes - Metadata for records (Part 1: Principles) as „data describing the
context, content and structure of records and their management through time.‟ Recordkeeping
metadata is used to „identify, authenticate and contextualise records and the people, processes and
systems that create, manage, maintain and use them, and the policies that govern them.‟

Recordkeeping is essential to the conduct of business, whether within government or the private
sector. Making and keeping accurate records, and managing them appropriately for as long as they
have value, provides reliable evidence of business decisions and actions, and enables organisations
to be accountable to customers, clients and the community.

In the Australian Government, the role of the National Archives is to provide leadership and
guidance to government agencies in recordkeeping through the development of standards and
guidelines, and by working with agencies to ensure that government records are:
 created, controlled and managed within agency systems;
 retained for as long as necessary; and
 where appropriate, transferred to NAA to be preserved and made accessible into the future.

While the basic principles of good recordkeeping – accuracy, reliability, authenticity,
accountability, accessibility – do not change over time, a big challenge facing us now is to continue
to apply these principles in the digital, or e-government, environment. One of the major tools
which can assist us in doing digital recordkeeping well is recordkeeping metadata.

The National Archives has been at the forefront of metadata initiatives in the Australian
environment since the late 1990s. It led the development of a metadata standard to describe online
Commonwealth Government resources, culminating in the Australian Government Locator Service

(AGLS) metadata standard for online resource discovery which, in 2002, also became an Australian
Standard, AS 5044.

During the development of AGLS, the National Archives realised that there was also a need for a
metadata standard to guide Commonwealth agencies regarding the metadata required for the
control, management, retrieval and use of records in agency systems. Such a standard needed to
include, but go beyond, metadata for resource discovery. As a result, the National Archives
developed the Recordkeeping Metadata Standard for Commonwealth Agencies (RkMSCA) which,
when it was published in 1999, was a world first.

The metadata set presented in this Standard consists of 20 metadata elements (8 of which are
mandatory) and 65 sub-elements. The metadata set is „record-centric‟, in that all elements are
described in relation to records, and are not considered independently of records. For example, the
element Agent (organisations and people) can only be used in terms of its relationships with, or
actions on, records. The Standard itself is entirely text-based. Although use examples are included
the Standard is, in essence, a technical specification, presenting detailed usage rules and conditions,
and information about element inter-relationships.

The Australian Recordkeeping Metadata Environment
Over the past few years the Australian recordkeeping community has taken a new approach to the
development of recordkeeping metadata standards. Concurrent with the development of the
RkMSCA was the development of the Recordkeeping Metadata Schema (RKMS), the product of a
Monash University research project in which the National Archives was a research partner.

The RKMS, published in 2000, represented a move away from the single-entity model, based
purely on the record, to a multiple-entity model for recordkeeping metadata, build around Records,
Agents and Business. The reason for this was partly to address perceived inadequacies in standards
built around the record - seen as hindering the capture of the full range of contextual information
required to make records meaningful over time - and partly to make recordkeeping metadata
standards more relevant in the broader business context.

The RKMS is implementation-neutral, and represents, in a conceptual model, several layers of
recordkeeping metadata needed to describe and contextualise records over time. The schema does
not presuppose or dictate any particular kind of implementation, enabling it to be used to develop
implementations for specific business systems, as well as dedicated electronic document and
records management (EDRM) and archival management systems, in any government jurisdiction or
business sector.

When developing the 1999 RkMSCA, the National Archives made a conscious decision to stay with
the single-entity model, as it was felt at the time that this would be less confusing to our key

Since 2000, two State records authorities – State Records New South Wales and the State Records
of South Australia - have adopted the RKMS multiple-entity approach in developing recordkeeping
metadata standards for their respective jurisdictions.

The recordkeeping community has now moved to standardise this new approach to recordkeeping
metadata with the release, in late 2004, of a new Standard AS/ISO 23081.1 Information and
documentation – Records management processes - Metadata for records (Part 1: Principles). This
Standard builds on the Monash University model, and introduces an expanded metadata model with
six entities, including Mandate, and incorporating Relationship as an entity in its own right. It also

presents a high-level process view of metadata, listing the different types of metadata required for
different business and recordkeeping processes.

AS/ISO 23081.1 is a high-level principles document which will be supplemented in 2006 by an
Australian Standard in two parts. The first of these will present a detailed conceptual model and
general guidance on its implementation, while the second will focus on specific implementation
advice. This work is being performed by the Standards Australia IT 21/7 Recordkeeping Metadata
committee, of which the National Archives is a member organisation.

Challenges in Implementing Metadata Standards for Recordkeeping
As the National Archives discovered after the release of the 1999 Standard, there are several
challenges to the successful development and implementation of metadata standards for

The broad challenge in the recordkeeping sense is ensuring that the recordkeeping principles
discussed earlier continue to be applied in the digital environment, and that the contextual richness
required for the use and understanding of records over time and within different environments is
captured and maintained. The successful implementation of recordkeeping metadata in records
management and business information systems is a key component of this.

Within this broader context, the other challenges for recordkeeping metadata standard
implementation are related to three main issues:
 selling the relevance - positioning recordkeeping metadata as an important part of doing
   business, and as a key component of agency business systems that make and keep records.
 making standards understandable and usable to ensure uptake by key audiences; and
 enabling recordkeeping metadata interoperability across systems and organisations, and its reuse
   for different purposes.

Some of these issues came to light in the years immediately following the release of the RkMSCA,
when it became obvious that there had not been a high level of implementation by the National
Archives‟ key audience – Commonwealth agencies.

The Standard raised a number of implementation issues for agencies. From the feedback received,
it was clear that agencies saw the following as major impediments to implementation of the
 the detailed text-based nature of the documentation;
 the lack of accompanying implementation guidelines;
 the lack of data models and schemas to assist agency ICT staff and/or system vendors in
    understanding and implementing the metadata set in their systems;
 inconsistencies regarding obligations for use and repeatabilities of some metadata elements and

Current Developmental Work
Throughout 2005, the National Archives has been working on two major – and related –
recordkeeping metadata standards. The first of these is a revision of the 1999 version of the
RkMSCA, where feedback from Australian Government agencies, and the move to multiple-entity
recordkeeping metadata standards, are influencing the nature and shape of the revisions.

The second is the recent development of the Australian Government Email Metadata Standard
(AGEMS), which is based on the RkMSCA.

The New Version of the RkMSCA
Briefly, the new version of the RkMSCA is structured around four separate entities – Record,
Agent, Business Activity and Mandate. Each of these entities has a number of subtypes associated
with it, as shown on the accompanying slide.

The Record entity allows for information to be captured about the properties of records in their
different manifestations, whether these be as single data objects or documents, or aggregations of
records, such as files or groups of records stored in directories.

The Agent entity is about documenting the actors involved in the business activities in which
records are created, managed and used, be they the organisation as a whole, a workgroup, an
individual, or a system that performs some business function.

The Business Activity entity enables the capture of information about the business functions of an
organisation. This can be both at a broad level, and at the level of specific business activities
performed by organisational workgroups, including information about business processes and the
transactions that comprise those activities.

The Mandate entity is intended for the capture of information about the various drivers for doing
business, including recordkeeping in support of business. It can be used to link business activities
to their legislative or regulatory environment, to the expectations of customers or citizens, and to
whole-of-government or organisational policies, procedures and business rules.

Each of the four entities / entity subtypes can be further described using a set of associated
properties, or attributes. Some of these are common across the four entities – for example,
identifiers, titles, and summary or abstract information. Others are applicable to only a single
entity, or to a subset of the entities.

The multiple-entity approach allows for independent, hence richer, description of the other
important factors in recordkeeeping. However, as we are discovering, it also adds to the complexity
of the metadata, particularly in terms of the need to document the many and varied relationships
between the different entities, at a number of different levels, over time. The devil is certainly
proving to be in the relationships.

It also raises questions regarding the boundaries of the standard. For example, the digital
environment provides opportunities for further metadata to be captured during records‟ active use
within an agency‟s business environment, much of which would be invaluable for the descriptive
work carried out by archival institutions on records of archival value transferred into their custody.
The idea of the capture of metadata once, for reuse many times, is an attractive one – but
determining the best place to capture it in the first place, and whose responsibility it is, is an issue,
and one that is related to the challenge of metadata interoperability.

Implementation Tools for the Revised Recordkeeping Metadata Standard
Version 2 of the Recordkeeping Metadata Standard will come with a number of new resources and
tools to assist in its implementation.

Firstly, models of the revised metadata entity and element set are being developed using the Object
Role Modeling (ORM) method. ORM is a conceptual modelling approach that views the world in
terms of objects and the roles they play. Using what is known as the “conceptual schema design
procedure” (CSDP), these objects and roles are discovered and expressed using elementary natural
language sentences called „relationships‟. It is particularly suited to modelling a metadata set like
the RkMSCA (ie – metamodelling) for two main reasons:
   It is more expressive than some other data modelling techniques, thereby allowing a high level
    of detail to be included, and a more rigorous analysis of the „Universe of Discourse‟ (UoD) to
    be achieved.
   ORM models can be populated with data instances or examples, allowing for more complete
    validation of the facts and relationships with domain experts using natural language.

We are finding that the ease with which validation can be done assists greatly in the identification
of metadata crossovers or duplication across boundaries, thereby making more visible those areas
for potential metadata reuse. This process is also useful for verifying element obligation and
repeatability, which will assist us in addressing the perceived inconsistencies of the earlier

Additionally, ORM models can be used to develop and generate machine-readable schemas. We
will be developing and making available both relational and XML schemas for use by agencies and
system vendors. The intention is to provide periodic updates to these as new requirements are
identified, or when new versions or subsets of the Standard are released.

In addition to the development of implementation guidelines to accompany the new version of the
Standard, the National Archives will also develop:
 updated metadata usage examples;
 use cases for specific business scenarios;
 different views of the metadata element set – for example, in addition to static object views,
    providing views of metadata required for both generic business processes and specific
    recordkeeping processes; and
 subsets of the Standard to be used for particular purposes or particular types of records – the
    Australian Government Email Metadata Standard is the first example of this kind of product.

The new version of the Recordkeeping Metadata Standard is due for release in the second quarter of

The Australian Government Email Metadata Standard (AGEMS)
In late 2004, the CIO‟s Committee (CIOC) asked the Information Interoperability Working Group
(IIWG) to address the issue of standardising the attachment of protective security markings to
emails in accordance with the Protective Security Manual (PSM) and the Defence Signals
Directorate (DSD) Australian Government Information and Communications Technology Security
Manual, ACSI 33. The main catalyst for this was a requirement to manage the transmission of
security classified emails using remote wireless technology, specifically over the Blackberry

It soon became apparent that protective security markings were not the only form of metadata that
could be usefully standardised to travel with email. The rapid uptake, over the last few years, of
email by government as a convenient way of conducting business means that standards, technical
protocols, and business processes governing the use of email often do not reflect the official nature
of transactions that now take place using the technology.

Given the informal management of email communications, the truth is that legal, accountability,
security and information management needs of the Australian Government are not being adequately
addressed in this area. The development of an email metadata standard based on government
business and information management requirements was seen as a mechanism for addressing these

The IIWG appointed a sub-committee, chaired by the National Archives, to develop a metadata
standard for Australian Government email. The sub-committee comprised members from major
stakeholder agencies – the Australian Government Information Management Office (AGIMO),
Department of Defence, DSD, Centrelink, DIMIA and ATO.

The sub-committee saw the RkMSCA as a good starting point for identifying metadata elements to
be included in the email metadata standard. The RkMSCA had already defined a common set of
recordkeeping metadata for use across the Australian Government. To define a subset from scratch
would have been both counter-productive, and a barrier to interoperability between the two

Following several iterations, the final draft of the email metadata standard contained metadata
elements for documenting:
 Agents, such as email senders and recipients (whether human or system);
 Rights management, including - but not limited to - Protective Security markings;
 Identity, including identifiers, titles/subjects and keywords;
 Email histories;
 Relationships between emails;
 File references; and
 Other useful business metadata, such as Precedence and Importance.

The set of elements incorporated in the draft standard constituted a subset of the RkMSCA element
set, with some extensions. Each element included was based on a determination of its usefulness in
the business context.

The AGEM Standard itself is largely implementation independent – it stipulates the what, rather
than the how. However, it does note that there are a number of possible means, or combinations of
means, by which metadata can travel with email. For example, it is possible to store metadata
associated with email in the:
 message header (including the „From‟, „To‟ and „Subject‟ fields);
 body of the email message itself;
 Secure MIME (S/MIME) packaging of email.

The implementation solution favoured by the National Archives is for the metadata to travel in
suitably labelled metadata fields in message headers, using header fields as defined in the IETF
Internet Mail Format Standard RFC 2822, or proprietary fields that are semantically equivalent.

RFC 2822 provides detailed descriptions of structured metadata fields for use in Internet email
headers. These fields are intended to carry most of the semantic information for email messages in
transit between organisations. The majority of existing email applications already make some use
of these structured metadata fields in message headers, or provide semantically equivalent
proprietary fields.

However, the National Archives realises that there are some technical issues inherent in going down
this path. While a number of metadata elements in AGEMS are semantically equivalent with fields
defined in RFC 2822, there are others that constitute currently unapproved extensions to RFC 2822.
There is also the issue of what happens to information in email headers at gateways – it may be
ignored, it may be changed, or it may be stripped out. This means there is work to be done in re-
engineering gateway processing protocols, so as to facilitate the automated processing of the
metadata upon despatch and receipt, as well as reconfiguring desktop interfaces of email
applications to display the relevant metadata to end users.

Some Australian Government agencies have already been working independently on solutions to
these issues, but it would be preferable if the Australian Government could work closely with
proprietary and open source software vendors, and perhaps technical standards bodies, to make the
required functionality a standard part of email applications deployed by the Government. There is
also interest in extending any Australian Government wide solution to other government
jurisdictions. This is particularly important in terms of the protective markings issue, as the PSM is
now also applicable to all State and Territory government jurisdictions which access Australian
Government resources. It also bodes well for future government-to-government (G2G)

The draft version of AGEMS was approved by the IIWG and endorsed by the CIOC earlier this
year. It is currently being supplemented with products to assist with implementation. These are
being developed in a phased manner, with the Protective Markings issue being addressed first.

AGEMS Implementation Products
Two products have so far been developed, with AGIMO taking the lead on one, and the ATO on the

The AGIMO product, Implementation Guide for Email Protective Markings for Australian
Government Agencies, presents a Subject line solution to the Protective Markings issue and
describes email client requirements, user guidelines and server behaviour rules that must be
considered in implementing the solution. It suggests that including protective markings in the
subject line is an interim solution, but one that can be easily implemented by agencies in the short
term, as they move to meet DSD‟s new ACSI 33 requirements for protective markings in email by
April 2006.

The ATO product, (Basic) Standard for Applying Australian Government Protective Markings to
Internet Email Messages, is a more technical document that defines how protective security
markings should be formatted for inter-agency email messages in both Subject line and RFC 2822
Internet Message Header Extension implementations.

Further products are planned for 2006. The National Archives and the Australian Bureau of
Statistics have agreed to become reference sites for the implementation of the entire AGEM
Standard in, respectively, MS Outlook/Exchange and IBM Domino/Lotus Notes. It is envisaged
that setting up these sites will result in the development of functional requirements for software
tools and plug-ins, that could then be made available to Australian Government agencies to assist
them with full AGEMS implementation.

AGEMS will be placed on the National Archives website prior to its formal release, probably in
December this year. The two supplementary protective markings products were approved for
release in early October, and discussions are currently taking place regarding their online
availability and links to AGEMS.

Summing Up
This session has covered challenges in the development and implementation of metadata standards
for recordkeeping in the Australian Government sector. It used as examples two Standards the
National Archives of Australia has been working on over the course of 2005 – the revision of the
1999 Recordkeeping Metadata Standard for Commonwealth Agencies (RkMSCA), and the new
Australian Government Email Metadata Standard (AGEMS).

The major challenges encountered relate to selling the relevance of metadata standards to key
audiences, making standards understandable and usable, and enabling metadata interoperability
across systems and between government organisations.

In discussing the new metadata standard developments and the implementation efforts by the
National Archives and partner agencies, this session has shown that there is a greater awareness
now – by many areas in government - of what the major issues and challenges are and the best ways
to address them. One of the most important lessons is that, in order to produce relevant,
implementable and interoperable metadata standards, there must be partnerships and ongoing
dialogue between all parties concerned – metadata standards developers, intended users and other


To top