Docstoc

General Information

Document Sample
General Information Powered By Docstoc
					         ASEMS SAFETY & ENVIRONMENTAL MANAGEMENT SYSTEMS
                           AUDIT QUESTIONS
                              Version 1.9

This is the audit question set that has been developed and agreed between ASEG, its Safety and
Environmental specialists (Advantage and Enviros) and the Functional Safety Management Offices
for use when undertaking audits of IPTs. It can also be used and tailored for IPTs to undertake
internal audits at lower levels i.e. individual projects. Whilst the majority of the questions have been
developed against the requirements of POSMS and POEMS and incorporated as Generic Safety and
Environmental questions there are some domain specific questions that will only apply to those IPTs
with Airworthiness and OME requirements.

The following gives guidance to the fill colouring used on the Audit Questions sheet:

           Management System Element Header
           Individual POSMS/POEMS procedure header
           Question grouping header

           Generic safety and environment question - these are the questions that should be asked
           as a matter of course unless they have been tailored out and marked "NO" in the
           'SCOPED' column when scoping the audit or there is compelling evidence to make it "Not
           Scoped" during the audit. See Audit Scoring guidelines below
           Generic safety and environment supplementary question - these are questions that may
           be asked to asist the auditor obtain further evidence in support of a main question. If not
           used questions are to be marked "Not Scoped" as per scoring guidelines below
           OME Domain specific question - to be used only when it has been agreed that OME
           forms a significant part of an IPT's activities. If not used questions are to be marked "Not
           Scoped" as per scoring guidelines below
           Airworthiness Domain specific question - to be used only when it has been agreed that
           Airworthiness forms a significant part of an IPT's activities. If not used questions are to be
           marked "Not Scoped" as per scoring guidelines below


         ASEMS SAFETY & ENVIRONMENTAL MANAGEMENT SYSTEMS
                      AUDIT SCORING GUIDELINES
1. The scoring system utilised within the auditing regime has been developed by ASEG

2. When developing the scope or tailoring an audit or it becomes obvious that a question does not
apply to the IPT/project being audited (e.g. because the project is not at the stage in the CADMID
cycle to make the question appropriate) then that question can be scoped out of the audit and
marked „NO‟ (indicating Not Scoped) in the „SCOPED‟ cell for that question.


3. During the audit the auditor can award a score up to the maximum for each management system
element depending on the evidence presented and the degree of compliance with the guidelines for
the element questions. This score is put into the 'Actual Score' cell on the Audit Score worksheet.


4. When all questions have been answered and the actual score for each management system
element has been put into the 'Audit Score' worksheet the Audit Level that has been achieved
according to the preset criteria shown on the „Audit Score‟ worksheet will be automatically calculated.
         ASEMS SAFETY & ENVIRONMENTAL MANAGEMENT SYSTEMS
                       AUDIT LEVEL GUIDELINES
1.    The Audit Levels from 0 to 5 are representative of an IPT's current level of safety and
environmental management thus giving CDP and CDL confidence that their legal obligations will be
met.
2.       Level 0 - The Level 0 organisation has little or no understanding of the need for and benefits
to be gained from implementing formal safety and environmental management systems. There is
little or no management commitment as safety & environmental management is not seen as a key
business risk. Resources have not been specifically allocated for safety & environmental
management hence a formal management system has not yet been implemented nor have relevant
safety & environmental processes been developed OR the organisation has failed to achieve a
satisfactory basic level of implementation in one or more of the management system areas.
3.    Level 1 - Although aware of the potential benefits of managing safety and environmental
issues, the organisation has not implemented the management systems effectively and is either
experimenting with the application of safety/environmental management or is operating management
systems with fundamental weaknesses. There is a basic level of management commitment but this
has not permeated throughout the organisation. Safety & environmental resource allocation is often
Ad-Hoc or assigned inappropriately according to past experience. Processes are being developed
but not in a structured way.
4.    Level 2 - Management commitment is permeating throughout the organisation with safety &
environmental management being seen as a key business risk although resources are only available
to cover key issues. A formal safety & environmental management system is being implemented in a
structured way and the organisation at this level has built a foundation where basic
safety/environmental management processes have been developed and implemented. The benefits
of managing safety & environmental issues is appreciated and being communicated throughout the
organisation
5.    Level 3 - The organisation is committed to effective management of safety & environmental
issues with the majority of staff aware of the benefits of and their roles and responsibilities in safety &
environmental management. A Level 3 organisation has built the management of safety and
environment into routine business processes and implemented them throughout. Processes are
formalised and widespread. Resources are allocated to cover all issues.

6.     Level 4 - Mature safety/environmental management. The Level 4 organisation shows a pro-
active approach to safety/environmental management in all project activities. As a result safety &
environmental management is an inherent routine process where information is actively used to
improve processes. The organisation fully understands its stakeholders' requirements, has plans and
arrangements in place to address their concerns and is fully committed to active engagement.
Sufficient resources are allocated.

7.      Level 5 - The organisation is very mature where safety & environmental management is
inherent in all activities and continual improvement of processes is an in-built function. The benefits
of its efforts have been realised and are actively communicated throughout the organisation. The
culture of a fully resourced Level 5 organisation is characterised by open communication and a
determination to manage safety/environmental issues at source.

As a general guide:
a.   IPTs/Projects would be expected to implement a safety and environmental management
system in order to achieve Audit Level 4 prior to equipment entering into service.
b.  IPTs with equipment already in service would be expected to have a safety and environmental
management system consistent with the requirements of Audit Level 4
       ASEMS AUDIT QUESTION SET

          Date of Audit                            Audit Reference Number               IPT/Project being audited                   Audit Scope



                                                        Lead Auditor                             Interviewee                        Audit Objective
                                                                                                                                    To evaluate the adequacy and effectiveness of safety and environmental management systems established by IPTL or Duty Holder to deliver safe and
                                                                                                                                    environmentally sound projects, equipments and services i.a.w. Legislation, MoD Corporate and Functional Domain policy
                                             List:
Scoped Proposed level              Audit      Domain ID   Audit Requirements                  Audit Question                                  General Guidance for Auditor                           Domain Specific Guidance for Auditor                    Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee             Question                 Source Reference
                                    No.
 YES           POLICY
 YES      SMP00: Project Safety Management System & Environmental Management System
                 IPTL          1.0           GEN          SEMIs                What is the IPT's policy and how does it intend to   What is being looked for here is a documented policy            Although the documented SEMS should be
           Pre audit review of                                                 comply with the SofS Policy, MoD Policy, standards   statement of intent by the IPTL defining the responsibilities   examined, the whole audit (see below) should
             documentation                                                     and legislation appertaining to the management of    and relationships, which promote a positive safety and          consider what practices are happening, not just
                                                                               acquisition safety and environment?                  environmental culture that will secure the implementation       those that are written in the SEMS documents. The
                                                                                                                                    and continued development of the IPT‟s                          complete audit will examine the extent to which the
                                                                                                                                    equipment/application/service related safety and                SEMS complies with the requirements of ASEMS
                                                                                                                                    environmental policy. This may be a separate document or        and relevant JSPs.
                                                                                                                                    incorporated into a safety and environmental plan (SEMP).
                                                                                                                                    The IPTL needs to demonstrate that there are written            Guidance for the minimum standards for the SMS
                                                                                                                                    processes and managerial structures which:                      for contractors comes from Def Stan 00-56 Part 2,
                                                                                                                                    a. Establishes and maintains management control within          8.3.1.
                                                                                                                                    the IPT.                                                        The EMS should be compliant with the requirements
                                                                                                                                    b. Promotes co-operation between the IPTL and his               of JSP418
                                                                                                                                    project managers, project managers and their                    Review this question at the end of the audit based
                                                                                                                                    stakeholders so that safety and environmental issues            on the average compliance with the rest of the
                                                                                                                                    associated with acquisition and in-service support              question set.
                                                                                                                                    becomes a collaborative effort.                                 The Safety Management System (SMS) may be
                                                                                                                                    c. Ensures communication of necessary information within        combined with the Environmental Management
                                                                                                                                    the IPT and stakeholders.                                       System (EMS) into a SEMS, or the systems may be
                                                                                                                                    d. Secures the competence of project staff.                     aligned or they may be kept separate. In all cases it
                                                                                                                                    e. Ensures compliance with legislation, MOD standards           is to be based on a Safety Case approach that
                                                                                                                                    and procedures.                                                 addresses all aspects of through life safety for the
                                                                                                                                                                                                    equipment
                                                                                                                                                                                                    JSP520 Part 1 (Policy) 1.1.4

                 IPTL          2.0           GEN                               How does the IPT policy define control of safety     Evidence within the documented system may include:              OME SMS JSP 520
           Pre audit review of                                                 and environmental aspects of acquisition and in-     a. Acknowledgement of the IPTL's S&E delegation                 IPTL letter of delegation from SofS via CDM.
             documentation                                                     service support within the IPT?                      b. Organisational structure that clearly defines roles and      JSP 520 Part 1 (Policy) 1.1.4 part1, 2.1.3, 3.6.2,
                                                                                                                                    responsibilities.                                               DEF STAN 00-56
                                                                                                                                    c. Chairmanship of the IPT‟s S&E Committee (ESC).
                                                                                                                                    d. Ownership of the IPT‟s S&E Management System
                                                                                                                                    (SEMS).
                                                                                                                                    e. Endorsement of key documents supporting key stages
                                                                                                                                    of projects.
                                                                                                                                    f. A framework for the identification of hazards, the
                                                                                                                                    assessment of risk and implementation of controls to
                                                                                                                                    mitigate the risks.
                                                                                                                                    g. Configuration Management of key documents
                                                                                                                                    h. Setting of safety and environmental targets
                                                                                                                                    i. Carrying out periodic reviews of the SEMS and the
                                                                                                                                    provision of an internal assurance programme.
                                                                                                                                    j. A commitment to continual improvement

                 IPTL          3.0           GEN                               How does the IPT policy promote co-operation       Evidence within the documented system may include                 OME SMS JSP 520
           Pre audit review of                                                 between the IPTL, his project managers,            requirements for:                                                 IPTL letters of delegation for project staff,
             documentation                                                     contractors and stakeholders?                      a. Delegation of S&E responsibilities to key staff.               specifically staff occupying safety & environmental
                                                                                                                                  b. A stakeholder file for each project                            posts.
                                                                                                                                  c. A communication strategy with stakeholders                     JSP 520 Part 1 (Policy) 7.3.1.4(i), 5.3.2.4
                                                                                                                                  d. Formal project safety and environmental meetings
                                                                                                                                  e. A RACI chart
                                                                                                                                  f. Feedback of accidents and incidents and to assess their
                                                                                                                                  impact on the equipment/service provided.
                                                                                                                                  g. Identification and management of key interfaces.
                 IPTL          4.0           GEN                               How does the IPT policy ensure communication of Evidence within the documented system should include                 Project Safety & Environmental management Plan
           Pre audit review of                                                 necessary information both within the IPT and with requirements for:                                                 (SEMP) shall include details of required
             documentation                                                     stakeholders?                                      a. A communication strategy with stakeholders for each            competences for staffs holding positions of
                                                                                                                                  project.                                                          responsibility for Safety and Environmental activities
                                                                                                                                  b. Stakeholders to be consulted and their requirements,           within the project.
                                                                                                                                  views and responsibilities taken into account during key          The SEMP will also include if required a training
                                                                                                                                  S&E activities.                                                   plan for personnel not trained in the relevant
                                                                                                                                  c. Where safety requirements are not met, shortfalls and          competencies. the requirement should also flow
                                                                                                                                  solutions are brought to the attention of users.                  down to programme contractors, to ensure the IPT
                                                                                                                                  d. Providing adequate information to potential contractors        are confident that their contractors have the relevant
                                                                                                                                  or technical partners to respond to ITT.                          experience.
                                                                                                                                  e. Ensuring changes to design or the requirements of               JSP 520 Part 1 (Policy) 7.2.2, 7.2.3, 5.3.2.4
                                                                                                                                  Customer 1 and 2 are communicated to the project safety
                                                                                                                                  officer in a timely manner.
                 IPTL          5.0           GEN                               How does the IPT policy ensure the competence of Evidence within the documented system may include                   All individuals with significant OME safety
           Pre audit review of                                                 IPT/project staff and contractors?               requirements for:                                                   management duties and those claiming to be SQEP
             documentation                                                                                                      a. Resources and framework to achieve System Safety                 (e.g. safety managers,/focal points, OME safety
                                                                                                                                Functional Competences                                              advisors, ISA's, SME's, and contracted staff), shall
                                                                                                                                b. Terms of Reference that properly reflect the roles and           be audited against the appropriate national
                                                                                                                                responsibilities of staff with delegated S&E duties.                occupational standars for explosives criteria.
                                                                                                                                c. A Training Plan                                                  JSP520 Part 1 (Policy)           7.2.3
                                                                                                                                d. Assessing the competence of contractors and technical
                                                                                                                                partners
                                                                                                                                e. Timely and appropriate instruction and advice is
                                                                                                                                provided to other authorities for the operation and support
                                                                                                                                of equipment and services.
                 IPTL          5.1           GEN                               If the contractor's safety management is not to be                                                                   Irrespective of the standard selected, each duty
           Pre audit review of                                                 conducted iaw Def Stan 00-56 how is this approach                                                                    holder shall adopt a risk based approach.with
             documentation                                                     to be justified in the safety argument?                                                                              suitable emphasis placed by the IPT on the level of
                                                                                                                                                                                                    scrutiny that is appropriate.
                                                                                                                                                                                                    JSP520 Part 1 (Policy) 3.2.4
 YES      MANAGEMENT REVIEW




       Version 1.9.2 April 2008                                                                                                                                                              Page 3 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID        Audit Requirements                        Audit Question                                 General Guidance for Auditor                         Domain Specific Guidance for Auditor                      Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                      Source Reference
                                      No.
                 IPTL          6.0             GEN              POSMS/POEMS AAP03           How have Management Reviews of the SMS &            Looking for documented evidence in line with AAP03 that
           Pre audit review of                                                              EMS or combined SEMS been undertaken?               Senior Management have reviewed the SMS & EMS or
             documentation                                                                                                                      combined SEMS for continuing suitability, adequacy and
                                                                                                                                                effectiveness.
                 IPTL          6.1             GEN              POSMS/POEMS AAP03           What is the composition of the Management           As a minimum the Review Team should be led by the IPTL
           Pre audit review of                                                              Review Team?                                        and consist of the safety/environmental focal point and
             documentation                                                                                                                      other senior management within the IPT
                 IPTL          6.2             GEN              POSMS/POEMS AAP03           What is the frequency of Management Reviews?        As a minimum reviews should take place every 3 years
           Pre audit review of                                                                                                                  although for most IPTs an annual review would be more
             documentation                                                                                                                      appropriate. Reviews should also take place prior to major
                                                                                                                                                milestones e.g. IG, MG. ISD, OSD. Also looking for a
                                                                                                                                                documented justification of the periodicity chosen.
 YES      PLANNING
 YES      Safety & Environmental Stakeholders
            S & E Manager   7.0             GEN                 POSMS SMP01                 How have the MoD and external Safety &              "Stakeholder" is defined in ASEMS as "any individual or IPT shall identify stakeholders that are engaged in
            Project Manager                                     POEMS EMP01                 Environmental Stakeholders been identified and      group concerned with or affected by the safety or       Safety & Environmental activities.
                                                                                            recorded?                                           environmental performance of an organisation ."         The OME IPTL shall ensure that a safety committee
                                                                                                                                                                                                        is established that includes co-operation of all
                                                                                                                                                Does the Project know who will be involved in all the roles
                                                                                                                                                (e.g. User, Platform IPT, Regulator, Approval Authority,relevant stakeholders including the Contractor. The
                                                                                                                                                Provider of GFE)?                                       safety committee shall oversee, review and endorse
                                                                                                                                                MoD and External Stakeholders would be eg: DEC,         safety management and safety engineering
                                                                                                                                                Customer 2, DLO, Industry, other IPTs, Subject Matter   activeties. Input from all stakeholders will be
                                                                                                                                                Experts.                                                required to enable a demonstrably safe syatem to
                                                                                                                                                                                                        be produced.
                                                                                                                                                                                                        JSP520 Part 1 (Policy)        5.3.2.4
            S & E Manager     7.1             GEN             POSMS SMP01                Are the Stakeholders' requirements for /             For instance:                                             SEA:
            Project Manager                                   POEMS EMP02                contributions of Safety & Environmental information • Safety &/or Environmental information to let them do     Emergency Preparedness - Outside of the Naval
                                                                                         known and recorded?                                  local Risk Assessments at Trials sites, Workshops;        Authority certification regime, what emergency
                                                                                                                                              • Safety &/or Environmental performance of this system as arrangements are in place e.g. oil spill.
                                                                                                                                              it will fit into a higher-level System or Platform;       The safety committee shall identify hazard owners
                                                                                                                                              • Safety &/or Environmental submission in a particular    to implement measures and to ensure effective
                                                                                                                                              format, at a specific time to allow for required          stakeholder involvement during risk management
                                                                                                                                              Safety/Environmental Approvals.                           activities.
                                                                                                                                                                                                        JSP520 Part 1 (Policy) 5.3.2.4
 YES      Safety & Environmental Legislation / Policy / Standards and Certification Requirements Identification
            S & E Manager     8.0             GEN             POSMS SMP01                How have you planned to meet the Safety &            Plans for Identification (by suitably competent persons), The OME IPT is encouraged to use
            Project Manager                                   POEMS EMP01                Environmental Legislation, Policy, Standards and Recording (in a useful manner, and not just a list), Review POSMS/POEMS to assist them in generating
                                                                                         Certification requirements that are relevant to you? (at a minimum of an annual basis) of national and         suitable and sufficient evidence within their OME
                                                                                                                                              international requirements as appropriate. Also, do they  safety case and their SMS. The OME IPT shall
                                                                                                                                              have a plan for responding to changes. How do they plan specify all mandatory controls for the safe use of
                                                                                                                                              to assure themselves of the completeness of the process. OME and cooperate with other duty holders to
                                                                                                                                              Are there any exemptions planned, and how is this to be   ensure they are contained within their SMS.
                                                                                                                                              managed?                                                  JSP520 Part 1 (Policy) 3.3.4.h.
 YES      Safety Regulator / Approval / Assurance Authority Identification
            S & E Manager     9.0             GEN             POSMS SMP01                Show how the lead Functional Safety Management For example LSSO, ADRP, SSMO, DOSG. CNNRP, Naval SEA: for example:
            Project Manager                                   POEMS EMP01                Office, Safety Regulators and/or Approval            Authorities. Has a DOSG Safety Adviser been appointed? How and where have the Key Hazards and Naval
                                                              JSP520                     authorities appropriate to the project have been     Is he/she a standing member of the following committees, Authority Certification Requirements been defined?
                                                                                         identified, recorded and engaged with.               Safety Panel, Modification panel, Development/ Post       At what stage and how has the Naval Authority
                                                                                                                                              Design Service Progress meeting                           reviewed and provided advice to the Duty Holder on
                                                                                                                                                                                                        their Safety Requirement levels and targets for
                                                                                                                                                                                                        stated capabilities or performance requirements?

                                                                                                                                                                                                              Does the Duty Holder possess a suite of Certificates
                                                                                                                                                                                                              of Safety to support justification arguments in the
                                                                                                                                                                                                              Ship Safety Case that risks associated with the
                                                                                                                                                                                                              regulated area or Key Hazards are assessed
                                                                                                                                                                                                              ALARP?

                                                                                                                                                                                                              Are procedures/processes in place to ensure liaison
                                                                                                                                                                                                              occurs between the Duty Holder, Commanding
                                                                                                                                                                                                              Officer and Naval Authorities, such that significant
                                                                                                                                                                                                              changes to the material state are reported to all
                                                                                                                                                                                                              bodies?

                                                                                                                                                                                                              Have any exemptions from MoD certification been
                                                                                                                                                                                                              issued, and if so does equivalent certification exist if
                                                                                                                                                                                                              required?IPT shall provide identification of the
                                                                                                                                                                                                              safety advisor relating to all OME rewquirements.
                                                                                                                                                                                                              Safety advisor should be a standing member of the
                                                                                                                                                                                                              project safety panel and all committees relating to
                                                                                                                                                                                                              OME safety. The IPT shall provide documented
 YES      MOD's Safety / Environmental Management Plan                                                                                                                                                        evidence of the formal tasking arrangement with
            S & E Manager     10.0          GEN                 POSMS SMP03,                Is there a SMP & Environmental Management Plan Safety Management Plan is defined in ASEMS as "A                   At the early stages of a project the OME IPT shall
           Project Manager                                      POEMS EMP06                 (EMP) that covers the whole remaining life of the document that defines the strategy for addressing safety        produce an OME Safety and Environmental
                                                                                            Project, through to Disposal?                     and documents the Safety Management System for a                Management System,This evolving body of safety
                                                                                                                                              specific project."                                              case evidence shall be used as the basis of
                                                                                                                                              This document is not to be confused with the                    successive reviews conducted by both the IPT and
                                                                                                                                              Environmental Impact Assessment Plan and                        the OSRP at key project milestones throughout the
                                                                                                                                              Environmental Management Plan (Setting Objectives and           acquisition cycle.
                                                                                                                                              Targets, EMP03 and EMP06 respectively).                         JSP520 Part 1 (Policy) 3.6.2
                                                                                                                                              Not just about the immediate future: there must be a
                                                                                                                                              Strategy defined for the whole remaining lifecycle which
                                                                                                                                              defines the equipment/system(s) capability, purpose,
                                                                                                                                              operating cycle, environment and its interfaces with other
                                                                                                                                              equipment/systems.
                                                                                                                                              This SMP & EMP could be combined as a SEMP.

            S & E Manager         10.1         GEN              POSMS SMP03,                Has the safety strategy been described and the
            Project Manager                                     POEMS EMP06                 rationale on which the safety argument is to be
                                                                                            based explained with the intended approval route?

            S & E Manager         10.2         GEN              POSMS SMP03,                How do you decide on the frequency of reviews       Looking for evidence that the SEMP is up to date and that
            Project Manager                                     POEMS EMP06                 and revisions to the plan?                          previous reviews have happened.




       Version 1.9.2 April 2008                                                                                                                                                                         Page 4 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID    Audit Requirements                         Audit Question                                 General Guidance for Auditor                         Domain Specific Guidance for Auditor                   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         10.3          GEN         POSMS SMP03,                Have any changes been made to the SEMP to            No guidance given.                                            If the equipment is new and currently following the
            Project Manager                                 SMP13                       reflect new safety & environmental risks that have                                                                 CADMID cycle the IPT shall provide a Risk Log of
                                                            POEMS EMP06,                been revealed in service?                                                                                          all perceived risks the equipment may be subjected
                                                            JSP 520                                                                                                                                        to, or even develop during programme trials and
                                                                                                                                                                                                           evaluations. If the equipment is currently inservice
                                                                                                                                                                                                           the log shall identify recent risks identified during
                                                                                                                                                                                                           service use.
                                                                                                                                                                                                           JSP 520 Part 2           5.3.5

            S & E Manager         10.4          GEN         POSMS SMP03,                Is the SEMP linked to other related documents?       E.g. Environmental Plan(s), TLMP, IPT or Project SEMS
            Project Manager                                 POEMS EMP06                                                                      documents, Contractor‟s Safety & Environmental Plan,
                                                                                                                                             Disposal Plan
            S & E Manager         10.5          GEN         POSMS SMP03,                Does the contractor's/designer's SMP/EMP
            Project Manager                                 POEMS EMP06                 correlate with and support the IPT/Project        Is the Contractor's SEMP consistent with the requirements
                                                                                        SMP/EMP?                                          within DEF-STAN 00-56?
            S & E Manager         11.0          GEN         POSMS SMP03,                Is the SEMP content consistent with POSMS and     See POSMS SMP03/G/01
            Project Manager                                 POEMS EMP06                 POEMS requirements?                               And Def Stan 00-56 Part 2 para 8.5
                                                                                                                                          Guidance on EMP likely in future revision of POEMS
            S & E Manager         11.1          GEN         POSMS SMP03,                Does the plan include Risk Assessment activities? No guidance given.                                        Project SEMP shall include detailed evidence of risk
            Project Manager                                 POEMS EMP06                                                                                                                             activities including methodologies for Qualitative
                                                            JSP 520 [modified]                                                                                                                      and Quantitative assessments. Evidence provided
                                                                                                                                                                                                    shall also include Tolerability and ALARP
                                                                                                                                                                                                    evaluations against the tolerability criteria.

                                                                                                                                                                                                           DEF STAN 00-56 (Part 1) / 3 Section 10
                                                                                                                                                                                                           DEF STAN 00-56 (Part 2) / 3 Section 10
                                                                                                                                                                                                           JSP 520(Part 1)  3.5.3.h       5.3.4.1.

            S & E Manager         11.2          GEN         POSMS SMP03,                Does the plan include a periodic review of the       The Safety & Environmental Assessments should be              OME Environmental issues shall be assessed as
            Project Manager                                 POEMS EMP06                 results of the Safety & Environmental Assessment     reviewed as a minimum whenever circumstances or               part of the risk assessment process.
                                                            JSP 430 Part 2 (modified)   process?                                             requirements change i.e. changes in the role, design          JSP 520 (Part 1)      3.5.3.h.
                                                                                                                                             intent, maintenance or operational usage profile.
            S & E Manager         11.3          GEN         POSMS SMP03,                Have the system boundaries and interfaces been       As well as system description looking for details of          Project SEMP should include system boundaries
            Project Manager                                 POEMS EMP06                 included in the system description?                  procedures for controlling integration, communicating risk    and capability boundaries in relation to the OME
                                                            JSP 520                                                                          and updating safety cases                                     system requirement.
                                                                                                                                                                                                           JSP 520 (Part 1) Policy 3.4.2.
            S & E Manager         11.4          GEN         POSMS SMP03, 2.1.1(b)       How does the plan record safety & environmental      Looking for the plan to contain a safety &/or environmental
            Project Manager                                 POEMS EMP06                 timescales, milestones and other date related        programme which is defined in ASEMS as "the part of the
                                                                                        information?                                         plan that documents safety and /or environmental
                                                                                                                                             timescales, milestones and other date related information."

            S & E Manager         11.5/OME001   OME         JSP 520                     What are the interfaces between the OME              Has a plan been documented for the integration of the         Has a plan been documented for the integration of
            Project Manager                                                             subsystem and all platforms designated to carry      OME with the platform ? Is a record of completed              the OME with the platform ? Is a record of
                                                                                        the munition and have they been agreed between       integration procedures maintained ?                           completed integration procedures maintained ?
                                                                                        the platform IPTLs and the OME IPTL?
                                                                                                                                                                                                           DEF STAN 00-56 (Part 1) / 3 Section 12
                                                                                                                                                                                                           DEF STAN 00-56 (Part 2) / 3 Section 12
                                                                                                                                                                                                           JSP 520 (Part 1)   2.3.4.

            S & E Manager         11.6/AW001    AW          JSP 553 1.39                Does the SMP specify overall platforms operations
            Project Manager                                                             safety target
            S & E Manager         11.6/AW002    AW          JSP 553 1.39                Doea the safety target specified in the SMP
            Project Manager                                                             distinguish between peacetime and hostile
                                                                                        operations?
            S & E Manager         11.6/AW003    AW          JSP 553 1.38 - 1.44         Show how the project/system safety targets have
            Project Manager                                                             been derived from the overall design criteria
            S & E Manager         12.0          GEN         POSMS SMP03,                Has the SEMP been reviewed and endorsed by the Looking for evidence that the SEMP has been authorised
            Project Manager                                 POEMS EMP06                 Safety and Environmental Committee?               by the IPT(L) (delegated authority/duty holder).

            S & E Manager         12.1          GEN         POSMS SMP03,                Has the SEMP been forwarded to all the authorities
            Project Manager                                 POEMS EMP06,                that manage equipment/systems that interface with
                                                            JSP 454 0507                the project you manage?
            S & E Manager         12.2          GEN         POSMS SMP03 3.3.2           How have you ensured that the planned Safety and POSMS (SMP03 3.3.2) requires the SMP to be accepted
            Project Manager                                 POEMS EMP06                 Environmental activities for the project are       formally by the MOD delegated authorities for
                                                                                        acceptable for support and operating authorities? Procurement, Support and Operation.
 YES      Screening & Scoping
            S & E Manager   13.0                GEN         POEMS EMP02                 How has the IPT identified the environmental          Example methodology provided in EMP02, Form                  JSP 520 (Part 1)           3.5.3.h. 5.3.6.5.b
            Project Manager                                                             aspects and potential impacts of the project over its EMP02/F/01 - Environmental Feature Matrix. This or
                                                                                        whole lifecycle?                                      similar methodology to be used. It is only necessary to
                                                                                                                                              complete columns a to i at this stage.
                                                                                                                                              Note: If a formal Environmental Impact Assessment (EIA)
                                                                                                                                              has been undertaken (EMP05), generally by consultants
                                                                                                                                              or industry, then the requirements of EMP02 should have
                                                                                                                                              been met through this process. However, EIAs may not
                                                                                                                                              cover all of the stages of the CADMID lifecycle and
                                                                                                                                              EMP02 may need to be revisited.


            S & E Manager         13.1          GEN         POEMS EMP02                 Is there a reason for any CADMID stages to be        With legacy equipment it is generally acceptable to look
            Project Manager                                                             excluded?                                            from that point forward e.g. for in-service equipment with
                                                                                                                                             no further manufacture, upgrades, major overhauls etc
                                                                                                                                             required there is often no need to compile the matrix for
                                                                                                                                             the CADM stages.
            S & E Manager         13.2          GEN         POEMS EMP02                 Can you demonstrate that the team identifying        Competency will be based on a thorough understanding of
            Project Manager                                                             aspects and potential impacts is competent to do     the project; and experience, knowledge, education and
                                                                                        so?                                                  training on environmental management.
            S & E Manager         13.3          GEN         POEMS EMP02                 What evidence is there to show that the              At a minimum the Committee should review the output to
            Project Manager                                                             Environmental Committee were involved in the         ensure that it covers all significant issues. There should
                                                                                        process?                                             be documented evidence (e.g. minutes of the Committee
                                                                                                                                             meeting) of this discussion taking place.
            S & E Manager         14.0          GEN         POEMS EMP02                 What process is in place for updating and refining   Any significant changes to the project capability; any
            Project Manager                                                             the aspects and impacts with current information?    environmental incidence; findings from research etc
                                                                                                                                             should result in a review of the Environmental Feature
                                                                                                                                             Matrix to ensure assumptions remain valid.




       Version 1.9.2 April 2008                                                                                                                                                                      Page 5 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID   Audit Requirements                    Audit Question                                 General Guidance for Auditor                               Domain Specific Guidance for Auditor      Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                Source Reference
                                      No.
            S & E Manager         15.0         GEN         POEMS EMP02          Has the IPT completed an Environmental Impact      This report should include:                                JSP 520 (Part 1)              5.3.6.5.b
            Project Manager                                                     Screening & Scoping Report for the project and the - Reference to the information sources used to compile the
                                                                                outputs utilised to inform the planning process??  matrix;
                                                                                                                                   - An overview of the main potential environmental impacts
                                                                                                                                   of the project;
                                                                                                                                   - Comment on which CADMID stages are likely to have the
                                                                                                                                   greatest environmental impact
                                                                                                                                   - Which, if any, of the life cycle stages will be excluded
                                                                                                                                   from further assessment;
                                                                                                                                   - Any other limitations or restrictions that may be placed
                                                                                                                                   on assessment requirements.

            S & E Manager         16.0         GEN         POEMS EMP02          Has the IPT prepared an initial Environmental        Where the EIA process (See EMP03 and EMP05) is
            Project Manager                                                     Impact Statement for the project?                    unlikely to be engaged until after initial gate, an initial EIS
                                                                                                                                     (Environmental Impact Statement) for the project should
                                                                                                                                     be prepared. The intention at this stage is to give an
                                                                                                                                     overview of the main issues and data available and is not
                                                                                                                                     intended to be a final EIS as detailed in EMP05.

            S & E Manager         16.1         GEN         POEMS EMP02          Does this initial EIS appear to cover all of the     Does it reflect the information in the Environmental              .
            Project Manager                                                     projects significant environmental issues?           Feature Matrix and known impacts arising from similar
                                                                                                                                     equipment / technologies, past incidents etc.
            S & E Manager         17.0         GEN         POEMS EMP02          Has the IPT completed the optional Streamlined       If some stages of CADMID have been excluded from
            Project Manager                                                     Life Cycle Assessment form EMP02/F/02 and            further study, is the IPT justified in making this decision?
                                                                                excluded some stages of CADMID from the plans
                                                                                for further study? On what basis were these
                                                                                decisions made?
 YES      Environmental Impact Assessment Plan
            S & E Manager   18.0           GEN             POEMS EMP03          Has the IPT completed an Environmental Impact     Looking for evidence that the outputs of this EIA plan are JSP 520 (Part 1)               3.5.3.h      5.3.6.5.b
           Project Manager                                                      Assessment (EIA) Plan for the project?            consistent with findings so far.
                                                                                                                                  It is likely that an Environmental Impact Assessment(s) will
                                                                                                                                  be required if significant issues have been identified or if
                                                                                                                                  there is insufficient information to determine the extent of
                                                                                                                                  any potentially significant impact.
            S & E Manager         18.1         GEN         POEMS EMP03          Has the IPT sought advice on whether or not EIA's Looking for documented evidence.                             JSP 520 (Part 1)             5.3.6.5.b.
            Project Manager                                                     are necessary?                                    Advice can be sought from SMEs, Environmental
                                                                                                                                  Committee, ASESG, Consultants, Suppliers.
                                                                                                                                  An independent judgement by the project Environmental
                                                                                                                                  Focal Point (or equivalent) may be considered too
                                                                                                                                  subjective unless the decision is clear cut.
                                                                                                                                  Any decisions should be recorded with an explanation as
                                                                                                                                  to how the decision was reached. Form EMP03/F/01 can
                                                                                                                                  be used for this purpose.

            S & E Manager         18.2         GEN         POEMS EMP03          If EIA's have been deemed necessary has the IPT      EIA's may be performed by IPT staff if there exists         JSP 520 (Part 1)           5.3.6.5.b.
            Project Manager                                                     identified who is going to be responsible for        sufficient competence and resources. With more complex
                                                                                undertaking the EIA's and how did it reach this      projects, environmentally sensitive projects or where there
                                                                                decision?                                            is a lack of internal competency and/or resources the EIA's
                                                                                                                                     may best be performed by a third party. Form EMP03/F/01
                                                                                                                                     can be used for documenting decisions.

 YES      Impact Priority Evaluation
            S & E Manager      19.0            GEN         POEMS EMP04          Has the IPT documented the methodology to be         Looking for evidence that the methodology is clear and in JSP 520 (Part 1)             5.3.6.5.b.
            Project Manager                                                     used for priority evaluation of potential            accordance with POEMS requirements as suggested in
                                                                                environmental impacts?                               EMP04.
                                                                                                                                     EMP04 provides clear guidance on a typical methodology
                                                                                                                                     that can be used. If the IPT deviates from this guidance
                                                                                                                                     then reasons need to be documented, and the proposed
                                                                                                                                     system needs to be scrutinised for clarity and
                                                                                                                                     appropriateness.
            S & E Manager         19.1         GEN         POEMS EMP04          Are all significant issues reflecting the significance There is a degree of subjectivity to all such evaluations    JSP 520 (Part 1)        5.3.6.5.b.
            Project Manager                                                     of the potential impacts covered in evaluations        and this is acceptable. However the evaluations must be
                                                                                completed by the IPT?                                  consistent. There is a tendency on occasion for some
                                                                                                                                       impacts to be downgraded as the IPT may consider them
                                                                                                                                       to be "difficult" issues which are better being 'swept under
                                                                                                                                       the carpet'.
            S & E Manager         19.2         GEN         POEMS EMP04          Did the IPT seek guidance and assistance with          In straightforward cases the IPT may complete the impact JSP 520 (Part 1)            5.3.6.5.b.
            Project Manager                                                     completing the evaluations from members of the         priority evaluation work themselves. However, in more
                                                                                environmental committee and / or contractors and complex cases it is advisable for the IPT to seek
                                                                                advisors?                                              assistance from other parties. Form EMP04/F/01 can be
                                                                                                                                       used to record any guidance and assistance given.
 YES      Environmental Impact Assessment & Reporting
            S & E Manager   20.0           GEN             POEMS EMP05          Have EIAs been completed where identified as         In form EMP02/F/02 - Streamlined Life Cycle Assessment JSP 520 (Part 1)                5.3.6.5.b.
           Project Manager                                                      required?                                            and in EMP03 the IPT will have begun the process of
                                                                                                                                     determining whether a full EIA is required. A need for an
                                                                                                                                     EIA will be reinforced if the results of the priority
                                                                                                                                     evaluation in EMP04 show significant impacts, or where it
                                                                                                                                     is difficult to assign scores due a lack of knowledge /
                                                                                                                                     understanding of the potential impacts.
            S & E Manager         20.1         GEN         POEMS EMP05          Who did you use to undertake the EIAs ?              Looking for evidence that the author (inc.contractor is     JSP 520 (Part 1)           5.3.6.5.b.
            Project Manager                                                                                                          competent.
                                                                                                                                     When choosing a suitable advisor/contractor/supplier,
                                                                                                                                     IPTs should consider the following
                                                                                                                                     competencies:
                                                                                                                                     - Previous experience of EIA and EIA processes;
                                                                                                                                     - Knowledge of the Application Domain eg naval, air or
                                                                                                                                     land systems;
                                                                                                                                     - Understanding of the environmental risks and media
                                                                                                                                     potentially affected eg previous experience of working with
                                                                                                                                     military systems;
                                                                                                                                     - Working knowledge of ISO 14001 and ISO 14040.
                                                                                                                                     An advisor/contractor/supplier should be chosen on the
                                                                                                                                     strength of their combined competencies across these
                                                                                                                                     areas.




       Version 1.9.2 April 2008                                                                                                                                                                  Page 6 of 32
     ASEMS AUDIT QUESTION SET

Scoped Proposed level              Audit     Domain ID    Audit Requirements                     Audit Question                                General Guidance for Auditor                          Domain Specific Guidance for Auditor                Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee             Question                 Source Reference
                                    No.
          S & E Manager         21.0         GEN         POEMS EMP05              Have the EIAs covered all of those stages of       Completion of EMP01 - EMP04 will have highlighted the        JSP 520 (Part 1)           5.3.6.5.b.
          Project Manager                                                         CADMID cycle where potential significant impacts   significant issues. All of these issues should be covered in
                                                                                  have been highlighted?                             the EIA. Note: It is not necessary for all the stages of
                                                                                                                                     CADMID to be included in the EIA. For example, with
                                                                                                                                     legacy equipment approaching disposal stage, it may be
                                                                                                                                     most appropriate to limit the EIA to disposal issues only.


          S & E Manager         22.0         GEN         POEMS EMP05              Have the EIAs followed the format suggested in     This Guidance Sheet is intended as a checklist of quality
          Project Manager                                                         Guidance Sheet EMP05/G/01?                         rather than content.
          S & E Manager         23.0         GEN         POEMS EMP05              Have the EIAs identified ways in which positive    The EIA and IPT should accentuate the positive impacts of
          Project Manager                                                         impacts could be optimised?                        the project. Generally, this can most easily be
                                                                                                                                     demonstrated by showing the environmental
                                                                                                                                     improvements gained from the acquisition of replacement
                                                                                                                                     technologies.
          S & E Manager         24.0         GEN         POEMS EMP05              Has the IPTL authorised all of the Environmental   The EIS is a non technical summary of the EIA Report.
          Project Manager                                                         Impact Statements?
          S & E Manager         25.0         GEN         POSMS SMP10,             How have the system Safety Requirements been       Authorities should be consulted on defining the best           Duty holder shall identify safety requirements in
          Project Manager                                JSP 430 Part 2, JSP520   derived?                                           practice, availability of standards, procedures and in the     consulation with their equipment capability
                                                                                                                                     identification of appropriate Safety Requirements.             Customer (ECC)
                                                                                                                                     Safety Requirements shall cover the entire system              Initial safety requirements shall be developed,
                                                                                                                                     throughout its Acquisition Cycle (taking into account all      ensuring safety legislation, regulations and MoD
                                                                                                                                     areas, including Capability, People, Property,                 policies are adheared to.
                                                                                                                                     Environment).                                                  JSP 520 Part 1 (Policy) 5.3.1        5.3.6.2.
                                                                                                                                                                                                    DEF STAN 00-56 (Part 1) / 3 Section 11
                                                                                                                                                                                                    DEF STAN 00-56 (Part 2) / 3 Section 11
                                                                                                                                                                                                    DEF STAN 07-85 (Part 4) / 1

          S & E Manager         25.1         GEN         POSMS SMP 10 2.1.1(b)    How have the needs of Stakeholders been            Stakeholders include Authorities for Higher-Level              JSP 520 (Part 1)        5.3.2.4
          Project Manager                                                         addressed in the project safety requirements?      Systems, Safety Regulators and Approval Authorities.
                                                                                                                                     Looking for evidence that these have been consulted from
                                                                                                                                     the earliest stages of the Project lifecycle.
          S & E Manager         25.2         GEN         POSMS SMP10,             How have Safety Requirements been set              Looking for evidence that SMP 01/F/03 the Register of          When defining Requirements, a top-level Safety
          Project Manager                                SMP01                    addressing relevant Legislation and Policy?        Safety Legislation and Other Significant Requirements has      Assessment is useful for categorising Requirements
                                                                                                                                     been used as an input to safety requirements.                  and justifying their selection as follows:
                                                                                                                                     Looking for evidence that Safety Requirements have set         • Requirements for full compliance with relevant
                                                                                                                                     goals for:                                                     legislation;
                                                                                                                                     • Safety Management                                            • Requirements to provide evidence that MOD has
                                                                                                                                     • Risk / hazard capturing techniques                           safety levels at least as good as statute where
                                                                                                                                     • Design / maintenance standards                               legislation does not apply;
                                                                                                                                     • Performance criteria                                         • Requirements to prove from first principles that
                                                                                                                                     • Human Factors Integration                                    target levels demonstrably reduce risks to ALARP
                                                                                                                                     Examples of Safety Requirements include:                       levels.
                                                                                                                                     • Safety & Environmental Management objectives or
                                                                                                                                     Performance Indicators (PIs);
                                                                                                                                     • Acceptance (success) criteria for work to be completed
                                                                                                                                     by each Acquisition milestone;
                                                                                                                                     • Defining the tolerable risk zone boundaries by
                                                                                                                                     establishing compatible risk categories, conventions and
                                                                                                                                     tolerability criteria for cascading to equipment Safety
                                                                                                                                     Cases and for integrating Safety Assessments from staff
                                                                                                                                     into the Safety Case
                                                                                                                                     • Requiring justification of the appropriateness of selected
                                                                                                                                     hazard identification, risk analysis and assessment
                                                                                                                                     techniques;
                                                                                                                                     • Selection of the upper and lower ALARP tolerability
                                                                                                                                     boundaries (e.g. risk no more than 'n' incidents in a
                                                                                                                                     prescribed time-scale);
                                                                                                                                     • Defining target Software Integrity Levels (SIL) to be met;
                                                                                                                                     • Defining best practice for an evolution e.g. where higher
                                                                                                                                     standards / targets are achievable, conversely agreeing
                                                                                                                                     industry norms or criteria, where justified e.g. emergency
          S & E Manager         25.3         GEN         POSMS SMP 10 2.1.1       How have Safety Integrity and Performance          Looking for evidence that the PHI&A was used to identify       JSP 520 (Part 1)        3.1.1.c.   7.3.1.4.j.
          Project Manager                                                         Requirements been set for any safety-related       any Safety-related functionality and that Safety
                                                                                  functionality?                                     Requirements (and Safety elements of SRD) were based
                                                                                                                                     on the tolerable Risk for failure of these functions and
                                                                                                                                     domain-specific baseline criteria (where relevant)
          S & E Manager         25.4         GEN                                  How have Safety Integrity Requirements been set    Looking for evidence that the PHI&A was used to identify
          Project Manager                                                         for any safety-related software?                   any Safety-related functionality and that Safety
                                                                                                                                     Requirements (and Safety elements of SRD) were based
                                                                                                                                     on the tolerable risk for failure of these functions and
                                                                                                                                     domain-specific baseline criteria (where relevant).


          S & E Manager         26.0         GEN         POSMS SMP10,             How have the Safety Requirements been defined Safety Requirements should be reviewed by project safety JSP 520 (Part 1)                  6.3.1.
          Project Manager                                JSP 430 Part 2           such that effective assessments can be conducted committees to assess the target levels set for the
                                                                                  against a measurable goal/target ?               requirements and the success in their achievement at
                                                                                                                                   least at agreed milestones.
                                                                                                                                   A Safety Requirement may be defined quantitatively or
                                                                                                                                   qualitatively;
                                                                                                                                   Qualitatively: Deterministic standards, demonstrated as
                                                                                                                                   good or best-practice, for a mature, well understood risk
                                                                                                                                   may be used to define the Safety Requirement. Direct
                                                                                                                                   comparison against a deterministic standard may be used
                                                                                                                                   to assess the Safety Requirement.
                                                                                                                                   Quantitatively: Numerical targets, high-level principles,
                                                                                                                                   work practices for more novel risks, often supported by
                                                                                                                                   quantitative Safety Assessment and base-lined to target
                                                                                                                                   levels which are considered best practice.




          S & E Manager         27.0         GEN         POSMS SMP10              How are Safety & Environmental Requirements        Expect high-level statements in URD but specifics in SRD JSP 520 (Part 1)             6.3.1.
          Project Manager                                                         covered in the URD and SRD?                        (not just "make it Tolerably Safe, work to Def Stan 00-56
                                                                                                                                     and produce a Safety Case").
          S & E Manager         27.1         GEN         JSP 520                  How are system safety requirements and criteria    No guidance given.                                        JSP 520 (Part 1)            6.3.1.
          Project Manager                                                         being developed and recorded within the URD?
          S & E Manager         27.2         GEN         JSP 520                  How have safety objectives set by the Central      No guidance given.                                             JSP 520 (Part 1)       6.3.1.
          Project Manager                                                         Customer in the URD been entered into the SRD?



     Version 1.9.2 April 2008                                                                                                                                                                 Page 7 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID      Audit Requirements                        Audit Question                                    General Guidance for Auditor                           Domain Specific Guidance for Auditor                  Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                   Source Reference
                                      No.
            S & E Manager         28.0         GEN           POEMS EMP03                Does the environmental management plan show            EMP should contain action plan with subtasks,                    JSP 520 (Part 1 )             3.6.2.
            Project Manager                                                             how it is planned to achieve the environmental         responsibilities, deadlines.
                                                                                        objectives and targets, is this consistent with        Suggested templates are given in POEMS:
                                                                                        POEMS requirements?                                    Form EMP06/F/01 – Setting Objectives and Targets;
                                                                                                                                               Form EMP06/F/02 – Environmental Objectives and
                                                                                                                                               Targets Register;
                                                                                                                                               Form EMP06/F/03 – Environmental Management Plan
                                                                                                                                               Record Sheet.
 YES      Risk estimation, Risk & ALARP Evaluation, Risk Reduction & Risk Acceptance / Environmental Management Plan (Setting Objectives & Targets)
 YES      Risk Estimation
            S & E Manager     29.0          GEN             POSMS SMP06               What is your process for Risk Estimation ?   Risk Estimation for Accidents (not Hazards – they have no JSP 520 (Part 1)                             5.3.3
            Project Manager                                                                                                        Harm).
                                                                                                                                   Need to demonstrate consideration for both Consequence
                                                                                                                                   (Severity) and Likelihood (Frequency) in a systematic way
                                                                                                                                   for all identified possible Accidents.


            S & E Manager         29.1         GEN           POSMS SMP06,               Where relevant, how was the Risk Class Matrix          - The process of developing The qualitative and             JSP 520 (Part 2)                        5.3.5
            Project Manager                                  JSP 430 Part 2             developed and endorsed?                                quantitative definitions for The likelihood and severity of
                                                                                                                                               hazards, and The subsequent Risk Class Matrix should be
                                                                                                                                               reviewed during The audit
                                                                                                                                               - The Risk Class Matrix should be presented to The Safety
                                                                                                                                               Committee for review and endorsement.
                                                                                                                                               - The definition of The likelihood and severity of hazards
                                                                                                                                               and accidents, and The Risk Class Matrix should be
                                                                                                                                               clearly documented in The PSMP.'
            S & E Manager         30.0         GEN           POSMS SMP06                What Risk Estimation techniques have you               Why have these techniques been chosen, how do you                JSP 520 (Part 2)                   5.3.5
            Project Manager                                                             planned to use?                                        intend to demonstrate the effectiveness, how will it be
                                                                                                                                               reviewed (independence, competence etc.), where is it
                                                                                                                                               recorded?
            S & E Manager       30.1          GEN            POSMS SMP06,               How do you plan to transfer risk data between
            Project Manager                                  Air Eng (Pol) AVSG Audit   relevant organisations?                                Commodity/Platform IPTs etc.
 YES      Criteria for Tolerability and ALARP
            S & E Manager       31.0          GEN            POSMS SMP 07 10.2.1        Have tolerability criteria been defined and recorded Criteria should be defined in advance,                             JSP 520 (Part 1)                   4.1.3
            Project Manager                                  JSP 430 Part 2             before being applied?                                - The defined Safety Requirements should be used to
                                                             Def Stan 00-56 Part 2                                                           specify the boundaries between the UNACCEPTABLE,
                                                                                                                                             ALARP OR TOLERABLE and BROADLY ACCEPTABLE
                                                                                                                                             risk regions of the ALARP triangle (e.g. risk no more than
                                                                                                                                             'n' incidents in a prescribed time-scale);
                                                                                                                                             - The Safety Requirements should be used to determine
                                                                                                                                             the qualitative and quantitative definitions for the
                                                                                                                                             likelihood and severity of hazards, and the subsequent
                                                                                                                                             Risk Class Matrix defined in the PSMP.
                                                                                                                                             - The tolerable risk zone boundaries will establish
                                                                                                                                             compatible risk categories, conventions and tolerability
                                                                                                                                             criteria for cascading to equipment Safety Cases and for
                                                                                                                                             integrating Safety Assessments.


            S & E Manager         31.1         GEN           POSMS SMP 07 10.2.1        Have tolerability criteria been benchmarked before                                                                      JSP 520 (Part 1)                   4.1.3
            Project Manager                                                             being applied?
            S & E Manager         31.2         GEN           POSMS SMP 07 10.2.3,       Do tolerability criteria consider different categories E.g. first party / trained personnel directly involved, second   Evidenc within the risk register that all categories
            Project Manager                                  JSP520                     of people who might be affected by accidents,          party / bystanders, and third party / members of the             have been included and assessed against the
                                                                                        capability, property, environment?                     general public.                                                  tolerability criteria.
                                                                                                                                                                                                                JSP 520 Part 1 (Policy) 5.3.10.3, 4.1.3.
                                                                                                                                                                                                                DEF STAN 00-56 (Part 1) / 3 10.7.1
                                                                                                                                                                                                                DEF STAN 00-56 (Part 2) / 3 10.7.1

            S & E Manager         32.0         GEN           POSMS SMP 07 10.3          How do you plan to justify your ALARP claims?          Duty holders are recommended to use predominantly           JSP 520 (Part 1)                3.2.5
            Project Manager                                  Def Stan 00-56 Part 2                                                             qualitative techniques and benchmark against the HSE's
                                                             JSP 430 Part 2                                                                    tolerability guidelines when conducting CBA or selecting
                                                                                                                                               quantitative targets or seek latest guidance.
                                                                                                                                               Quantitative safety or reliability targets should be
                                                                                                                                               considered for high risk safety critical or safety related
                                                                                                                                               areas wherever possible.
                                                                                                                                               Where a quantitative Risk Assessment has been
                                                                                                                                               considered appropriate it may be necessary to set a
                                                                                                                                               numerical requirement target appropriate to the whole unit.
                                                                                                                                               Within the ALARP principle, typical targets should be
                                                                                                                                               identified to define the boundary between the
                                                                                                                                               UNACCEPTABLE, ALARP and ACCEPTABLE REGIONS



            S & E Manager         32.1         GEN           POSMS SMP 07 10.3.2 &      Have ALARP evaluations considered existing Good For common Risks (e.g. electrical Safety) you can argue
            Project Manager                                  10.3.3                     Practice (e.g. Legislation, Approved Codes of   ALARP by compliance with Good Practice (e.g. Electrical
                                                                                        Practice)?                                      Wiring Regulations), provided that they are relevant and
                                                                                                                                        up to date.
                                                                                                                                        No need for Cost Benefit Analysis for the common or
                                                                                                                                        garden Safety issues.
            S & E Manager         32.2         GEN           POSMS SMP 07 10.3.4 et     How are ALARP evaluations made for non-                Non-common risks in this context can be taken to mean
            Project Manager                                  seq 10.3.10                common risks?                                          'novel' risks.
                                                                                                                                               If there are novel Safety issues, may require Cost Benefit
                                                                                                                                               Analysis (Cost of Improvement vs Safety Benefit).
                                                                                                                                               See Supplementary 29.3
                                                                                                                                               Have organisations such as Science Innovation
                                                                                                                                               Technology (Scientific Risk) (DGS&A) been consulted?

            S & E Manager         32.3         GEN           Air Eng (Pol) 020910       How is 'reasonably practicable' defined if the term    Evidence of factors applied when considering gross
            Project Manager                                                             is used?                                               disproportion (see SMP09/G/01 footnote)
                                                                                                                                               Evidence of Value of Preventing a Fatality (VPF)
                                                                                                                                               [POSMS SMP 07 10.3.9 suggests £1M]
                                                                                                                                               Evidence that Absolute Cost rather than Project Budgets
                                                                                                                                               are a decisive factor.
                                                                                                                                               Evidence that Cost Benefit Analysis (CBA) considers
                                                                                                                                               injuries and other costs as well as deaths.

 YES      Risk Reduction / Setting Environmental Objectives & Targets


       Version 1.9.2 April 2008                                                                                                                                                                          Page 8 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID      Audit Requirements                         Audit Question                                   General Guidance for Auditor                          Domain Specific Guidance for Auditor                Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                   Source Reference
                                      No.
            S & E Manager         33.0         GEN           POSMS SMP08                 How have the results of HI&A, Risk Estimation and     Looking for evidence that the safety and environmental
            Project Manager                                                              Evaluation, Environmental Impact Priority             assessment are not just documenting issues, but
                                                                                         Evaluation and EIAs been used to influence the        influencing the Design, Training, Procedures,
                                                                                         design solution and support arrangements to           Maintenance etc for better safety and environmental
                                                                                         reduce risk and potential environmental impacts?      performance.
                                                                                                                                               Support arrangements include technical publications,
                                                                                                                                               maintenance, training.
                                                                                                                                               Risk reduction by Hazard Controls, Accident Controls and
                                                                                                                                               setting Environmental Objectives & Targets.

            S & E Manager         33.1         GEN           POSMS SMP08                 Has Risk and Environmental Impact Reduction           Cannot claim that Risk is ALARP unless the designers /
            Project Manager                                                              been considered from early stages of the Project?     Project have actively looked for ways to improve safety.
                                                                                                                                               This question is most relevant at pre- main gate life-
                                                                                                                                               cycle stages.

            S & E Manager         33.2         GEN           POSMS SMP 08 7.2.1,         How have you decided which risk reduction and         The project should demonstrate awareness of order of
            Project Manager                                  POEMS EMP06                 environmental impact reduction (through setting       precedence of the following:
                                                                                         environmental objectives and targets) measures to     • Eliminate the Hazard (e.g by re-specification or re-
                                                                                         adopt?                                                design).
                                                                                                                                               • Reduce the risk by implementing engineered mitigation
                                                                                                                                               strategies.
                                                                                                                                               • Reduce the risk by implementing mitigation strategies
                                                                                                                                               based on human factors.
                                                                                                                                               Environmental objectives and targets should consider the
                                                                                                                                               following:
                                                                                                                                               - Adverse environmental impacts
                                                                                                                                               - Positive environmental impacts
                                                                                                                                               - Relevant environmental Standards
                                                                                                                                               - Stakeholder concerns
                                                                                                                                               - Technical Issues
                                                                                                                                               - Operational requirements
                                                                                                                                               - Financial cost

            S & E Manager         33.4         GEN           POEMS EMP06                 Has justification for the set objectives and targets
            Project Manager                                                              been documented?
            S & E Manager         33.5         GEN           ES (Land)                   How do you plan to demonstrate that all identified No guidance given.
            Project Manager                                                              mitigation has been undertaken on identified
                                                                                         potential accidents, hazards and significant
                                                                                         environmental impacts?
            S & E Manager         33.6         GEN           POSMS SMP05                 How do you plan to revisit the HI&A to examine the Changing the system (adding a new Control) may
            Project Manager                                                              safety effects of risk reduction measures selected? introduce new hazards or change the risks of other
                                                                                                                                                 hazards, this should be recognised by the IPT and
                                                                                                                                                 activities to manage it should be planned where it has not
                                                                                                                                                 already taken place.
            S & E Manager         33.7         GEN           JSP 454 0510                What plans are in place to manage agreement and
            Project Manager                                                              resolution with the operating authorities if there is a
                                                                                         possible conflict between safety objectives and         e.g. Are there documented internal procedures, or agreed
                                                                                         operational requirements?                               CSA's etc. with the OU?
 YES      IMPLEMENTATION AND OPERATION
 YES      Safety & Environmental Resources
            S & E Manager   34.0           GEN               POSMS SMP01,                How does your project / organisation ensure that      A resource plan is required to ensure there is adequate
            Project Manager                                  SMP03,                      the necessary resources are available to meet         provision (e.g. sufficient funds) to meet management
                                                             JSP 430 Part 2 (modified)   management objectives for the SEMS?                   objectives in terms of intramural capital, training costs and
                                                                                                                                               contractor support where necessary, links to TLMP.

            S & E Manager         35.0         GEN           JSP 553                     Has The Director Equipment Capability (DEC) and       Has the Director Equipment Capability (DEC) and the IPTL
            Project Manager                                                              the IPTL committed sufficient funds to ensure a       adequately resourced essential safety and environmental
                                                                                         safe design in accordance with the ALARP              analysis activities and corrective actions arising from
                                                                                         Principle and considering environmental issues ?      them?

 YES      Safety & Environmental Management Organisation
            S & E Manager   36.0          GEN            POSMS SMP03 2.1.1(a)            How do you ensure that safety & environmental          Evidence that the safety & environmental management
            Project Manager                                                              responsibilities are recognised and properly          structure and organisation is defined e.g. in the plan &/or
                                                                                         allocated ?                                           management system.
            S & E Manager     36.1          GEN          JSP 430 Part 4                  How are lines of communication defined ?              No guidance given.
           Project Manager
 YES      Record of Safety & Environmental Management Responsibilities and Accountabilities
            S & E Manager     37.0          GEN          POSMS SMP03               How has the Project established the safety &                Who does what on the Project for Safety & Environmental
           Project Manager                                                         environmental responsibilities for the different            Management (clear on Interfaces and Authority)?
                                                                                   authorities involved ?                                      SMP03 suggests use of a RACI chart to define roles for
                                                                                                                                               the safety activities although equivalent techniques are
                                                                                                                                               acceptable.
                                                                                                                                               RACI: Responsible, Accountable, Consulted, Informed
                                                                                                                                               Documented SMS or Customer/Supplier Agreements are
                                                                                                                                               examples of similar recording methods.
                                                                                                                                               EMP01/F/03 in POEMS can be used to record the project
                                                                                                                                               environmental responsibilities.



            S & E Manager         37.1         GEN           POSMS SMP03,                Does the Safety Management Plan define               No guidance given.                                               Visability of risk management stakeholders, copies
            Project Manager                                  JSP 520                     activities, and personnel responsibilities necessary                                                                  of meeting relating to risk.
                                                                                         to ensure that safety risks remain ALARP                                                                              JSP 520 Part 1 (Policy) 3.4.4, 3.4.5
                                                                                         throughout the life of the store?                                                                                     JSP 520 Part 2 (Guidance) 5.3.3
                                                                                                                                                                                                               DEF STAN 00-56 (Part 1) / 4 Section 10
                                                                                                                                                                                                               DEF STAN 00-56 (Part 2) / 4 Section 10
 YES      Record of Safety & Environmental Management Interfaces
            S & E Manager     38.0          GEN            POSMS SMP01,                  How are safety & environmental management             Look for evidence that interfaces are recognised, defined
           Project Manager                                 SMP03,                        interfaces with other organisations managed?          and agreed with all parties. This may be recorded in the
                                                           POEMS EMP01,                                                                        plan &/or management system documents.
                                                           Air Eng (Pol)                                                                       POEMS EMP01 - Stakeholder & Standards Identification.
                                                           020910[modified]                                                                    Procedure introduces 'Communications Plan' which will
                                                                                                                                               cover such interfaces.




       Version 1.9.2 April 2008                                                                                                                                                                         Page 9 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID    Audit Requirements                        Audit Question                                  General Guidance for Auditor                       Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                 Source Reference
                                      No.
            S & E Manager         39.0         GEN         POSMS/POEMS SSP01          How are the relevant safety and environmental         E.g. communications plan - description of checks on
            Project Manager                                POEMS EMP01                Legislation, Policy, Standards and Certification      communications of signals to ensure that they are
                                                           Air Eng (Pol)              Requirements communicated to Stakeholders and         transmitted to all appropriate stakeholders by the central
                                                           020910[modified]           external suppliers?                                   communication function.
                                                                                                                                            Evidence of formalised transfer of data to/from other
                                                                                                                                            organisations (eg IPTs Strike Command ATC etc.)
                                                                                                                                            External suppliers are e.g. Industry, Commodity IPTs, GS
                                                                                                                                            IPT and DFG

 YES      Safety Management Accountability and Delegation
            S & E Manager   40.0            GEN           SEMIs,                      Is a letter of delegation (or terms of reference)     No guidance given.
            Project Manager                               ES (Land)                   recorded for the authority for managing equipment
                                                                                      safety and is it up to date?
            S & E Manager         40.1         GEN         SEMIs,                     Has the IPTL issued formal letters of sub-
            Project Manager                                JSP 553                    delegation to competent individuals and are they
                                                                                      up to date?
            S & E Manager         40.2         GEN         Air Eng (Pol) 020910       Are safety delegations reviewed after a significant   No guidance given.
            Project Manager                                [modified]                 organisation change has taken place ?

 YES      IPT / Project Safety Manager and Environmental Manager
            S & E Manager      41.0          GEN            POSMS SMP01               Has an IPT Safety and Environmental Manager           For IPT (not contractor).
            Project Manager                                 POEMS EMP01               with appropriate SM/EM Competence been                Not necessarily full time on Safety & Environment.
                                                                                      appointed for this Project?                           Competence in System Safety, not SHEF.
                                                                                                                                            Relevant Domain Safety experience (e.g. Ordnance)?
                                                                                                                                            Evidence of working knowledge of ISO14001, 14040.

 YES      Document and Record Control
            S & E Manager  42.0                GEN         POSMS/POEMS SSP03          What system do you operate to ensure effective        Looking for evidence of:
           Project Manager                                                            Document Control ?                                    a. an assigned document control co-ordinator(s) with
                                                                                                                                            appropriate qualifications and resources
                                                                                                                                            b. guidelines for determining documents requiring control
                                                                                                                                            c. labelling standards
                                                                                                                                            d. information provided to appropriate personnel
                                                                                                                                            e. an up to date record of who receives controlled
                                                                                                                                            documents and which documents they hold
                                                                                                                                            f. legible and identifiable records
                                                                                                                                            g. periodic review and revision (as necessary) of
                                                                                                                                            controlled documents, approved by authorised personnel
                                                                                                                                            prior to issue
                                                                                                                                            h. current versions of controlled documents available
                                                                                                                                            where required
                                                                                                                                            i. prompt removal of obsolete documents from points of
                                                                                                                                            issue and use
                                                                                                                                            j. appropriate personnel informed of and provided with
                                                                                                                                            document changes when they occur
                                                                                                                                            k. review of adequacy and modifications made at
                                                                                                                                            appropriate times e.g. annually, when major accidents /
                                                                                                                                            incidents occur which identify document control as a
                                                                                                                                            cause.


            S & E Manager         42.1         GEN         POSMS/POEMS SSP03          How do you classify and safeguard the records
            Project Manager                                                           which are identified as vital?
            S & E Manager         42.2         GEN         POSMS/POEMS SSP03          How do you store records for long-term (30years) Looking for both vital records and controlled documents.
            Project Manager                                                           so they are easily retrievable and protected against Looking for evidence that a retention time has been
                                                                                      loss?                                                defined.

            S & E Manager         42.3/AW004   AW          JSP 553 6.1.2, 6.5, 6.16   Annex J 6. Aircraft Document Set (ADS). Ensure        No guidance given.
            Project Manager                                                           that the IPT has undertaken final production and
                                                                                      validation of the ADS, which includes:
                                                                                      • A/Crew Manual
                                                                                      • Flt Reference Cards
                                                                                      • Operating Data Manual
                                                                                      • Support Policy Statement
                                                                                      • Statement of Operating Intent & Usage (SOIU)
                                                                                      • Engineering APs.

            S & E Manager         42.3/AW005   AW          JSP 553 Annex S Appdx 1    Who maintains the Master Record Index and is it       No guidance given.
            Project Manager                                Part G1,                   iaw JSP553?
                                                           Annex J Appdx 1 JA1.13
 YES      Contractor Safety & Environmental Competence
            S & E Manager     43.0           GEN           POSMS SMP10 2.1.3          How has the Safety & Environmental Competence         Not just Domain competence, but Safety & Environmental
           Project Manager                                                            of Contractors [competent organisation and            Management. Have they written Safety Cases before?
                                                                                      competent personnel] been considered when             Are they familiar with environmental management
                                                                                      awarding contracts?                                   principles? Do they understand that safety and
                                                                                                                                            environmental issues must influence the solution, not just
                                                                                                                                            be another documentary deliverable?


            S & E Manager         43.1         GEN         POSMS SMP10 2.1.3,         Are the safety responsibilities of contractors, and
            Project Manager                                JSP 454 0515               other third parties, involved in the project(s) clearly
                                                                                      defined in the contract?
            S & E Manager         43.2         GEN         POSMS SMP10,               What procedures are in place to check that              No guidance given.
            Project Manager                                Air Eng (Pol) 020910       external suppliers (eg Industry, Commodity IPTs,
                                                                                      GS IPT and DFG) comply with the safety and
                                                                                      environmental requirements put on them?
            S & E Manager         43.3/AW006   AW          JSP 553 3.24               How was the competency of the organisation              No guidance given.
            Project Manager                                                           selected to design, build and test the
                                                                                      aircraft/equipment determined?
 YES      Training, Awareness & Competences




       Version 1.9.2 April 2008                                                                                                                                                                    Page 10 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID    Audit Requirements                        Audit Question                                   General Guidance for Auditor                      Domain Specific Guidance for Auditor                 Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                 Source Reference
                                      No.
            S & E Manager         44.0         GEN         POSMS/POEMS SSP02,         Can it be shown that the staff occupying key safety No guidance given.                                            Detailed evidence within the SEMP, TLMP that
            Project Manager                                POSMS SMP01, SMP03,        and environmental posts are Suitably Qualified and                                                                individuals occupying S&E posts are suitably
                                                           JSP 520                    Experienced?                                                                                                      qualified and have attended the relevant courses.
                                                                                                                                                                                                        Identify individuals and provide details of their
                                                                                                                                                                                                        specific experience
                                                                                                                                                                                                        Describe the procedures to monitor and support
                                                                                                                                                                                                        individuals that are not deemed to be suitably
                                                                                                                                                                                                        qualified and experienced within the S&E field.
                                                                                                                                                                                                        Identification of courses that are available to the
                                                                                                                                                                                                        relevant staff.
                                                                                                                                                                                                        All the above for external contractors providing
                                                                                                                                                                                                        support to the programme.
                                                                                                                                                                                                        JSP 520 Part 1 (Policy) 7.1 - 7.3
                                                                                                                                                                                                        DEF STAN 00-56 (Part 1) / 4 7.1 - 7.4
                                                                                                                                                                                                        DEF STAN 00-56 (Part 2) / 4 7.1 - 7.4

            S & E Manager         44.1         GEN         JSP 553                    Has the IPT issued formal letters of delegation to     No guidance given.
            Project Manager                                                           competent individuals?
            S & E Manager         44.2         GEN         JSP 520                    Do safety / environmental related posts have           No guidance given.
            Project Manager                                                           defined competences?
            S & E Manager         44.3         GEN         Air Eng (Pol) 020910       How, and by whom, is it decided if a member of         Looking for evidence of Training Needs Analysis, safety
            Project Manager                                                           staff meets the competency requirements for safety     training plan, work-life experience etc.
                                                                                      / environmental significant roles?
            S & E Manager         44.4         GEN         JSP 520 [modified]         Do the job descriptions of personnel delegated with    No guidance given.
            Project Manager                                                           safety / environmental authority identify
                                                                                      competences required with associated training
                                                                                      requirements?
            S & E Manager         44.5         GEN         Air Eng (Pol) 020910       How often, and by whom, are staff competencies         No guidance given.
            Project Manager                                                           reviewed to ensure that the staff remain competent
                                                                                      for safety / environmentally significant roles?

            S & E Manager         44.6         GEN         JSP 520                    How are arrangements defined for supervision of     No guidance given.
            Project Manager                                                           staff who have not yet satisfactorily completed the
                                                                                      required training?
            S & E Manager                      AW          JSP 553 3.16 - 3.21, Annex Has the IPTL ensured that all individuals receiving No guidance given.
            Project Manager                                C                          airworthiness delegation are competent, aware of
                                                                                      the extent of their responsibilities and have
                                                                                      accepted the areas of delegated responsibility
                            44.7/AW007
 YES      Safety & Environmental Committee
            S & E Manager   45.0           GEN             POSMS SMP02                Has a Project Safety Committee (PSC) and               The Safety and Environmental Committees can be            JSP 520 Part 1 (Policy)                      5.3.2.4
            Project Manager                                POEMS EMP01                Environmental Committee (or similar body serving       combined or kept separate. Safety Committee is defined
                                                                                      this purpose) been formed?                             in ASEMS as "a group of stakeholders that exercises,
                                                                                                                                             oversees, reviews and endorses safety management and
                                                                                                                                             safety engineering activities."
                                                                                                                                             Ask about both IPT/Project‟s Committee, and Contractor‟s
                                                                                                                                             PSC (if separate). The members of the Environmental
                                                                                                                                             Committee should include representatives of the main
                                                                                                                                             MOD stakeholders and those having particular expertise
                                                                                                                                             relevant to the equipment or service being acquired, eg
                                                                                                                                             SMEs (Subject Matter Experts). Members of the
                                                                                                                                             Committee external to MOD may comprise industry or
                                                                                                                                             scientific experts, consultants or academics who can
                                                                                                                                             provide advice to the IPT on environmental issues related
                                                                                                                                             to the project.


            S & E Manager         45.1         GEN         POSMS SMP02                Where appropriate have Safety and/or
            Project Manager                                POEMS EMP01                Environmental Committees been formed to set the
                                                           JSP 454 0506               policy and strategy for similar groups of projects
                                                                                      and oversee the activities of individual project
                                                                                      safety/environmental committees?
            S & E Manager         45.2         GEN         POSMS SMP02                Can you demonstrate that Suitably Qualified and        E.g. Minutes of Committee meetings and compare
            Project Manager                                POEMS EMP01                Experienced People (SQEP) Stakeholders and             attendees with stakeholders.
                                                                                      Subject Matter Experts attend the PSC &                Evidence that appropriate level of stakeholder
                                                                                      Environmental Committee meetings?                      representation is maintained.
            S & E Manager         45.3         GEN         POSMS SMP02                Are the PSC and Environmental Committees               If not IPTL the project should provide evidence of where
            Project Manager                                POEMS EMP01                Chaired by the IPTL or another member of the IPT       this delegation is recorded.
                                                                                      with a recorded appropriate authority (Letter of
                                                                                      Delegation or explicit TORs) ?
            S & E Manager         45.4         GEN         POSMS SMP02                How and where are the TORs defined?                    Compare PSC TORs with SMP02/G/01 example TORs
            Project Manager                                POEMS EMP01                                                                       (Section 4 indicates membership)
            S & E Manager         46.0         GEN         POSMS SMP02                Is there evidence to show that the PSC &               See supplementary questions below.                         Is the PSP functioning correctly and are decisions
            Project Manager                                POEMS EMP01                Environmental Committee is operating properly ?                                                                   made supported by sound methodology in relation
                                                           JSP 520                                                                                                                                      to Safety & environmental issues surounding system
                                                                                                                                                                                                        OME.
            S & E Manager         46.1         GEN         POSMS SMP02                Are records of PSC & Environmental Committee           Is there a time limit for actions to be completed?
            Project Manager                                POEMS EMP01                meetings kept to show how safety & environmental       Do the minutes show how previous actions have been
                                                           ES (Land)                  issues have been addressed, an audit trail of          closed?
                                                                                      discussions, advice, decisions and timely              How are all outstanding actions from the various
                                                                                      completion of actions arising?                         committee meetings tracked?
            S & E Manager         46.2         GEN         POSMS SMP02                Is there a mechanism in place to review and revise     No guidance given.
            Project Manager                                POEMS EMP01                the Environmental and Safety Cases when
                                                                                      changes occur that affect the project or information
                                                                                      available?
            S & E Manager         46.3         GEN         POSMS SMP02                Does the Committee review the Environmental /          No guidance given.
            Project Manager                                POEMS EMP01                Safety Cases at set intervals?
            S & E Manager         47.0         GEN         POSMS SMP02                How do you decide on the frequency of PSC &            No guidance given.
            Project Manager                                POEMS EMP01                Environmental Committee meetings, given the
                                                                                      nature of the Project, stage in lifecycle etc ?
 YES      Safety Requirements & Contracts
 YES      Types of Requirements
            S & E Manager   47.1/AW008         AW          JSP 553 Annex C1 para 4, Which design standards have been specified in the
            Project Manager                                Annex E 1.2              contract?
            S & E Manager   47.1/AW009         AW          JSP 553                  How were the design standards selected?
            Project Manager
            S & E Manager   47.1/AW010         AW          JSP 553 Annex C1 para 8 Has the appropriate authorisation been obtained to
            Project Manager                                                        confirm the selection?



       Version 1.9.2 April 2008                                                                                                                                                                    Page 11 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID   Audit Requirements                    Audit Question                                     General Guidance for Auditor                          Domain Specific Guidance for Auditor                 Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                 Source Reference
                                      No.
            S & E Manager         47.1/AW011    AW          JSP 553 1.39         The cumulative probability of the loss of an aircraft No guidance given.
            Project Manager                                                      due to a technical fault and the cumulative
                                                                                 probability of a technical failure of the aircraft
                                                                                 (inclusive of its systems, structure and stores)
                                                                                 which could result in the death of any air crew or
                                                                                 passengers, should both be assessed to be of the
                                                                                 order of one in a million per flying hour (probability
                                                                                 of occurrence 1 x 10-6 per flying hour) when
                                                                                 operated within the conditions used for the
                                                                                 airworthiness demonstration.
            S & E Manager         47.1/AW012    AW          JSP 553 1.39         Aircraft derived from civil passenger aircraft and     No guidance given.
            Project Manager                                                      used by the MoD in the passenger carrying airliner
                                                                                 role should meet a higher standard of safety. Such
                                                                                 aircraft may be considered to be airworthy if the
                                                                                 cumulative probability of the loss of the aircraft due
                                                                                 to a technical fault and the cumulative probability of
                                                                                 a technical failure of the aircraft (inclusive of its
                                                                                 systems, structure and stores) which could result in
                                                                                 the death of any aircrew or passengers, are both
                                                                                 assessed to be of the order of one in ten million per
                                                                                 flying hour (probability of occurrence 1 x 10 7 per
                                                                                 flying hour) when operated within the conditions
                                                                                 used for the airworthiness demonstration.



 YES      Hazard Identification & Analysis and Risk Estimation
            S & E Manager      48.0           GEN             POSMS SMP04        Has Preliminary Hazard Identification (PHI) taken         This is a coarse level of hazard identification
           Project Manager                                                       place for this project (covering all candidate            commensurate with maturity of the system (e.g. at
                                                                                 technologies/solutions if applicable)?                    Concept).
            S & E Manager         48.1          GEN         POSMS SMP04,         How did PHI explore hazards due to the required           Seeking evidence that functional failure analysis or similar   SEA: to include
            Project Manager                                 JSP 430 Part 2       capability/functionality as well as design solution (if   technique has been used – don‟t have to know the design        '- A review of historical information including:
                                                                                 known at that stage)?                                     solution or even technologies to understand some of these      1. DRACAS reports
                                                                                                                                           big issues.                                                    2. Ship's incident investigations
                                                                                                                                                                                                          3. Formal Boards of Inquiry
                                                                                                                                                                                                          4. CHASP database
                                                                                                                                                                                                          5. OpDefs
                                                                                                                                                                                                          6. Phoenix Fire and Flood Database
                                                                                                                                                                                                          7. Naval Accident Register
                                                                                                                                                                                                          8. Near Miss Reports

            S & E Manager         48.2          GEN         POSMS SMP04          Can you demonstrate that SQEP Stakeholders and Users, Operating Authority, Higher-level and Lower-level
            Project Manager                                                      Subject Matter Experts were involved in PHI?   Systems, Project, Technology experts etc.
                                                                                                                                Note: As line managers, an Operating Authority has
                                                                                                                                responsibility for the identification of hazards associated
                                                                                                                                with system operation.
                                                                                                                                Note: As Senior Managers they are also responsible for
                                                                                                                                the review of Safety Assessments and the audit of Safety
                                                                                                                                Requirements and SMS.

            S & E Manager         48.3          GEN         POSMS SMP04,         Can you demonstrate that the Hazard Identification - Operational profiles, maintenance, operator
            Project Manager                                 JSP 430 Part 2       team had a sound understanding of the system ?     competencies within a given Functional Environment;
                                                                                                                                    Note: not all team members require this.
            S & E Manager         48.4          GEN         POSMS SMP04          Can you demonstrate that the Hazard Identification - The application and limitations of selected HAZID
            Project Manager                                                      team had a sound understanding of the hazard       techniques; Note: not all team members require this.
                                                                                 identification process?
            S & E Manager         48.5          GEN         POSMS SMP04          Can you demonstrate that the Hazard Identification - The existing and/or commonly known Hazards of this or
            Project Manager                                                      team had a sound understanding of the              similar systems; Note: not all team members require this.
                                                                                 environment, equipment in service usage and its
                                                                                 hazards?
            S & E Manager         48.6/OME002   OME         JSP 520              If this is a new OME requirement; does it comply   If not; how will non-compliance be justified? If this is              If the equipment / system is new the OME IPT shall
            Project Manager                                                      with STANAG 4439?                                  legacy OME is it being reviewed to identify opportunities to          provide evidence that it has been assessed against
                                                                                                                                    achieve IM compliance? What methodologies have been                   STANAG 4439 to ensure it complies with the
                                                                                                                                    adopted for Design Assessments?                                       Insensitive munitions (IM) requirements. For
                                                                                                                                                                                                          equipment that does not comply with the IM
                                                                                                                                                                                                          requirements, dispensation should be sought by the
                                                                                                                                                                                                          relevant authority.
                                                                                                                                                                                                          If the store is classed as a legacy system, the OME
                                                                                                                                                                                                          IPT shall provide evidence that the store is under
                                                                                                                                                                                                          review to identify opportunities for IM insersion.
                                                                                                                                                                                                          JSP 520 Part 1 (Policy) 3.8
                                                                                                                                                                                                          JSP 520 Part 2 (Guidance) 5.3.6.4(e)

            S & E Manager         49.0          GEN         POSMS SMP04, SMP05   Was Preliminary Hazard Identification and Analysis More than just listing Hazards: including Hazard Analysis
            Project Manager                                                      (PHI&A) used to identify and examine possible      to examine Cause - Consequences.
                                                                                 Accidents and Accident Sequences for the Project?

            S & E Manager         49.1          GEN         POSMS SMP04          How were Hazard Analysis Techniques chosen?
            Project Manager
            S & E Manager         49.2          GEN         POSMS SMP04          How are the results of PHI&A recorded?             Looking for evidence that records are readily auditable
            Project Manager                                                                                                         e.g. through using Hazard log &/or safety case.
            S & E Manager         49.3          GEN         POSMS SMP04          Did PHI&A cover all lifecycle stages and operating Use under all levels of threat e.g. Peacetime, Training,
            Project Manager                                                      profiles, including storage, commissioning, use,   Operations Other Than War, Warfighting.
                                                                                 emergency situations and disposal ?

            S & E Manager         49.4          GEN         POSMS SMP04          How has the PHI&A considered human error as a             No guidance given.
            Project Manager                                                      cause of hazards?                                         Add links to software questions
            S & E Manager         49.5          GEN         POSMS SMP04          How has the PHI&A considered Software (failure)           Looking for evidence that hazard coverage of effect
            Project Manager                                                      as a cause of hazards?                                    software failure has on a system has been addressed.
                                                                                                                                           Looking for evidence of setting requirements for software
                                                                                                                                           integrity.
            S & E Manager         49.6          GEN         POSMS SMP04          How has the PHI&A considered external factors
            Project Manager                                                      (e.g. interfacing systems, environment, physical
                                                                                 location) as a cause of hazards?
            S & E Manager         49.7          GEN         POSMS SMP04          How does the PHI&A consider acute and chronic             Examples: Acute effects - arising suddenly / short term,
            Project Manager                                                      health effects?                                           and Chronic effects - developing slowly / long term e.g.
                                                                                                                                           RSI, hearing damage




       Version 1.9.2 April 2008                                                                                                                                                                   Page 12 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID    Audit Requirements                    Audit Question                                     General Guidance for Auditor                         Domain Specific Guidance for Auditor     Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         49.8/AW013    AW          JSP 553 2.22, 4.50     Has Safety analysis been carried out on Human            No guidance given.
            Project Manager                                                        error rates relevant to the tasks to be carried out by
                                                                                   the aircrew, maintenance staff and other personnel
                                                                                   who could affect airworthiness?
            S & E Manager         49.8/AW014    AW          JSP 553 2.22           Has the contribution of human error to                   No guidance given.
            Project Manager                                                        airworthiness been assessed and reduced to
                                                                                   ALARP
            S & E Manager         49.8/AW015    AW          JSP 553 4.37 - 4.46,   Have weapons installations been managed iaw              Def Stan 00-970 defines an Aircraft Armaments
            Project Manager                                 Annex S Appdx 1        JSP 553 and Def Stan 00-970?                             (Weapons) Installation as comprising aircraft weapons
                                                                                                                                            and associated equipment, including electrical circuits and
                                                                                                                                            software, concerned with the carriage, monitoring, control,
                                                                                                                                            release, firing or jettison of the stores. Where stores are
                                                                                                                                            carried internally, all mechanisms required for the
                                                                                                                                            operation of weapon bay doors are considered part of the
                                                                                                                                            weapons installation.

            S & E Manager         49.8/AW016    AW          JSP 553 4.34 - 4.39,   Have hazards inherent with the installation,
            Project Manager                                 Annex S                carriage or operation of explosive or pyrotechnic
                                                                                   devices been assessed and minitgated?
 YES      Hazard Analysis
            S & E Manager         50.0          GEN         POSMS SMP05            How are the PHI&A results updated and refined            Examples of current information might include detailed
           Project Manager                                                         with current information ?                               design, detailed functional description, newly identified
                                                                                                                                            hazards, operating experience
            S & E Manager         50.1          GEN         POSMS SMP05            How have you decided on the Hazard Analysis              Techniques commensurate with complexity, technology,
            Project Manager                                                        techniques to use ?                                      risk etc
            S & E Manager         50.2          GEN         POSMS SMP05            Does the hazard analysis include safety                  Assessment to include reliability data on systems and
            Project Manager                                                        assessments of operation and maintenance                 components
                                                                                   procedures?
            S & E Manager         50.3          GEN         POSMS SMP05            Was software safety analysis (SSA) carried out to
            Project Manager                                                        analyse hazardous failures of software?
            S & E Manager         50.4          GEN         POSMS SMP05            Did the SSA ensure that safety requirements are
            Project Manager                                                        correctly implemented?
            S & E Manager         50.5          GEN         POSMS SMP05,           Has safety analysis been carried out on the safety       No guidance given.
            Project Manager                                 JSP 553                performance of the human-machine interface?

            S & E Manager         50.6          GEN         POSMS SMP05            Have the results of HI&A been reviewed by those     Review may be technical assurance, hazard analysis
            Project Manager                                                        with appropriate knowledge and experience?          (perhaps by Safety Committee). Review should be carried
                                                                                                                                       out by suitably independent people.
            S & E Manager         50.7/OME003   OME         JSP 520                Are all energetic components defined?               No guidance given.                                      OME IPT should be in posession of Hazard Data
            Project Manager                                                                                                                                                                    Sheets foe all energetic materials used in the
                                                                                                                                                                                               production of the equipment / system.
                                                                                                                                                                                               For platform IPTs that have OME sub-systems fitted
                                                                                                                                                                                               the platform IPT should have documented evidence
                                                                                                                                                                                               of the energetics that will be carried, stored and
                                                                                                                                                                                               operated by the platform.         JSP520(Part 1)
                                                                                                                                                                                               5.3.1.5.
            S & E Manager         50.7/OME004   OME         JSP 520                What is the continuing procedure to assess the life How is the shelf life of energetics tracked during a    JSP 520 Part 1 (Policy) 3.7, 5.3.11.2
            Project Manager                                 JSP 482                of energetic components under actual operational munition's life?                                           JSP 520 Part 2 (Guidance) 5.3.6.4(b)
                                                                                   and storage conditions ?
 YES      Risk Acceptance
            S & E Manager         51.0          GEN         POSMS SMP 09 7.2.2     What is the process for reviewing and approving          Must be both Tolerable and ALARP (it is possible for a        JSP 520 (Part 1)        5.2.
            Project Manager                                                        risks as “tolerable and ALARP” ?                         risk to be ALARP but unacceptable and this must be
                                                                                                                                            reduced further).
                                                                                                                                            Looking for evidence of who is involved in the review and
                                                                                                                                            approval.

            S & E Manager         51.1          GEN         POSMS SMP 09/G/01      Are risks approved by an appropriate authority    For definition of "appropriate authority" see "Management JSP 520 (Part 1)
            Project Manager                                                        (management level), commensurate with risk level? Level for Acceptance of Risk" in SMP09/G/01

            S & E Manager         51.2          GEN         POSMS SMP 09 7.1.2     Where risks are not “tolerable and ALARP” are            Must be both Tolerable and ALARP (it is possible for a
            Project Manager                                                        they then subject to Risk Reduction and further          Risk to be ALARP but unacceptable and this must be
                                                                                   estimation and evaluation?                               reduced further).
            S & E Manager         51.3/OME005   OME         JSP520                 When assessing OME safety were OME derived               No guidance given.                                            JSP520 (part 1) 3.5.3
            Project Manager                                                        risks to the following areas investigated ?
                                                                                   ·      Capability
                                                                                   ·      People
                                                                                   ·      Property
                                                                                   ·      Environment


            S & E Manager         51.3/OME006   OME         JSP520                 Have risks associated with disposal phase of the       No guidance given.                                              JSP 520 (Part 1)        5.3.6.5.
            Project Manager                                                        OME‟s life been assessed and then passed on to
                                                                                   the Disposal Authority. If not yet; is there a plan in
                                                                                   place to capture and assess such risks?
 YES      Hazard Log




       Version 1.9.2 April 2008                                                                                                                                                                     Page 13 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID   Audit Requirements                   Audit Question                                   General Guidance for Auditor                            Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                Source Reference
                                      No.
            S & E Manager         52.0         GEN         POSMS SMP11,         How do you provide traceability of the Hazard &/or Expected evidence for Safety is the Hazard Log, defined
            Project Manager                                JSP 430 Part 2       Control Management processed to show how           in ASEMS as "the continually updated record of the
                                                                                safety issues are being dealt with and resolved ?  Hazards, accident sequences and accidents associated
                                                                                                                                   with a system. It includes information documenting risk
                                                                                                                                   management for each Hazard and Accident".

                                                                                                                                      1. Cause of the hazard;
                                                                                                                                      2. Accident sequence;
                                                                                                                                      3. Hazard consequence, likelihood and risk classification;
                                                                                                                                      4. Hazard description prior to mitigation;
                                                                                                                                      5. Hazard manager and hazard owner;
                                                                                                                                      6. Reference of Risk Management (risk assessment work);
                                                                                                                                      7. Assumptions made during risk assessment;
                                                                                                                                      8. Actions taken to control risk;
                                                                                                                                      9. Confirmation that mitigation actions were completed
                                                                                                                                      with no further action required (hazard closure statement);
                                                                                                                                      10. ALARP justification statements and supporting
                                                                                                                                      evidence in the form of;
                                                                                                                                      - Safety criteria report;
                                                                                                                                      - Material / system survey reports;
                                                                                                                                      - Design defect reports;
                                                                                                                                      - Breakdown and failure reports;
                                                                                                                                      - Reports of technical design reviews;
                                                                                                                                      - Quality, reliability and safety audit reports;
                                                                                                                                      - Cost Benefit Analysis
                                                                                                                                      - Accident and incident reports.
                                                                                                                                      - Reference to supporting safety analysis

            S & E Manager         52.1         GEN         POSMS SMP11,         Does the Hazard Log include the areas suggested Looking for evidence that SMP11/G/01 has been
            Project Manager                                                     by SMP11/G/01 ?                                 consulted:
                                                                                                                                • Accident data;
                                                                                                                                • Hazard data;
                                                                                                                                • Hazard and Accident Controls data;
                                                                                                                                • Risk data;
                                                                                                                                • Journal (running log or diary of events and decisions)
                                                                                                                                Are definitions of Hazard and Accident understood and
                                                                                                                                complied with (usually Hazards are a mixture of Causes,
                                                                                                                                Accidents, Issues and some real Hazards)?
                                                                                                                                Risks for Accidents (not Hazards).
            S & E Manager         52.2         GEN         POSMS SMP11,         Where is there a definition of the authority levels to No guidance given.
            Project Manager                                                     view, amend and approve entries in the Hazard
                                                                                Log ?
            S & E Manager         52.3         GEN         POSMS SMP11,         How is the Hazard Log maintained to ensure that        Expect evidence that PSC regularly reviews Hazard Log to
            Project Manager                                JSP 430 Part 2       records are accurate, up-to-date and complete?         ensure actions are completed and unacceptable risks are
                                                                                                                                       resolved.
            S & E Manager         52.4         GEN         POSMS SMP11,         How do you ensure that “ownership” of hazards          Some hazards may be outside the scope (e.g. a higher-
            Project Manager                                                     and controls is clearly identified?                    level system).
                                                                                                                                       Looking for evidence that these are communicated to an
                                                                                                                                       appropriate authority.
            S & E Manager         52.5         GEN         POSMS SMP11,         Is a recognised tool being used to implement the       Does it provide the necessary functionality and audit trail ?
            Project Manager                                                     Hazard Log?                                            Although the DPA and DLO's preferred corporate hazard
                                                                                                                                       log tool is the CASSANDRA database system, which
                                                                                                                                       projects can tailor to meet their needs, individual projects
                                                                                                                                       are free to develop any computerised database or hazard
                                                                                                                                       management program for their use provided that the
                                                                                                                                       solution satisfies the objectives of POSMS Procedure
                                                                                                                                       SMP11 and is under strict configuration control to ensure
                                                                                                                                       a robust audit trail.



 YES      Operational Controls
            S & E Manager    53.0              GEN         POSMS SMP12,         Have controls been or plan to be put in place for     Controls are required:                                            JSP 520 (Part 1)      3.5.3.d
           Project Manager                                 POEMS EMP07,         System testing and trialling?
                                                           JSP520                                                                     - Where required to achieve objectives and targets.
                                                                                                                                      - Where required to achieve standard and stakeholder
                                                                                                                                      requirements
                                                                                                                                      - Where required to achieve standard and stakeholder
                                                                                                                                      requirements
                                                                                                                                      - To control all priority environmental aspects
                                                                                                                                      - To control the environmental aspects which could give
                                                                                                                                      rise to an environmental emergency situation
                                                                                                                                      - To deal with an emergency situation if it was to occur

                                                                                                                                      Although the IPT may not be responsible for directly
                                                                                                                                      developing operational controls for system testing and
                                                                                                                                      trialling, it is the IPTs responsibility to pass on to the body
                                                                                                                                      responsible for undertaking the trials and tests, any
                                                                                                                                      pertinent project related environmental information
                                                                                                                                      (Environmental Case), so that control measures are
                                                                                                                                      developed to minimise any potentially significant
                                                                                                                                      environmental impacts.
                                                                                                                                      The IPT should assure itself that the operational controls
                                                                                                                                      developed have taken into account the environmental
                                                                                                                                      information passed on.




       Version 1.9.2 April 2008                                                                                                                                                                  Page 14 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID    Audit Requirements                        Audit Question                                  General Guidance for Auditor                           Domain Specific Guidance for Auditor                 Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         54.0          GEN         POSMS SMP12, SMP13,        Have controls been or plan to be put in place for    controls are required:
            Project Manager                                 POEMS EMP07, JSP520        Operations?
                                                                                                                                            - Where required to achieve objectives and targets.
                                                                                                                                            - Where required to achieve standard and stakeholder
                                                                                                                                            requirements
                                                                                                                                            - Where required to achieve standard and stakeholder
                                                                                                                                            requirements
                                                                                                                                            - To control all priority environmental aspects
                                                                                                                                            - To control the environmental aspects which could give
                                                                                                                                            rise to an environmental emergency situation
                                                                                                                                            - To deal with an emergency situation if it was to occur

                                                                                                                                            Although the IPT is generally not expected to develop
                                                                                                                                            operational controls for operations, it is the IPTs
                                                                                                                                            responsibility to pass on to the user any pertinent project
                                                                                                                                            related environmental information (Environmental Case),
                                                                                                                                            so that the user can develop appropriate operational
                                                                                                                                            controls to minimise any potentially significant
                                                                                                                                            environmental impacts.
                                                                                                                                            The IPT should assure itself that the operational controls
                                                                                                                                            developed have taken into account the environmental
                                                                                                                                            information passed on.

            S & E Manager         54.1/OME008   OME         JSP 520                    Have submissions to depart from established       No guidance given.                                                  JSP 520 (Part 1)         5.3.11.2
            Project Manager                                                            operating baselines for the OME been submitted to                                                                     JSP 520 (Part 2)         5.3.11.4
                                                                                       the OSRP for approval?
            S & E Manager         55.0          GEN         POSMS SMP12, SMP13,        Have controls been or plan to be put in place for Controls are required:
            Project Manager                                 POEMS EMP07, JSP520        Disposal?
                                                                                                                                         - Where required to achieve objectives and targets.
                                                                                                                                         - Where required to achieve standard and stakeholder
                                                                                                                                         requirements
                                                                                                                                         - Where required to achieve standard and stakeholder
                                                                                                                                         requirements
                                                                                                                                         - To control all priority environmental aspects
                                                                                                                                         - To control the environmental aspects which could give
                                                                                                                                         rise to an environmental emergency situation
                                                                                                                                         - To deal with an emergency situation if it was to occur

                                                                                                                                            It is the IPTs responsibility to ensure that the body directly
                                                                                                                                            responsible for disposal of the equipment, in most
                                                                                                                                            instances the Disposal Services Agency, is fully informed
                                                                                                                                            of all of the significant environmental issues with regards
                                                                                                                                            to disposal considerations, for that equipment. e.g a
                                                                                                                                            common issue connected with disposal will be the
                                                                                                                                            presence of hazardous and restricted materials in the
                                                                                                                                            equipment. The IPT must inform the DSA of the presence
                                                                                                                                            of the materials and if possible their location and
                                                                                                                                            quantities.

 YES      Safety Case & Case Report
 YES      Management of Safety Case information and its content
            S & E Manager    56.0          GEN              POSMS SMP12,               What arrangements exist for the construction and   Safety Case is defined in ASEMS as "a structured
            Project Manager                                 JSP 520                    systematic maintenance of a Safety Case?           argument, supported by a body of evidence that provides
                                                                                                                                          a compelling, comprehensible and valid case that a
                                                                                                                                          system is safe for a given application in a given
                                                                                                                                          environment."
            S & E Manager         56.1          GEN         POSMS SMP12                How do you control the configuration of the Safety Cannot have a strong Safety Case if it is not up to date
            Project Manager                                                            Case body of information ?                         and well managed.
            S & E Manager         56.2          GEN         POSMS SMP12                How is Safety Case data organised and/or           For example some software tool for organising, managing
            Project Manager                                                            presented ?                                        and presenting the Evidence (e.g. a Goal Structuring
                                                                                                                                          Notation package, a document management system).

            S & E Manager         56.3          GEN         POSMS SMP12, SMP13,        How is the Safety Case amended to take account E.g. the way it is operated or maintained
            Project Manager                                 JSP 553                    of the changes to the design, or a change of use to
                                                                                       the existing design ?
            S & E Manager         56.4          GEN         JSP 520                    If the Project is open to competition is a Safety   No guidance given.                                                JSP 520( part 1) 3.4.4
            Project Manager                                                            Case being developed to an appropriate level for
                                                                                       all contenders?
            S & E Manager         56.5/OME009   OME         JSP 520                    Is the procedure defined which monitors changes No guidance given.                                                    JSP 520 (Part 1) 3.1.1.c., 3.4.3, 5.3.11.2., 5.4.2.
            Project Manager                                                            to the build standard, environment or usage of the
                                                                                       OME and determines resubmission of the Safety
                                                                                       Case to OSRP ?
 YES      Safety Case Conclusion
            S & E Manager    57.0               GEN         POSMS SMP12              Does the Safety Case demonstrate that the full         That is what the Safety Case is for: it must be up to date
            Project Manager                                                          extent of development, trials and in service use can   and if the users want to use the system in ways not
                                                                                     be safely conducted?                                   covered by the Safety Case, it should be revised.
            S & E Manager         57.1          GEN         POSMS SMP12              Does the Safety Case demonstrate satisfactory
            Project Manager                                                          resolution of human factors issues?
            S & E Manager         57.2          GEN         POSMS SMP12              Does the Safety Case contain adequate safety
            Project Manager                                                          appraisal amd mitigation where COTS systems are
                                                                                     utilised?
            S & E Manager         57.3          GEN         POSMS SMP12              Does the Safety Case identify safety critical
            Project Manager                                                          software functionality and describe how the risks
                                                                                     have been mitigated to an appropriate level?
            S & E Manager         57.4          GEN         POSMS SMP12              How has the safety evidence supporting the use of      There must be evidence that this strategy has been
            Project Manager                                                          COTS or Software of Unknown Parentage (SOUP)           independently verified.
                                                                                     been constructed?
            S & E Manager         57.5          GEN         POSMS SMP12              Does the safety case address decommissioning
            Project Manager                                                          and disposal of the system or equipment?
            S & E Manager         57.6          GEN         POSMS SMP12 para 1.2.3, To what extent has the Safety Case and its
            Project Manager                                 JSP 553                  supporting evidence been subjected to
                                                                                     independent assessment?
            S & E Manager         57.7          GEN         POSMS SMP12,             Can you explain how the safety case is credible,       Are the goals and strategies defined, is all the supporting
            Project Manager                                 Air Eng (Pol) AVSG Audit complete, comprehensible, convincing?                  evidence referenced and location known to underpin the
                                                            [modified]                                                                      goal




       Version 1.9.2 April 2008                                                                                                                                                                       Page 15 of 32
     ASEMS AUDIT QUESTION SET

Scoped Proposed level             Audit      Domain ID    Audit Requirements                          Audit Question                                     General Guidance for Auditor                         Domain Specific Guidance for Auditor                   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee            Question                  Source Reference
                                   No.
          S & E Manager         57.8/AW017   AW          JSP 553 2.42, Table 2.2,     Does the safety case detail the evidence for air         For all military aircraft types, their associated equipment
          Project Manager                                4.34                         worthiness including safety analysis ansd the            and software, the Aircraft Designer has satisfactorily
                                                                                      results of tests and trials carried out by the           demonstrated, in a Safety Case, the airworthiness of the
                                                                                      designer and independent evaluation such as              design. The demonstration of airworthiness may include
                                                                                      aircraft test & evaluation (AT&E)?                       design analysis, application of specified standards (such
                                                                                                                                               as Def Stan 00-970) and procedures, historical evidence
                                                                                                                                               of successful use of particular design features, system
                                                                                                                                               tests, and ground and air tests to arrive at an overall
                                                                                                                                               assessment of airworthiness. The demonstration should
                                                                                                                                               be as specified in the aircraft or modification contract
                                                                                                                                               including the operating conditions to be applied, and
                                                                                                                                               should be undertaken during the development phase for
                                                                                                                                               the aircraft or modification.


          S & E Manager         57.8/AW018   AW          JSP 553 1.39, 2.16, 4.34     Has the IPTL ensured that the aircraft and               The IPTL is to derive project-specific design standards
          Project Manager                                                             equipment for which he holds delegated                   from these criteria.
                                                                                      airworthiness responsibility meets the
                                                                                      airworthiness design criteria detailed in Para 1.39.

          S & E Manager         57.8/AW019   AW          JSP 553 2.16                 Does the safety case describe limitations and the
          Project Manager                                                             statement of operating use and intent (SOIU) and
                                                                                      related documents that support the release to
                                                                                      service (RTS)
          S & E Manager         57.8/AW020   AW          JSP 553 2.4.3, 2.44 - 2.46   Does the safety case describe the design and           No guidance given.
          Project Manager                                                             provide a justification for its airworthiness?
          S & E Manager         57.8/AW021   AW          JSP 553 1.40, 2.20, 2.42     Does the Aircraft Safety Case cover the use of:        No guidance given.
          Project Manager                                Table 2.2, 4.60,             Aerial Delivery Equipment (ADE) Helicopter Under-
                                                                                      slung Load Equipment (HUSLE),
                                                                                      either individually or as a part of a generic range of
                                                                                      equipment?
                                                                                      Have the safety implications to the delivery aircraft,
                                                                                      aircrew and other personnel on-board and on the
                                                                                      ground been assessed?




          S & E Manager         57.8/AW022   AW          JSP 553 4.63                 Does the Safety Case for each AEA detail the             An Aircrew Equipment Assembly (AEA) is declared
          Project Manager                                                             extent of the clearance, and the limitations of the      airworthy by a statement based on the Safety Case for the
                                                                                      AEA for specific aircraft or roles in Service            equipment, issued by the AEA IPTL for the purpose of in-
                                                                                      operation?                                               Service use.
          S & E Manager         57.8/AW023   AW          JSP 553 4.64                 Has the Aircraft IPTL ensured that the AEA Safety        No guidance given.
          Project Manager                                                             Case statement fully supports the proposed use on
                                                                                      his platform?
                                                                                      Does it include any limitations on the use of the
                                                                                      AEA in the RTS of the aircraft where they affect, or
                                                                                      contribute to, limitations that affect the aircraft or
                                                                                      aircrew?
          S & E Manager         57.8/AW024   AW          JSP 553 2.49.5, 4.5.5.3,     Has the Safety Case been assessed independently          For air-launched GWs, the GW Release (see para 4.39)
          Project Manager                                Fig 4.1, 4.6.4               of the designer?                                         provides the safety justification and audit trail.
          S & E Manager         57.8/AW025   AW          JSP 553 4.12, 4.59           Does the equipment Safety Case cover the safety          No guidance given.
          Project Manager                                                             features of the equipment and the way it is to be
                                                                                      safely interfaced to the aircraft?

          S & E Manager         57.8/AW026   AW          JSP 553 4.56 - 4.64, 5.8 -   Does the aircraft safety case address the airborne       No guidance given.
          Project Manager                                5.10                         carriage and/or operation of non aircraft specific
                                                                                      equipment?
          S & E Manager         57.8/AW027   AW          JSP 553 4.60 - 4.61          Does Safety analysis of Aerial Delivery Equipment        No guidance given.
          Project Manager                                                             include an assessment of: aircraft :
                                                                                      All-up-Mass (AUM);
                                                                                      floor loading;
                                                                                      Centre of Gravity (CofG) during carriage and
                                                                                      delivery;
                                                                                      load stability and restraint;
                                                                                      the elasticity of strops;
                                                                                      the type of delivery concerned;
                                                                                      critical factors such as wind and turbulence;
                                                                                      and failure cases such as of side-guidance,
                                                                                      parachutes, or release units.
          S & E Manager         57.8/AW028   AW          JSP 553 4.7                  Were the Defence Ordnance Safety Group                   This includes documenting the evidence for airworthiness
          Project Manager                                                             (DOSG) or Airworthiness, Design Requirements             in the Safety Case, deriving the release documents and
                                                                                      and Procedures (ADRP) armament specialists               producing the Aircraft Document Set (ADS) and
                                                                                      consulted, at an early stage, about the sections of      Development of the Safety Case.
                                                                                      the Safety Case dealing with weapons or
                                                                                      explosives?
          S & E Manager         57.8/AW029   AW          JSP 553 4.53 - 4.55          How do existing safety records and analysis show         Legacy equipments include:
          Project Manager                                                             that the safety cases for legacy aircraft and            All equipments already in service.
                                                                                      equipment are ALARP?                                     Commercial-off-the-shelf (COTS) equipments for stand
                                                                                                                                               alone use.
                                                                                                                                               COTS equipments for incorporation into new or existing
                                                                                                                                               systems.
                                                                                                                                               (See also Non Aircraft-Specific Equipment Safety at Para
                                                                                                                                               4.53).

          S & E Manager         57.8/AW030   AW          JSP 553 4.54                 What criteria is used to manage any backlog of           As a minimum, the IPTL is to:                                  Award marks if ISA is appointed or if appointment is
          Project Manager                                                             work required on legacy equipments already in            Identify and review the existing safety justifications (eg the deemed Not Required and there is evidence to
                                                                                      service?                                                 results of trials and advice from specialist organisations). support this decision otherwise score 0
                                                                                                                                               Produce a Safety Assessment Report containing a
                                                                                                                                               summary of the existing safety justifications and a safety
                                                                                                                                               appraisal of the change based on these.


          S & E Manager         57.8/AW031   AW          Air Eng (Pol) AVSG Audit     41. How is the integrity of the Safety Case(s)      No guidance given.
          Project Manager                                JSP 553 4.59                 ensured for different variants when airframe and/or
                                                                                      engine updates are made?




     Version 1.9.2 April 2008                                                                                                                                                                          Page 16 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID    Audit Requirements                          Audit Question                                    General Guidance for Auditor                          Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         57.8/AW032    AW          JSP 553 4.5.5.3, 4.5.5.4    Has the IPTL considered the need for advice             No guidance given.
            Project Manager                                                             provided by independent bodies, when presented
                                                                                        with the Form 100 (or equivalent), supported by the
                                                                                        Safety Case?
                                                                                        Was the safety case modified in accordance with
                                                                                        advice provided by independent bodies?
            S & E Manager         57.8/AW033    AW          JSP 553 2.16                Does the Safety Assessment demonstrate that the
            Project Manager                                                             aircraft meets its safety targets?
            S & E Manager         57.8/AW034    AW          JSP 553 2.64, 5.63          Determine whether Structural Integrity (SI)             To ensure that the SI risk remains acceptably low
            Project Manager                                                             considerations and specific assurance measures          throughout the life of the A/C
                                                                                        have been included in the TLMP?
                                                                                        Does the SI Plan form part of the Safety Case?
            S & E Manager         57.8/AW035    AW          JSP 553 Annex K4            Have the IPT arranged for the installation of a         No guidance given.
            Project Manager                                                             permanent Operational Loads Measurement (OLM)
                                                                                        system in a sample of the fleet –to record data.

            S & E Manager         57.8/AW036    AW          JSP 553 Annex K7.1          Confirm that the ac operating profile and fatigue       No guidance given.
            Project Manager                                                             life, commensurate with the URD, are stated in the
                                                                                        Contract specification.
            S & E Manager         57.8/AW037    AW          JSP 553 Annex K7.2          Show how static and fatigue clearances have been        No guidance given.
            Project Manager                                                             developed (test & analysis), and that cleared
                                                                                        fatigue safe lives have been maintained ahead of
                                                                                        the fleet leader.
            S & E Manager         57.8/AW038    AW          JSP 553 Annex K7.3          Show that static & fatigue clearances have been         No guidance given.
            Project Manager                                                             supported by Static Type Record (STR) & Fatigue
                                                                                        Type Record (FTR) documents.
            S & E Manager         57.8/AW039    AW          JSP 553 Annex K7.4          Confirm that the IPT have arranged (thro SSG ASI)       No guidance given.
            Project Manager                                                             for initial issue of the SOIU & for subsequent
                                                                                        review (see 7.4.1 – 7.4.4).
            S & E Manager         57.8/AW040    AW          JSP 553 Annex K7.5          Confirm that a robust fatigue & usage monitoring        No guidance given.
            Project Manager                                                             system is in place for individual ac tracking of safe
                                                                                        life/damage tolerant features.
            S & E Manager         57.8/AW041    AW          JSP 553 Annex K7.7 -        Confirm that IPT are monitoring fleet fatigue           No guidance given.
            Project Manager                                 K7.11                       consumption and usage, have instigated fatigue
                                                                                        modification or refurbishment programmes where
                                                                                        necessary to maintain capability, promote
                                                                                        corrosion control activities & maintain a database
                                                                                        of structural concessions, repairs etc.
            S & E Manager         57.8/AW042    AW          JSP 553 5.65                Does the Statement of Operating Intent & Usage          No guidance given.
            Project Manager                                                             (SOIU) –convey current usage and future intent for
                                                                                        the ac to the ac designer.
            S & E Manager         57.8/AW043    AW          JSP 553 4.56                Does the aircraft safety case include:                  The decision of the Aircraft IPTL is to have primacy over
            Project Manager                                                             a. All limitations applicable to the carriage of        that of the equipment and commodity IPTLs.
                                                                                        stores or equipment
                                                                                        b. The proposed use of weapons and other
                                                                                        airborne equipment?
            S & E Manager         57.8/AW044    AW          JSP 553 1.40, 2.20, 2.44,   Do Aerial Delivery Equipment clearances for             Clearance for use of ADE from one aircraft type to another
            Project Manager                                 4.60 - 4.62                 operations specify each type and mark of aircraft?      cannot be assumed to read across to another aircraft type
                                                                                                                                                or mark
 YES      Safety Case Report
            S & E Manager    58.0               GEN         POSMS SMP12,                Does the Safety Case Report include the areas Safety Case Report is defined in ASEMS as "A report that
            Project Manager                                 JSP520                      suggested in POSMS Guidance sheet SMP12/G/01 summarises the arguments and evidence of the Safety
                                                                                        ?                                             Case, and documents progress against the Safety
                                                                                                                                      Programme."

                                                                                                                                                • Executive Summary;
                                                                                                                                                • System description;
                                                                                                                                                • Assumptions;
                                                                                                                                                • Progress against Programme;
                                                                                                                                                • Meeting Safety Requirements;
                                                                                                                                                • Emergency and Contingency Arrangements;
                                                                                                                                                • Operational information (e.g. limitations for Safe use);
                                                                                                                                                • ISA Report (if relevant);
                                                                                                                                                • Certification (e.g. CSOME)


            S & E Manager         58.1          GEN         POSMS SMP12                 Does the Safety Case Report include a justification See POSMS SMP12 Section 10.8
            Project Manager                                                             of the techniques and processes used to undertake
                                                                                        the Risk Assessment & Management ?

            S & E Manager         58.2          GEN         POSMS SMP12                 What is the process for producing and agreeing          Who writes them, who reviews them, who signs them off
            Project Manager                                                             Safety Case Reports ?                                   (what does the signature mean?) (see Section 1 of
                                                                                                                                                SMP12) ?
            S & E Manager         58.3          GEN         POSMS SMP12                 How do you decide when and by whom Safety               Expect this to be recorded in the Safety Management Plan
            Project Manager                                                             Case Reports are to be produced, reviewed,              e.g. for Initial Gate, Main Gate, Trials, Release to Service,
                                                                                        signed and submitted for approval ?                     Major Update etc
                                                                                                                                                NB The approval process and signatories may be different
                                                                                                                                                at each point.

            S & E Manager         58.4          GEN         POSMS SMP12,                Has a programme for submission of Safety Case           No guidance given.
            Project Manager                                 JSP 520 [modified]          Report to Safety Regulators / Approval Authorities
                                                                                        been agreed ?
            S & E Manager         58.5/OME007   OME         JSP 520                     Has the Risk Level category been agreed with            No guidance given.                                              JSP 520 (Part 1)   5.3.3
            Project Manager                                                             DOSG
            S & E Manager         58.5/OME008   OME         JSP 520                     Has the OSRP commented on the efficacy of the           No guidance given.                                              JSP 520 (Part 1)   6.5.1
            Project Manager                                                             proposed OME risk management activities and if
                                                                                        so what measures are planned to address these
                                                                                        comments ?
            S & E Manager         58.5/OME009   OME         JSP 520                     Does the Certificate contain any caveats or safety      What has been done to ensure that these caveats and             JSP 520 (Part 1)   6.5.1
            Project Manager                                                             instructions?                                           limitations are being complied with? How has the IPTL
                                                                                                                                                assured themselves that these caveats have been
                                                                                                                                                adequately adopted by the user?
            S & E Manager         58.5/OME010   OME         JSP 520 5.3.5               Did the assessment of S3 (provided by the DOSG          • Software integrity                                            JSP 520 (Part 1)   5.3.6.3
            Project Manager                                                             advisor) (Safety and Suitability for Service) for the   • Fuze design
                                                                                        OME include the relevant elements?                      • Material toxicity
                                                                                                                                                • Impulse noise
                                                                                                                                                • Human factors




       Version 1.9.2 April 2008                                                                                                                                                                          Page 17 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit      Domain ID    Audit Requirements                   Audit Question                                 General Guidance for Auditor                        Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         58.5OME011    OME         JSP 520 5.3.6.4       Does the OME SCR include references to the           ESTC Hazard classification, Explosive Qualification and JSP 520 (Part 1)            5.3.6.4
            Project Manager                                                       relevant clearances and certificates to support      explosive type qualification, Range safety measures,
                                                                                  introduction into service?                           LASER safety certification, IM assessment (STANAG
                                                                                                                                       4439), Dangerous Air Cargo Clearance (DACC), Aircraft
                                                                                                                                       Weapons Air carriage and Release, Aircraft Self Damage
                                                                                                                                       (ASD), Thermal Effects on Airborne Conventional
                                                                                                                                       Armament Stores and Equipment (TEACASE) and scrutiny
                                                                                                                                       by the Aircraft Weapons Ballistic Committee (AWBC))


            S & E Manager         58.5/OME012   OME         JSP 520 5.3.6.5       Prior to the introduction of the OME will the        Explosive Hazard Data Sheets (EHDS) or equivalent,        JSP 520 (Part 1 )         5.3.6.5
            Project Manager                                                       relevant documentation be produced by the IPT        Environmental hazard analysis or environmental impact
                                                                                  and included in or referenced in the OME SCR?        screening statement, IPT OME Safety and Environmental
                                                                                                                                       Management system, OME safety instruction,
                                                                                                                                       Appropriate operating environment statements e.g. NAES
                                                                                                                                       form (for OME to be used in the maritime environment),
                                                                                                                                       Explosive compatibility matrices, In-service surveillance
                                                                                                                                       plan, Disposal plan, Environment Questionnaire AOP15


 YES      Safety Case Review and Audit
            S & E Manager    59.0               GEN         POSMS SMP12 4.5,      How is the Safety Case reviewed ?                    Process should define:
            Project Manager                                 ES (Land)                                                                  • who reviews
                                                                                                                                       • what frequency / situations
                                                                                                                                       • how reported
                                                                                                                                       • how corrective actions are carried out
 YES      Defining the in-service SMS
            S & E Manager      60.0             GEN         POSMS SMP13           How can you demonstrate that Safety Management The Safety Programme should not be just documenting
           Project Manager                                                        activities in development influenced the in-service Safety, but influencing the Training, Procedures,
                                                                                  SMS ?                                               Maintenance etc for better safety.
                                                                                                                                      (see SMP13 Section 7.1.1), including aspects such as:
                                                                                                                                      • Implementation of Controls;
                                                                                                                                      • Safety Information management;
                                                                                                                                      • Safety Performance Reviews and Continuous
                                                                                                                                      Improvement;
                                                                                                                                      • Configuration management;
                                                                                                                                      • Risk Management (for modifications and enhancements);
                                                                                                                                      • Lines of Development (e.g. training, manpower numbers,
                                                                                                                                      equipment & technology)?

            S & E Manager         60.1/AW045    AW          JSP 553 1.33, 5.57    What policies have been developed which address To include:
            Project Manager                                                       Preventive Maintenance.                              • Flight servicing
                                                                                                                                       • Scheduled maintenance
                                                                                                                                       • Out-of-phase maintenance (inc. component lifing)
                                                                                                                                       Condition-based maintenance (inc. zonal maintenance
                                                                                                                                       req.)
            S & E Manager         60.1/AW046    AW          JSP 553 1.33, 5.58    Does the maintenance policy specify what latitudes No guidance given.
            Project Manager                                                       may be applied to the various periodicities & lives,
                                                                                  and who may authorise anticipation or deferment of
                                                                                  them.

            S & E Manager         60.1/AW047    AW          JSP 553 5.58          Do the schedules make clear which items are
            Project Manager                                                       safety-critical and which are non-safety critical
            S & E Manager         60.1/AW048    AW          JSP 553 5.57 - 5.62   Does the A/C contain lifed components with           No guidance given.
            Project Manager                                                       published maintenance periodicities or lives? What
                                                                                  special circumstances justify a different lifing
                                                                                  policy.
            S & E Manager         61.0          GEN         POSMS SMP13 10.1.1    How are the residual risks and their control         NB: Question only relevant for systems prior to in-service
            Project Manager                                                       measures to be examined before the                   or for mid-life updates.
                                                                                  equipment/system/capability comes into use (see      Land Systems has the Part 3 Safety Case to demonstrate
                                                                                  SMP13 Section 10.1.1) ?                              that all the necessary measures are actually in place
                                                                                                                                       before the system comes into service.
                                                                                                                                       Either a Part 3 Safety Case or something similar to show
                                                                                                                                       that the In-Service arrangements are in place and
                                                                                                                                       adequate.
                                                                                                                                       Looking for evidence of
                                                                                                                                       • Arrangements for training;
                                                                                                                                       • User and Maintainer documentation;
                                                                                                                                       • Support arrangements;
                                                                                                                                       • Limitations or restrictions on operation;
                                                                                                                                       • Emergency and contingency operation.


 YES      In-Service SMS
             S & E Manager        62.0          GEN         POSMS SMP13           Where is the In-service SMS documented ?             This needs to be done in advance of the system being
            Project Manager                                                                                                            brought into service.
                                                                                                                                       It must be specific to the Project/System, not just saying
                                                                                                                                       "each organisation has got a documented SMS”.

            S & E Manager         62.1          GEN                               Have all parties agreed with their role as described
            Project Manager                                                       ?
            S & E Manager         63.0          GEN         POSMS SMP13,          Has the methodology for managing risks through         No guidance given.                                         JSP 520(part 1) 3.4.3, 4.12.b, 3.2.1
            Project Manager                                 JSP 520               life been clearly defined along with the allocation of
                                                                                  responsibilities?
 YES      Safety Assurance during Changes
            S & E Manager    64.0               GEN         POSMS SMP13,          What systematic processes are defined to provide     No guidance given.                                           JSP 520(part 1) 5.3.11.2, 6.3.3.5
            Project Manager                                 JSP 520               assurance that any changes to the system, life
                                                                                  cycle, environment or usage are assessed for
                                                                                  impact on safety?
            S & E Manager         64.1          GEN         POSMS SMP13,          Have the arrangements to review any impact on        No guidance given.                                           JSP 520(part 1) 3.1.1.c.
            Project Manager                                 JSP 520               safety arising from modification, concession,
                                                                                  upgrade been documented ?
            S & E Manager         64.2          GEN         POSMS SMP13,          Can it be demonstrated that these arrangements       No guidance given.
            Project Manager                                 JSP 520               continue to be implemented ?




       Version 1.9.2 April 2008                                                                                                                                                                Page 18 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level               Audit      Domain ID    Audit Requirements                         Audit Question                                       General Guidance for Auditor                           Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee              Question                  Source Reference
                                     No.
            S & E Manager         64.3/AW049   AW          JSP 553 5.23, 5.40,        Provide evidence that any residual service        No guidance given.
            Project Manager                                Annex J Appendix 4         deviation (SD), service engineered changes and
                                                                                      clearances of limited evidence have been reviewed
                                                                                      for their safety implications and impact on the
                                                                                      safety case and release to service
            S & E Manager         64.3/AW050   AW          JSP 553 3.16               Co-ordination of design changes. Determine that No guidance given.
            Project Manager                                                           staff are competent (e.g. CV‟s, engineering
                                                                                      experience) to manage industrial designer and
                                                                                      Service engineered changes.
 YES      Release to Service
            S & E Manager    64.4/AW051        AW          JSP 553 2.54, 4.12      Has the Release to Service (RTS) been applied
           Project Manager                                                         using the Generic Aircraft Release Process
                                                                                   (GARP) as per JSP 553?
            S & E Manager         64.4/AW052   AW          JSP 553 Annex J         Show how the RTS has been/will be fully                       No guidance given.
            Project Manager                                Appendix 2 Part F       substantiated with evidence from the safety case,
                                                                                   aircraft document set (ADS) and clearances of
                                                                                   limited evidence (CLE)?
            S & E Manager         64.4/AW053   AW          JSP 553 Annex S Appdx 1 Does the format of the intended GW Release                    Guided Weapon Releases. Similarly, GW IPTLs are to
            Project Manager                                                        conform to JSP 553                                            ensure that all new GW Releases conform to the model
                                                                                                                                                 generic GW Release described in Annex S Appendix 1.
                                                                                                                                                 Annex S also contains advice on layout and content. GW
                                                                                                                                                 IPTLs may retain the format and content of existing GW
                                                                                                                                                 Releases provided there is agreement with users that no
                                                                                                                                                 worthwhile benefits would accrue from reformatting to the
                                                                                                                                                 generic model.

            S & E Manager         64.4/AW054   AW          JSP 553 4.44, Annex C2     Does the IPTL have a GW airworthiness LOD?                 The GW Release is to be approved by an IPTL with
            Project Manager                                                           What is the GW IPTL‟s understanding of the                 appropriate delegated airworthiness authority. Close
                                                                                      relationship between the GW Release and the host           consultation with the initial platform‟s IPTL will be required.
                                                                                      platform?                                                  The GW Release is the statement that an acceptable
                                                                                                                                                 Safety Case has been prepared and can form the basis for
                                                                                                                                                 a formal approval for operation on the aircraft types stated
                                                                                                                                                 within the limitations given. A signed GW Release
                                                                                                                                                 indicates that a GW is fit to be handled, maintained,
                                                                                                                                                 stored and transported by the Services irrespective of the
                                                                                                                                                 status of any platform and the availability of an MA
                                                                                                                                                 Release.

            S & E Manager         64.4/AW055   AW          JSP 553 4.15               Do you understand the role of Assistant Chief of Air Authority for the Initial issue of a MA Release (other than
            Project Manager                                                           Staff (ACAS) in signing off the initial RTS?         for mature aircraft or equipment) is to be given by the
                                                                                                                                           appropriate DPA Executive Director. Subsequently, re-
                                                                                                                                           issues and amendments are normally authorised iaw the
                                                                                                                                           delegated authority held by the IPTL or his staff. For
                                                                                                                                           mature aircraft or equipment, any initial MA Release
                                                                                                                                           required (such as for COMR aircraft) is to be authorised at
                                                                                                                                           2* level in the DLO; subsequent MA Release matters are
                                                                                                                                           authorised in the delegation to the DLO IPTL.


            S & E Manager         64.4/AW056   AW          JSP 553 4.19               Have you produced a master minimum equipment
            Project Manager                                                           list and minimum equipment list iaw JSP 553 for
                                                                                      military variants of transport aircraft?
            S & E Manager         64.4/AW057   AW          JSP 553 5.9, 5.10          What evidence is there that the key 'post
            Project Manager                                                           separation' equipment safety limitations have been
                                                                                      presented?
            S & E Manager         64.4/AW058   AW          JSP 553 3.35, 5.13, 5.16   Has the Service Operations Branch (SOB)
            Project Manager                                                           nominated a Release to Service Authority (RTSA)?

            S & E Manager         64.4/AW059   AW          JSP 553 3.35, 5.13, 5.16   Have appropriate airworthiness delegations been
            Project Manager                                                           made to the RTSA?
            S & E Manager         64.4/AW060   AW          JSP 553 5.16               Has the SOB obtained confirmation that all
            Project Manager                                                           operating support elements necessary to maintain
                                                                                      airworthiness are in place?
            S & E Manager         64.4/AW061   AW          JSP 553 5.11               How has the RTSA ensured that the 'post
            Project Manager                                                           separation' limitations have been detailed in the
                                                                                      platform RTS, and where appropriate carried
                                                                                      forward into the rest of the ADS?
            S & E Manager         64.4/AW062   AW          JSP 553 5.11               Have the RTS limitations been used in the
            Project Manager                                                           formulation of Contingency and War plans and
                                                                                      Flying & Operating orders?
            S & E Manager         64.4/AW063   AW          JSP 553 5.11               Have the limitations of the RTS been reflected in
            Project Manager                                                           Aircrew Manuals, Flight Reference Cards,
                                                                                      Operation Data Manuals, Flight Test Schedules
                                                                                      and technical publications?
            S & E Manager         64.4/AW064   AW          JSP 553 6.1, Annex J.40,   What arrangements have you made to ensure that
            Project Manager                                Annex J Appendix 2         the RTS and APs reflect the same amendment
                                                                                      state?

            S & E Manager         64.4/AW065   AW          JSP 553 5.44, 6.1.2        Have you delegated authority to OC HS (see Chap
            Project Manager                                                           6 of JSP 553) for the AM, FRCs, FTS?
            S & E Manager         64.4/AW066   AW          JSP 553 4.22, 4.23         Have you contracted for publication of repair
            Project Manager                                                           schemes iaw JSP553?
            S & E Manager         64.4/AW067   AW          JSP 553 3.24               Has the design organisation been approved under
            Project Manager                                                           the Design Approved Organisations Scheme
                                                                                      (DAOS)?
            S & E Manager         64.4/AW068   AW          JSP 553 1.25, 1.29         Where the aircraft is a variant of a civil type aircraft
            Project Manager                                                           or where the MoD is not the certifying authority
                                                                                      (e.g. foreign or collaborative programmes) does the
                                                                                      safety plan and certification regime demonstrate
                                                                                      sufficient equivalence for the alternative
                                                                                      arrangements to be accepted?

            S & E Manager         64.4/AW069   AW          JSP 553 4.27               Have you drawn up an Integrated Test, Evaluation
            Project Manager                                                           and Acceptance plan? (Acquisition Handbook page
                                                                                      22)




       Version 1.9.2 April 2008                                                                                                                                                                            Page 19 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID    Audit Requirements                       Audit Question                                 General Guidance for Auditor                          Domain Specific Guidance for Auditor   Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                 Source Reference
                                      No.
            S & E Manager         64.4/AW070   AW          JSP 553 4.40 - 4.46       Does the GW safety case have sufficient evidence A GW may have limited flight clearance (or none in peace-
            Project Manager                                                          to support full clearance or limited clearance with time), there must be sufficient safety evidence in the GW
                                                                                     special conditions?                                 Release to permit the ground handling, storage,
                                                                                                                                         transportation and disposal (emergency or otherwise) of a
                                                                                                                                         GW before it is delivered to any Service Unit.

            S & E Manager         64.4/AW071   AW          JSP 553 Annex B           With respect to UAVs sas the development, tests
            Project Manager                                                          and RTS been conducted iaw JSP553
            S & E Manager         64.4/AW072   AW          JSP 553 4.48, 4.49, Annex What are the Safety Targets for your UAVS?          Achievement of the airworthiness criteria requires an
            Project Manager                                B                         Have you considered the factors listed at 4.49      assessment of all the factors that influence the safety of
                                                                                                                                         the overall system. A UAV confined to use within a safe
                                                                                                                                         area may not require the same level of design integrity
                                                                                                                                         when compared to one that is to be cleared for regular
                                                                                                                                         operation over an urban population. Each system design
                                                                                                                                         should be tailored to its specified operational environment.

            S & E Manager         64.4/AW073   AW          JSP 553 4.49, Annex B     In assessing the safety of a UAV System the IPTL    The reliability of the air vehicle (including the structure,
            Project Manager                                                          must take into account several factors, some of     engine, avionics, etc.).
                                                                                     which apply equally to piloted aircraft, and any    The reliability of the command and control systems and
                                                                                     others that are relevant to the system and its      links.
                                                                                     intended operation and area of flight               The reliability of software
                                                                                                                                         The reliability of any flight termination systems.
                                                                                                                                         The mode of operation.
                                                                                                                                         The risk of collision with other aircraft.
                                                                                                                                         The density of civilian population in the area which it over-
                                                                                                                                         flies and the exposure time of that population.
                                                                                                                                         Any risk reduction techniques or procedures.
                                                                                                                                         Peace, tension, threat, TTW and war situations.
 YES      CHECKING AND CORRECTIVE ACTION
 YES      Safety Case Configuration Control and Currency
            S & E Manager    65.0            GEN           POSMS SMP12,              How can you show that recommendations made in No guidance given.
            Project Manager                                Air Eng (Pol) Adv Audit   the safety case are being / have been implemented
                                                                                     ?
 YES      Independent Safety Auditor
             S & E Manager   66.0              GEN         POSMS SMP01, SMP10        Has an Independent Safety Auditor (not Adviser)  Competence (experience, training, qualifications) in:
            Project Manager                                                          with the necessary competence been appointed for • Safety Management
                                                                                     this Project ?                                   • Auditing
                                                                                                                                      • Domain and technology issues
            S & E Manager         66.1         GEN         POSMS SMP10,              How did the IPT/Project assess the competency of No guidance given.
            Project Manager                                JSP 553                   the organisation that was selected to act as
                                                                                     independent assessors/auditors?
            S & E Manager         66.2         GEN         POSMS SMP01               If an ISA is appointed, what is the ISA's role?  For example the audit should look for evidence that the
            Project Manager                                                                                                           ISA has carried out the following:
                                                                                                                                      • Audited/Reviewed the Prime Contractor or the MoD
                                                                                                                                      Project.
                                                                                                                                      • Ensured there is no conflict between any elements of
                                                                                                                                      advice.
                                                                                                                                      • Included independent assessment as well as audit.
                                                                                                                                      • Gained evidence of written Terms of Reference for the
                                                                                                                                      ISA and ensured all parties have a common understanding
                                                                                                                                      of ISA‟s role.
                                                                                                                                      • Conducted audit of sub-contractors where applicable?
 YES      Audit (Internal and Peer)
            S & E Manager      67.0            GEN         POSMS/POEMS SSP01,        Is there a programme for safety / environmental     This is for the IPT doing self-checking and can be applied JSP 520(part 1) 3.4.3, 4.12.b, 3.2.1
           Project Manager                                 JSP 520                   internal self audits against your safety &          at all levels. This an opening question. This does not
                                                                                     environmental management system and plan?           relate to an ISA audit. This question is looking at whether
                                                                                                                                         they are auditing against their own safety and
                                                                                                                                         environmental management system and plan. The scope
                                                                                                                                         should be defined by the SEMP (which in turn should be
                                                                                                                                         against JSPs).
                                                                                                                                         ·      Is there a plan for self-audit of the SEMS?
                                                                                                                                         ·      Has the audit plan been followed?
                                                                                                                                         ·      How will the audit be carried out?
                                                                                                                                         ·      Who will carry out the audit
                                                                                                                                         ·      Have they been trained in auditing and safety
                                                                                                                                         /environmental management
                                                                                                                                         ·      Has all identified improvements been actioned in a
                                                                                                                                         timely manner?
            S & E Manager         67.1         GEN         POSMS/POEMS SSP01,        Who plans and schedules safety / environmental      No guidance given.
            Project Manager                                Air Eng (Pol) 020910      audits, reviews or surveys?
            S & E Manager         67.2         GEN         POSMS/POEMS SSP01,        Is the appropriate maximum interval between         No guidance given.                                              JSP 520(part 1) 5.3.11.2, 6.3.3.5
            Project Manager                                Air Eng (Pol) 020910      safety / environmental audits, reviews or surveys
                                                                                     defined?
            S & E Manager         67.3         GEN         POSMS/POEMS SSP01,        Who determines the scope of a safety /              No guidance given.                                              JSP 520(part 1) 3.1.1.c.
            Project Manager                                Air Eng (Pol) 020910      environmental audit, review or survey?
            S & E Manager         67.4         GEN         POSMS/POEMS SSP01,        How are the procedures for the conduct, reporting   No guidance given.
            Project Manager                                JSP 520                   and follow up action of internal audits defined?

 YES      Non-Conformance and Corrective Action
            S & E Manager  68.0            GEN             POSMS/POEMS AAP04         How do you manage non-conformance and
           Project Manager                                                           Corrective Action ?
            S & E Manager  68.1            GEN             POSMS/POEMS AAP04         Who decides if corrective action is necessary?      No guidance given.
           Project Manager                                 Air Eng (Pol) 020910
            S & E Manager  68.2            GEN             POSMS/POEMS AAP04         How are corrective actions monitored to ensure      No guidance given.
           Project Manager                                 Air Eng (Pol) 020910      that they are implemented and that they solve the
                                                                                     identified deficiency?
            S & E Manager         68.3         GEN         POSMS/POEMS AAP04         How have proposed measures or actions taken         No guidance given.
            Project Manager                                Air Eng (Pol) Adv Audit   tracked?
            S & E Manager         68.4/AW074   AW          JSP 553 7.8               Did the designer maintain a Data Reporting,
            Project Manager                                                          Analysis and Corrective Action Systems
                                                                                     (DRACAS)?
             S & E Manager     68.4/AW075   AW             JSP 553 7.8               Was it used to record safety data during
            Project Manager                                                          development?
             S & E Manager     68.4/AW076   AW             JSP 553 7.8               Was it used to improve safety management within
            Project Manager                                                          the contractor's organisation?
 YES      Incident Investigation and Management


       Version 1.9.2 April 2008                                                                                                                                                                   Page 20 of 32
       ASEMS AUDIT QUESTION SET

Scoped Proposed level                Audit     Domain ID     Audit Requirements                       Audit Question                                   General Guidance for Auditor                          Domain Specific Guidance for Auditor        Evidence   Auditor's Opinion   Actions   Actual Score
       of Interviewee               Question                  Source Reference
                                      No.
            S & E Manager         69.0         GEN          POSMS SMP13               How do you manage incident reporting and               Looking for evidence of an explicitly defined procedure for
            Project Manager                                                           investigation ?                                        recording and investigating safety significant occurrences.
                                                                                                                                             Looking for evidence of formal records of performance of
                                                                                                                                             the safety significant equipment or systems.
                                                                                                                                             Looking for evidence of a system or procedure in
                                                                                                                                             operation for recording, investigating and learning from
                                                                                                                                             safety / environmentally significant occurrences that affect
                                                                                                                                             safety /environment.
                                                                                                                                             Looking for evidence that the range and scope of safety
                                                                                                                                             significant occurrences is recorded explained or defined.




            S & E Manager         69.1         GEN          POSMS SMP13               Does the investigation of safety significant          Looking for evidence of both safety management failures
            Project Manager                                 Air Eng (Pol) 020910      occurrences include the identification of any         and human factors failing.
                                                                                      failures ?
            S & E Manager         69.2         GEN          POSMS SMP13               Is there a process for disseminating 'lessons learnt' No guidance given.
            Project Manager                                 Air Eng (Pol) 020910      from hazardous occurrences, tests and trials ?

            S & E Manager         69.3         GEN          POSMS SMP13               How, and by whom, are relevant 'lessons learnt'        Looking for evidence about how and by whom it is decided
            Project Manager                                 Air Eng (Pol) 020910      identified and disseminated from hazardous             what information is disseminated.
                                                                                      occurrences ?
            S & E Manager         69.4         GEN          POSMS SMP13               How, and by whom, is the effectiveness of the          No guidance given.
            Project Manager                                 Air Eng (Pol) 020910      dissemination monitored?
            S & E Manager         69.5         GEN          POSMS SMP13               How is it confirmed that 'lessons learnt' are          No guidance given.
            Project Manager                                 Air Eng (Pol) 020910      incorporated appropriately into the safety culture
                                                                                      and the IPT‟s long-term memory?
 YES      Safety / Environmental Performance and Compliance Measurement
            S & E Manager    70.0            GEN           POSMS/POEMS AAP02,         How is compliance with your safety &               If not is there a similar document that formalises
            Project Manager                                POSMS SMP13,               environmental targets to be / being demonstrated? compliance with the safety targets (usually a safety
                                                           Air Eng (Pol) AVSG Audit                                                      analysis in design document)? Is the appropriate
                                                                                                                                         document identified in the Safety Programme Plan? Follow
                                                                                                                                         up refer to AAP02 - who, how often, validation.
            S & E Manager         70.1         GEN          POSMS/POEMS AAP02,        Have any safety or environmental aspects failed to Looking for evidence to ensure risks are ALARP.
            Project Manager                                 POSMS SMP13,              demonstrate compliance with mandated
                                                            JSP 520                   performance or safety acceptance criteria from
                                                                                      Legislation or Policy or Regulation ?
            S & E Manager         70.2         GEN          POSMS/POEMS AAP02,        How has compliance with applicable MOD and civil No guidance given.                                         SEA: Examples include
            Project Manager                                 POSMS SMP13,              statutory regulations been achieved?                                                                        '- Ship's rigging Warrant and the lifting plant
                                                            JSP 520                                                                                                                               register;
                                                            JSP 430 Part 2                                                                                                                        '- Ship's pressure vessel register;
                                                                                                                                                                                                  '- Submarine hazardous (toxicity and fire) materials
                                                                                                                                                                                                  register
                                                                                                                                                                                                  '- Naval implementation of international
                                                                                                                                                                                                  environmental requirements (MARPOL)
                                                                                                                                                                                                  '- Naval interpretation of safety and environmental
                                                                                                                                                                                                  noise requirements.

            S & E Manager         70.3         GEN          POSMS/POEMS AAP02,        How has compliance with applicable international                                                                      SEA: Examples include
            Project Manager                                 POSMS SMP13,              regulations and standards been achieved?                                                                              '- IMO treaties including SOLAS and MARPOL
                                                            JSP 520                                                                                                                                         '- EU Directives
                                                            JSP 430 Part 2                                                                                                                                  '- Foreign port-authority regulations
                                                                                                                                                                                                            '- Multilateral environmental agreements
            S & E Manager         70.4         GEN          Compliance                What is done if there is any legislation/ regulation
            Project Manager                                                           that you know you cannot meet ?
            S & E Manager         71.0         GEN          Air Eng (Pol) 020910      How do you carry out trend analysis of safety /        Looking specifically for equipment or systems failures,
            Project Manager                                                           environmentally significant incidents ?                evidence of environmental impacts.
            S & E Manager         72.0         GEN          Air Eng (Pol) 020910      Is there a method for determining if the               No guidance given.
            Project Manager                                                           deterioration in one area of safety significant
                                                                                      equipment or systems has a safety impact on
                                                                                      another area?




       Version 1.9.2 April 2008                                                                                                                                                                        Page 21 of 32
Date of Audit   Audit Reference Number           IPT/Project being audited
00/01/00                   0                                 0
                      Lead Auditor                      Interviewee
                           0                                 0
                  List:
   Audit        Maximum        Scoped       Calculated   Weighting   Maximum   Actual
Question No.    Question                     Question                Possible Question
                 Score                        Score                   Score    Score
    POLICY
SMP00: Project Safety Management System & Environmental Management System
1.0                     10               0             0           3          30         0
2.0                     10               0             0           2          20         0
3.0                     10               0             0           2          20         0
4.0                     10               0             0           2          20         0
5.0                     10               0             0           2          20         0
5.1                     10               0             0           2          20         0
MANAGEMENT REVIEW
6.0                     10               0             0           2          20         0
6.1                     10               0             0           1          10         0
6.2                     10               0             0           1          10         0
PLANNING
Safety & Environmental Stakeholders
7.0                     10               0             0           2          20         0
7.1                     10               0             0           1          10         0
Safety & Environmental Legislation / Policy / Standards and Certification Requirements Identification
8.0                     10               0             0           2          20         0
Safety Regulator / Approval / Assurance Authority Identification
9.0                     10               0             0           2          20         0
MOD's Safety / Environmental Management Plan
10.0                    10               0             0           5          50         0
10.1                    10               0             0           2          20         0
10.2                    10               0             0           1          10         0
10.3                    10               0             0           1          10         0
10.4                    10               0             0           1          10         0
10.5                    10               0             0           1          10         0
11.0                    10               0             0           2          20         0
11.1                    10               0             0           1          10         0
11.2                    10               0             0           1          10         0
11.3                    10               0             0           1          10         0
11.4                    10               0             0           1          10         0
11.5/OME001             10               0             0           1          10         0
11.6/AW001              10               0             0           1          10         0
11.6/AW002              10               0             0           1          10         0
Date of Audit   Audit Reference Number    IPT/Project being audited
00/01/00                   0                          0
                      Lead Auditor               Interviewee
                           0                          0
                 List:
   Audit         Maximum    Scoped      Calculated Weighting Maximum   Actual
Question No.     Question                Question            Possible Question
                  Score                   Score               Score    Score
11.6/AW003               10          0            0        1        10        0
12.0                     10          0            0        2        20        0
12.1                     10          0            0        1        10        0
12.2                     10          0            0        1        10        0
Screening & Scoping
13.0                     10          0            0        2        20        0
13.1                     10          0            0        1        10        0
13.2                     10          0            0        1        10        0
13.3                     10          0            0        1        10        0
14.0                     10          0            0        2        20        0
15.0                     10          0            0        2        20        0
16.0                     10          0            0        2        20        0
16.1                     10          0            0        1        10        0
17.0                     10          0            0        1        10        0
Environmental Impact Assessment Plan
18.0                     10          0            0        2        20        0
18.1                     10          0            0        1        10        0
18.2                     10          0            0        1        10        0
Impact Priority Evaluation
19.0                     10          0            0        2        20        0
19.1                     10          0            0        1        10        0
19.2                     10          0            0        1        10        0
Environmental Impact Assessment & Reporting
20.0                     10          0            0        2        20        0
20.1                     10          0            0        1        10        0
21.0                     10          0            0        2        20        0
22.0                     10          0            0        1        10        0
23.0                     10          0            0        2        20        0
24.0                     10          0            0        1        10        0
25.0                     10          0            0        2        20        0
25.1                     10          0            0        1        10        0
25.2                     10          0            0        1        10        0
25.3                     10          0            0        1        10        0
25.4                     10          0            0        1        10        0
26.0                     10          0            0        2        20        0
Date of Audit   Audit Reference Number        IPT/Project being audited
00/01/00                   0                              0
                      Lead Auditor                   Interviewee
                           0                              0
                 List:
   Audit          Maximum        Scoped    Calculated Weighting Maximum        Actual
Question No.      Question                  Question               Possible Question
                   Score                     Score                  Score      Score
27.0                        10          0            0          2         20          0
27.1                        10          0            0          1         10          0
27.2                        10          0            0          1         10          0
28.0                        10          0            0          2         20          0
Risk estimation, Risk & ALARP Evaluation, Risk Reduction & Risk Acceptance / Environmental Management Plan (Setting Objectives & Targets)
Risk Estimation
29.0                        10          0            0          3         30          0
29.1                        10          0            0          1         10          0
30.0                        10          0            0          2         20          0
30.1                        10          0            0          1         10          0
Criteria for Tolerability and ALARP
31.0                        10          0            0          3         30          0
31.1                        10          0            0          1         10          0
31.2                        10          0            0          1         10          0
32.0                        10          0            0          3         30          0
32.1                        10          0            0          1         10          0
32.2                        10          0            0          1         10          0
32.3                        10          0            0          1         10          0
Risk Reduction / Setting Environmental Objectives & Targets
33.0                        10          0            0          2         20          0
33.1                        10          0            0          1         10          0
33.2                        10          0            0          1         10          0
33.4                        10          0            0          1         10          0
33.5                        10          0            0          1         10          0
33.6                        10          0            0          1         10          0
33.7                        10          0            0          1         10          0
IMPLEMENTATION AND OPERATION
Safety & Environmental Resources
34.0                        10          0            0          2         20          0
35.0                        10          0            0          1         10          0
Safety & Environmental Management Organisation
36.0                        10          0            0          2         20          0
36.1                        10          0            0          1         10          0
Record of Safety & Environmental Management Responsibilities and Accountabilities
37.0                        10          0            0          2         20          0
Date of Audit   Audit Reference Number     IPT/Project being audited
00/01/00                   0                           0
                      Lead Auditor                Interviewee
                           0                           0
                 List:
   Audit         Maximum       Scoped     Calculated Weighting Maximum   Actual
Question No.     Question                  Question            Possible Question
                   Score                     Score              Score    Score
37.1                     10             0           0        1        10        0
Record of Safety & Environmental Management Interfaces
38.0                     10             0           0        2        20        0
39.0                     10             0           0        2        20        0
Safety Management Accountability and Delegation
40.0                     10             0           0        2        20        0
40.1                     10             0           0        1        10        0
40.2                     10             0           0        1        10        0
IPT / Project Safety Manager and Environmental Manager
41.0                     10             0           0        2        20        0
Document and Record Control
42.0                     10             0           0        2        20        0
42.1                     10             0           0        1        10        0
42.2                     10             0           0        1        10        0
42.3/AW004               10             0           0        1        10        0
42.3/AW005               10             0           0        1        10        0
Contractor Safety & Environmental Competence
43.0                     10             0           0        2        20        0
43.1                     10             0           0        1        10        0
43.2                     10             0           0        1        10        0
43.3/AW006               10             0           0        1        10        0
Training, Awareness & Competences
44.0                     10             0           0        3        30        0
44.1                     10             0           0        1        10        0
44.2                     10             0           0        1        10        0
44.3                     10             0           0        1        10        0
44.4                     10             0           0        1        10        0
44.5                     10             0           0        1        10        0
44.6                     10             0           0        1        10        0
44.7/AW007               10             0           0        1        10        0
Safety & Environmental Committee
45.0                     10             0           0        3        30        0
45.1                     10             0           0        1        10        0
45.2                     10             0           0        1        10        0
45.3                     10             0           0        1        10        0
Date of Audit   Audit Reference Number      IPT/Project being audited
00/01/00                   0                            0
                      Lead Auditor                 Interviewee
                           0                            0
                 List:
   Audit        Maximum         Scoped      Calculated Weighting Maximum   Actual
Question No.     Question                    Question            Possible Question
                   Score                      Score               Score    Score
45.4                     10              0            0        1        10        0
46.0                     10              0            0        2        20        0
46.1                     10              0            0        1        10        0
46.2                     10              0            0        1        10        0
46.3                     10              0            0        1        10        0
47.0                     10              0            0        1        10        0
Safety Requirements & Contracts
Types of Requirements
47.1/AW008               10              0            0        1        10        0
47.1/AW009               10              0            0        1        10        0
47.1/AW010               10              0            0        1        10        0
47.1/AW011               10              0            0        1        10        0
47.1/AW012               10              0            0        1        10        0
Hazard Identification & Analysis and Risk Estimation
48.0                     10              0            0        2        20        0
48.1                     10              0            0        1        10        0
48.2                     10              0            0        1        10        0
48.3                     10              0            0        1        10        0
48.4                     10              0            0        1        10        0
48.5                     10              0            0        1        10        0
48.6/OME002              10              0            0        1        10        0
49.0                     10              0            0        2        20        0
49.1                     10              0            0        1        10        0
49.2                     10              0            0        1        10        0
49.3                     10              0            0        1        10        0
49.4                     10              0            0        1        10        0
49.5                     10              0            0        1        10        0
49.6                     10              0            0        1        10        0
49.7                     10              0            0        1        10        0
49.8/AW013               10              0            0        1        10        0
49.8/AW014               10              0            0        1        10        0
49.8/AW015               10              0            0        1        10        0
49.8/AW016               10              0            0        1        10        0
Hazard Analysis
50.0                     10              0            0        2        20        0
Date of Audit   Audit Reference Number     IPT/Project being audited
00/01/00                   0                           0
                      Lead Auditor                Interviewee
                           0                           0
                 List:
   Audit       Maximum       Scoped      Calculated Weighting Maximum   Actual
Question No.   Question                   Question            Possible Question
                 Score                      Score              Score    Score
50.1                   10             0             0       1        10        0
50.2                   10             0             0       1        10        0
50.3                   10             0             0       1        10        0
50.4                   10             0             0       1        10        0
50.5                   10             0             0       1        10        0
50.6                   10             0             0       1        10        0
50.7/OME003            10             0             0       1        10        0
50.7/OME004            10             0             0       1        10        0
Risk Acceptance
51.0                   10             0             0       3        30        0
51.1                   10             0             0       1        10        0
51.2                   10             0             0       1        10        0
51.3/OME005            10             0             0       1        10        0
51.3/OME006            10             0             0       1        10        0
Hazard Log
52.0                   10             0             0       2        20        0
52.1                   10             0             0       1        10        0
52.2                   10             0             0       1        10        0
52.3                   10             0             0       1        10        0
52.4                   10             0             0       1        10        0
52.5                   10             0             0       1        10        0
Operational Controls
53.0                   10             0             0       3        30        0
54.0                   10             0             0       3        30        0
54.1/OME008            10             0             0       1        10        0
55.0                   10             0             0       3        30        0
Safety Case & Case Report
Management of Safety Case information and its content
56.0                   10             0             0       3        30        0
56.1                   10             0             0       1        10        0
56.2                   10             0             0       1        10        0
56.3                   10             0             0       1        10        0
56.4                   10             0             0       1        10        0
56.5/OME009            10             0             0       1        10        0
Safety Case Conclusion
Date of Audit   Audit Reference Number           IPT/Project being audited
00/01/00                   0                                 0
                      Lead Auditor                      Interviewee
                           0                                 0
                 List:
   Audit        Maximum       Scoped         Calculated Weighting Maximum   Actual
Question No.    Question                      Question            Possible Question
                 Score                         Score               Score    Score
57.0                                     0             0        4         0        0
57.1                     10              0             0        1        10        0
57.2                     10              0             0        1        10        0
57.3                     10              0             0        1        10        0
57.4                     10              0             0        1        10        0
57.5                     10              0             0        1        10        0
57.6                     10              0             0        1        10        0
57.7                     10              0             0        1        10        0
57.8/AW017               10              0             0        1        10        0
57.8/AW018               10              0             0        1        10        0
57.8/AW019               10              0             0        1        10        0
57.8/AW020               10              0             0        1        10        0
57.8/AW021               10              0             0        1        10        0
57.8/AW022               10              0             0        1        10        0
57.8/AW023               10              0             0        1        10        0
57.8/AW024               10              0             0        1        10        0
57.8/AW025               10              0             0        1        10        0
57.8/AW026               10              0             0        1        10        0
57.8/AW027               10              0             0        1        10        0
57.8/AW028               10              0             0        1        10        0
57.8/AW029               10              0             0        1        10        0
57.8/AW030               10              0             0        1        10        0
57.8/AW031               10              0             0        1        10        0
57.8/AW032               10              0             0        1        10        0
57.8/AW033               10              0             0        1        10        0
57.8/AW034               10              0             0        1        10        0
57.8/AW035               10              0             0        1        10        0
57.8/AW036               10              0             0        1        10        0
57.8/AW037               10              0             0        1        10        0
57.8/AW038               10              0             0        1        10        0
57.8/AW039               10              0             0        1        10        0
57.8/AW040               10              0             0        1        10        0
57.8/AW041               10              0             0        1        10        0
57.8/AW042               10              0             0        1        10        0
57.8/AW043               10              0             0        1        10        0
Date of Audit   Audit Reference Number         IPT/Project being audited
00/01/00                   0                               0
                      Lead Auditor                    Interviewee
                           0                               0
                 List:
   Audit         Maximum     Scoped        Calculated Weighting Maximum   Actual
Question No.     Question                   Question            Possible Question
                   Score                     Score               Score    Score
57.8/AW044               10              0           0        1        10        0
Safety Case Report
58.0                     10              0          0           2          20    0
58.1                     10              0          0           1          10    0
58.2                     10              0          0           1          10    0
58.3                     10              0          0           1          10    0
58.4                     10              0          0           1          10    0
58.5/OME007              10              0          0           1          10    0
58.5/OME008              10              0          0           1          10    0
58.5/OME009              10              0          0           1          10    0
58.5/OME010              10              0          0           1          10    0
58.5OME011               10              0          0           1          10    0
58.5/OME012              10              0          0           1          10    0
Safety Case Review and Audit
59.0                     10              0          0           1          10    0
Defining the in-service SMS
60.0                     10              0          0           2          20    0
60.1/AW045               10              0          0           1          10    0
60.1/AW046               10              0          0           1          10    0
60.1/AW047               10              0          0           1          10    0
60.1/AW048               10              0          0           1          10    0
61.0                     10              0          0           2          20    0
In-Service SMS
62.0                     10              0          0           2          20    0
62.1                     10              0          0           1          10    0
63.0                     10              0          0           2          20    0
Safety Assurance during Changes
64.0                     10              0          0           3          30    0
64.1                     10              0          0           1          10    0
64.2                     10              0          0           1          10    0
64.3/AW049               10              0          0           1          10    0
64.3/AW050               10              0          0           1          10    0
Release to Service
64.4/AW051               10              0          0           1          10    0
64.4/AW052               10              0          0           1          10    0
Date of Audit   Audit Reference Number     IPT/Project being audited
00/01/00                   0                           0
                      Lead Auditor                Interviewee
                           0                           0
                 List:
   Audit         Maximum      Scoped      Calculated Weighting Maximum   Actual
Question No.     Question                  Question            Possible Question
                   Score                    Score               Score    Score
64.4/AW053                10           0            0        1        10        0
64.4/AW054                10           0            0        1        10        0
64.4/AW055                10           0            0        1        10        0
64.4/AW056                10           0            0        1        10        0
64.4/AW057                10           0            0        1        10        0
64.4/AW058                10           0            0        1        10        0
64.4/AW059                10           0            0        1        10        0
64.4/AW060                10           0            0        1        10        0
64.4/AW061                10           0            0        1        10        0
64.4/AW062                10           0            0        1        10        0
64.4/AW063                10           0            0        1        10        0
64.4/AW064                10           0            0        1        10        0
64.4/AW065                10           0            0        1        10        0
64.4/AW066                10           0            0        1        10        0
64.4/AW067                10           0            0        1        10        0
64.4/AW068                10           0            0        1        10        0
64.4/AW069                10           0            0        1        10        0
64.4/AW070                10           0            0        1        10        0
64.4/AW071                10           0            0        1        10        0
64.4/AW072                10           0            0        1        10        0
64.4/AW073                10           0            0        1        10        0
CHECKING AND CORRECTIVE ACTION
Safety Case Configuration Control and Currency
65.0                      10           0            0        3        30        0
Independent Safety Auditor
66.0                      10           0            0        2        20        0
66.1                      10           0            0        1        10        0
66.2                      10           0            0        1        10        0
Audit (Internal and Peer)
67.0                      10           0            0        3        30        0
67.1                      10           0            0        1        10        0
67.2                      10           0            0        1        10        0
67.3                      10           0            0        1        10        0
67.4                      10           0            0        1        10        0
Non-Conformance and Corrective Action
Date of Audit   Audit Reference Number     IPT/Project being audited
00/01/00                   0                           0
                      Lead Auditor                Interviewee
                           0                           0
                 List:
   Audit         Maximum      Scoped     Calculated Weighting Maximum   Actual
Question No.     Question                 Question            Possible Question
                  Score                    Score               Score    Score
68.0                     10            0           0        2        20        0
68.1                     10            0           0        1        10        0
68.2                     10            0           0        1        10        0
68.3                     10            0           0        1        10        0
68.4/AW074               10            0           0        1        10        0
68.4/AW075               10            0           0        1        10        0
68.4/AW076               10            0           0        1        10        0
Incident Investigation and Management
69.0                     10            0           0        3        30        0
69.1                     10            0           0        1        10        0
69.2                     10            0           0        1        10        0
69.3                     10            0           0        1        10        0
69.4                     10            0           0        1        10        0
69.5                     10            0           0        1        10        0
Safety / Environmental Performance and Compliance Measurement
70.0                     10            0           0        3        30        0
70.1                     10            0           0        1        10        0
70.2                     10            0           0        1        10        0
70.3                     10            0           0        1        10        0
70.4                     10            0           0        1        10        0
71.0                     10            0           0        1        10        0
72.0                     10            0           0        2        20        0
IPT/Project                                                                        0
Audit Reference                                                                    0
Date of Audit                                                                      00/01/00
Lead Auditor                                                                       0

                                                                                    Possible     Actual
Section                                                            Questions          Score      Score Section %            Section % cells formatted
POLICY                                                               1-5                130          0       0.00
PLANNING                                                             7-33              1020          0       0.00           0 - 19%
IMPLEMENTATION & OPERATION                                          34-64              2120          0       0.00           20 - 44%
CHECKING AND CORRECTIVE ACTION                                      65-72               400          0       0.00           45 - 69%
MANAGEMENT REVIEW                                                      6                 40          0       0.00           70 - 100%



                                                                TOTAL SCORE            3710          0       0.00


AUDIT LEVEL SCORE criteria                                                     0   Any section <20%
(for explanation of each level see General Information sheet)                  1   Overall % >25 AND minimum of 20% in each section
                                                                               2   Overall % >40 AND minimum of 30% in each section
                                                                               3   Overall % >55 AND minimum of 45% in each section
                                                                               4   Overall % >70 AND minimum of 55% in each section
                                                                               5   Overall % >85 AND minimum of 70% in each section

                                         AUDIT LEVEL ACHIEVED



                                                                Level 0

				
DOCUMENT INFO