NEH Safety Systems - Stanford Synchrotron Radiation Lightsource by liwenting

VIEWS: 20 PAGES: 24

									                     NEH Safety Systems

                           Enzo Carrone
                          June 30th , 2009




NEH Safety Systems                1   1                       Enzo Carrone
NEH ARR 2009                                 Enzo.Carrone@slac.stanford.edu
                        PPS and BCS Status


     Both systems are currently ready for safe
     operation to the FEE
          PPS successfully certified (IAT) on June 24-26;
          BCS components certified on June 26.

     Hutch systems
          HPS will be ready by the end of July.




NEH Safety Systems                   2   2                           Enzo Carrone
NEH ARR 2009                                        Enzo.Carrone@slac.stanford.edu
                      Safety Systems Section


        PPS and BCS Systems combined into a single
        Section.
        Full-time QC Group Leader (M. Saleski).
             Manages Review Process.
             Oversees Engineering Change Order Process.

        Full-time Documentation Manager (S. Starner).
             Documentation system compliant to DOE Order
             414.1C and ISO 9001-2000.


NEH Safety Systems                3   3                         Enzo Carrone
NEH ARR 2009                                   Enzo.Carrone@slac.stanford.edu
NEH Safety Systems   4   4                    Enzo Carrone
NEH ARR 2009                 Enzo.Carrone@slac.stanford.edu
                     Safety Systems Section Documentation Structure




NEH Safety Systems                          5   5                            Enzo Carrone
NEH ARR 2009                                                Enzo.Carrone@slac.stanford.edu
                                           Management of Change Plan
                                             CD-SS-MAN-00-01-02




          Design Review Plan                                                    Document Management Plan
         CD-SS-MAN-00-01-03                                                       CD-SS-MAN-00-02-02




       Engineering Change Order                        Software Configuration Management
         CD-SS-MAN-00-02-07                                  CD-SS-MAN-00-02-01




                               SLAC Guidelines for Operations
                                  SLAC-I-010-00100-000




NEH Safety Systems                                        6     6                                  Enzo Carrone
NEH ARR 2009                                                                      Enzo.Carrone@slac.stanford.edu
NEH Safety Systems   7   7                    Enzo Carrone
NEH ARR 2009                 Enzo.Carrone@slac.stanford.edu
                              Safety Systems QA

       Subject to a new Engineering Change Order (ECO)
       process
            Better tracking of requirements, specifications, reviews (peer and
            formal), and system documentation.
       New project QA approach
            Bench and pre-IAT field test of system software
            Acceptance tests performed for complex SLAC-built assemblies
            (such at stoppers)
            Cold and energized checkout of installed hardware
                 Und Complex lessons learned with subcontractors
            Formal IAT reviews with stakeholders
       Overall, better management of schedule
            Indirectly affects quality




NEH Safety Systems                            8   8                                 Enzo Carrone
NEH ARR 2009                                                       Enzo.Carrone@slac.stanford.edu
                      Documentation Control


        A single on line repository is available for all
        released documents.
        Formal Documentation Control is in place.
             Measures taken to regulate the Preparation,
             Change, Withdrawal, Review, Approval, Release,
             Distribution, Access, Availability, Storage,
             Disposal of documents.
             Document management Plan, Document Change
             Control Procedure, design Review Plan, Drawing
             Management Procedure, Engineering Change
             Order Procedure.
NEH Safety Systems                9   9                         Enzo Carrone
NEH ARR 2009                                   Enzo.Carrone@slac.stanford.edu
                     Configuration Control Elements

     Prevention of Unintended Change
          Physical Security of System
          Labeling
          Training

     Control of Intended Change
          Work Planning (adequate review of design)
          Work Authorization (RSWCF)
          Verification of Work (RSWCF)

     Periodic Confirmation of System Integrity
          Routine testing and inspections

NEH Safety Systems                   10 10                         Enzo Carrone
NEH ARR 2009                                      Enzo.Carrone@slac.stanford.edu
              Configuration Management Documentation



            Document Management Plan
                 Document Change Control Procedure
                     Document Change Order
            Design Review Plan
                 Software Configuration Management
                 Engineering Change Order Procedure
                     Engineering Change Order
            Drawing Management Procedure

NEH Safety Systems                     11   11                    Enzo Carrone
NEH ARR 2009                                     Enzo.Carrone@slac.stanford.edu
                     Software Configuration Management Support


       Software versions are checked during annual
       certification.
       Written procedures exist for extracting PPS
       code from CVS and for uploading it to
       PLCs.
       A documented training program tracks
       personnel PLC qualifications in the Section.



NEH Safety Systems                        12 12                          Enzo Carrone
NEH ARR 2009                                            Enzo.Carrone@slac.stanford.edu
                 Software Configuration Management Procedure

         PPS Software is stored
         in a dedicated PPS
         repository.
         Released software
         always has “N.0.0”
         version tag.
         Documented software
         bench testing is
         performed prior to
         deployment.



NEH Safety Systems                    13 13                          Enzo Carrone
NEH ARR 2009                                        Enzo.Carrone@slac.stanford.edu
               Software Configuration Control Issues
     Program Security:
          All communication to the safety-critical PLCs is
          through TCP/IP to ‘buffer’ Allen-Bradley PLC, then via
          DeviceNet (serial data communication).
          Safety-critical program ‘smart card’ cannot be written
          to while in the PLC
          Communication from the safety-critical PLCs is
          through DeviceNet to ‘buffer’ Allen-Bradley PLC and
          output to control system via TCP/IP
     Network Access Security:
          Hardwire Enable from MCC required
          Only specific IP addresses are allowed to issue PPS
          commands
     Physical Access Security:
          PLCs and DeviceNet are inside locked racks.

NEH Safety Systems                   14 14                           Enzo Carrone
NEH ARR 2009                                        Enzo.Carrone@slac.stanford.edu
                     Software Version Management


        Version Management:

             Software versions are checked during annual certification
             Written procedures Exist for extracting PPS code from CVS and
             uploading it to PLCs
             A documented training program tracks personnel PLC
             qualifications in the Section




NEH Safety Systems                        15 15                             Enzo Carrone
NEH ARR 2009                                               Enzo.Carrone@slac.stanford.edu
              Implementation, Operations, and Maintenance
                                                  Initiate RSWCF

                                                Implement Change
                                                                                                       Development
                                                                          Problems
                                               Initial Acceptance Test                                 and Review
                                                          Success
                                                                                                          Cycle
                                                   Close RSWCF


                              12 Months                                   Problems
                                               Safety Assurance Test
                                                           Success
                              6 Months                                    Problems
                                                  Interlock Checks
                                                           Success




                            Routine Testing
                            Per Guideline 27                             Problems
                                                System in Operation



                                                                             Assessment of Failure
                            Correct the Procedure                          Is the Failure Reportable?
                                                                                      Undesired
                                                                         Procedure                    Failed
                                                                                     Functionality
                                                                           Error                     Hardware
                                                                                      Discovered




                                                                                               Initiate RSWCF;
                          Assess Failure with RSO                                              Determine Tests
                       Administrative     Engineering
                         Mitigation        Change                                             Repair Hardware

                                                                                                                   Problems
                                                                                               Re-perform Test
                                                                                                         Success
                                                             Need for New
                                                              Functional                        Close RSWCF
                                                             Requirements




NEH Safety Systems                                                             16 16                                                           Enzo Carrone
NEH ARR 2009                                                                                                                  Enzo.Carrone@slac.stanford.edu
                     Proposed PLC PPS ‘Dev and Rev’
                                                                                Need for New PPS
                                                                                    System


                                                                                  Safety Functions
                                                                                   Requirements
                                                                                   Specification



                                               Validation Scope and
                                                                                Software Functions            Hardware Functions
                                                   Methodology
                                                                                  Determination                 Determination
                                                   Determination



                                                                                 Preliminary Design
                                                              Rework Proposal
                                                                                       Review
                                                                                (Project and RSO/RSC)
                                                                                            Success




                                                                                 Withdraw Software
                                                                                from Version-Control
                                                                                     Repository
                                                                                Software Design and
                                                                                    Development                         Hardware Design and
                                                                                                                            Development
                                    Safety Validation
                                        Planning                    Rework
                                                                    Software         Software
                                                                                   Bench Testing
                                              Bench Testing
                                               Specified?                                    Success

                                                                                Deposit Software in
                                                                                  Version-Control
                                                                                    Repository
                                                                                Assign New Version
                                                                                     Number

                                        Validation                                                        System Technical
                                                                                        Rework Software                           Rework Hardware
                         Rework     Procedure Review                                                       Design Review
                        Procedure                                                                         (Project and RSO/RSC)
                                            Success                                                                   Success




                                               Lifecycle Special Functions Key                            Implementation,
                                                                                                          Operations, and
                                             System Review                 System in
                                             or Assessment                 Operation
                                                                                                           Maintenance
                                                                                                               Cycle
                                             System Testing                Additional
                                              or Validation                  Cycle




NEH Safety Systems                                                                                17 17                                                              Enzo Carrone
NEH ARR 2009                                                                                                                                        Enzo.Carrone@slac.stanford.edu
                                                        Established SSS ECO
              Memo to Engineering
    Start     Group Leader initiating
                 change request

                                                         Notification to
                                                         ADSO/RSO of
                                                            job, risk
              NO      Proposal                            assessment,
                      Accepted

     Drop
    Request                 Yes
                                                        and review plan
                                                                              Emphasis on review and
                   Assign Engineer
                    ECO Initiated
                                                       RSO/RSC Review         development documentation;
                   Scope Defined
                                                                              Enables a project
                       Formal
                                                                              management approach;
                    Requirements
                     Produced                              Action Item


                   Engineer/Design
                                                          Management
                                                            System            Collects development and
                        Work
                                                        Implement Action
                                                             Items
                                                                              review docs for auditability.
                 Weekly Peer-
               Review ‘as needed,’
                 including Risk                        Action Item Close-
                  Assessment                           out Memo Issued


                                                         RSWCF
                      Ready for                    (and WAF if applicable)
              NO
                       Formal
                      Review?
                                                       Work is Performed

                            Yes
                                                       Assessment of work
                                                        performed during
                   Conduct Formal                       weekly meetings
                      Review

                                            Yes

                                                            Evaluation by
                                                  NO          Controls
                                        Approve
                                                             Department
                                                            Management




NEH Safety Systems                                                           18 18                          Enzo Carrone
NEH ARR 2009                                                                               Enzo.Carrone@slac.stanford.edu
                     FEE/NEH PPS Checkouts

     Cold Checkout
          Verifies Hardware and Cable Plant is installed
          and wired correctly
          Verifies No Ground Shorts in the Cable Plant
          EEIP Hardware and Racks

      Hot Checkout
           Verifies Functionality of Hardware, Devices,
           and PLCs
           Hi-Pot Cable Plant

NEH Safety Systems               19 19                         Enzo Carrone
                                                                      19
NEH ARR 2009                                  Enzo.Carrone@slac.stanford.edu
                         Review Status

     Final Design Review #2 (April 22, 2009)

     Initial Acceptance Test Review (internal
     and external)
          Starting this week.




NEH Safety Systems              20 20                     Enzo Carrone
                                                                 20
NEH ARR 2009                             Enzo.Carrone@slac.stanford.edu
                     PPS Review Status for FEE & NEH

      Major Review to Date
           LCLS SCR: May 31st 2007 [Bong, Stefan, Horton, Hastings]
           LCLS PDR: Feb 29th 2008 [Lessard, Tompkins, Schmerge,
           Anthony]
           LCLS FDR: Oct 22nd 2008 [Anthony, Mueller(LBNL), Lessard]
           RSC: Oct 31st 2008
           LCLS FDR2: Apr 22nd 2009 [Perry Anthony, Zoe Van
           Hoover, Luc Lessard, Paul Miller, Robert Mueller, Mike Woods]




NEH Safety Systems                        21 21                             Enzo Carrone
NEH ARR 2009                                               Enzo.Carrone@slac.stanford.edu
                     PPS FEE Schedule




NEH Safety Systems           22 22                       Enzo Carrone
                                                                22
NEH ARR 2009                            Enzo.Carrone@slac.stanford.edu
                     FEE/NEH Schedule




NEH Safety Systems           23 23                       Enzo Carrone
NEH ARR 2009                            Enzo.Carrone@slac.stanford.edu
                            Conclusions


      PPS/BCS has a Quality Assurance model in place;

      LCLS management is aware of the Safety Systems
      process and supports it throughout the project lifecycle
      (from conception to certification);

      LCLS, Operations, Radiation Physics are heavily
      involved in all review phases from early on (from
      requirements/specifications to user manual).




NEH Safety Systems                  24 24                            Enzo Carrone
NEH ARR 2009                                        Enzo.Carrone@slac.stanford.edu

								
To top