National Risk Management Research Laboratory NRMRL Quality Management Plan QMP - Quality System

QUALITY MANAGEMENT PLAN for the National Risk Management Research Laboratory (NRMRL) Office of Research and Development U.S. Environmental Protection Agency March 2002 Document Control Number NRMRLQA 001 rev 1 FOREWORD This Quality Management Plan for the National Risk Management Research Laboratory (NRMRL) was developed by NRMRL QA Team consisting of: Nancy H. Adams Quality Assurance Manager Air Pollution Prevention & Control Division Research Triangle Park, North Carolina Lauren Drees Quality Assurance Manager Sustainable Technology Division Cincinnati, Ohio Jim Owens Quality Assurance Manager Water Supply & Water Resources Division Cincinnati, Ohio Steve Vandegrift Quality Assurance Manager Subsurface Protection & Remediation Division Ada, Oklahoma Ann Vega Quality Assurance Manager Land Remediation & Pollution Control Division Cincinnati, Ohio Tom Wagner Director of Quality Assurance NRMRL & Quality Assurance Manager Technology Transfer and Support Division Cincinnati, Ohio The following individuals in the NRMRL QA staff contributed to authoring and reviewing the contents of this Quality Management Plan. Holly Ferguson, APPCD Paul Groff, APPCD Shirley Wasson, APPCD Sam Hayes (former QAM, WSWRD) ii National Risk Management Research Laboratory Quality Management Plan March 2002 APPROVALS ____Signed______________ Signature _04/22/02_ Date E. Timothy Oppelt, Director NRMRL _____Signed_____________ Signature _04/18/02_ Date Thomas J. Wagner, Director of Quality Assurance NRMRL ____Approved____________ Signature _06/17/02_ Date Nancy Wentworth, Director Quality Staff Office of Environmental Information (OEI) ________________________ Signature __________ Date Margaret N. Schneider, Acting Assistant Administrator OEI iii National Risk Management Research Laboratory Quality Management Plan March 2002 CONCURRENCES ____Signed______________ Signature _04/01/02_ Date Frank T. Princiotta, Director Air Pollution Prevention & Control Division ____Signed______________ Signature _04/15/02_ Date Robert A. Olexsey, Director Land Remediation & Pollution Control Division ____Signed______________ Signature _04/05/02_ Date Stephen Schmelling, Acting Director Subsurface Protection & Remediation Division ____Signed______________ Signature _04/17/02_ Date Subhas K. Sikdar, Director Sustainable Technology Division ____Signed______________ Signature _04/15/02_ Date Daniel J. Murray Jr., Director Technology Transfer and Research Division ____Signed______________ Signature _04/15/02_ Date Sally Gutierrez, Director Water Supply & Water Resources Division iv March 27, 2002 TABLE OF CONTENTS FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii APPROVALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii CONCURRENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv TABLE OF CONTENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v ABBREVIATIONS AND ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x 1.0 MANAGEMENT AND ORGANIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 NRMRL Mission Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 NRMRL Organizational Policy on Quality Assurance . . . . . . . . . . . . . . . . . . . . . 1 1.3 QA Organization in NRMRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 QA Roles and Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.5 Activities, Programs, and Legislative Acts Supported by the NRMRL Quality System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 QUALITY SYSTEM AND DESCRIPTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1 Graded Approach to Research Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 Quality-Affecting Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3 Principal Components of the Quality System . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 NRMRL Quality Management Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.2 Quality Assurance Annual Report and Work Plan (QWAARWP) . . . . . 14 2.3.3 Extramural Quality Management Plans . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.4 Systematic Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.5 Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3.6 Data Quality Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.7 QA Review of Research Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.8 Computer Tracking System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.9 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 PERSONNEL QUALIFICATIONS AND TRAINING . . . . . . . . . . . . . . . . . . . . . . . . . 20 PROCUREMENT OF ITEMS AND SERVICES (Extramural Actions) . . . . . . . . . . . . 22 4.1 Scope and Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.2 Competitive Extramural Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.2.1 Single-Project Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.2 Multiple-Project Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.3 Alternative Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 v 2.0 3.0 4.0 March 27, 2002 4.3 4.4 4.5 4.6 5.0 4.2.4 Additional Requirements for Competitive Extramural Actions Exceeding $500,000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Non-Competitive Extramural Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 QA Responsibilities on IAGs and CRADAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Evaluation of Quality Deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Evaluation of NRMRL’s Compliance with QMP . . . . . . . . . . . . . . . . . . . . . . . . 25 DOCUMENTATION AND RECORDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.1 EPA Records Management Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.2 Project Documentation and Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.3 Research Laboratory-Specific Documentation and Records . . . . . . . . . . . . . . . . 27 5.4 Software Documentation and Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 5.5 NRMRL Quality Documentation and Records . . . . . . . . . . . . . . . . . . . . . . . . . . 27 COMPUTER HARDWARE AND SOFTWARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 PLANNING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 IMPLEMENTATION OF WORK PROCESSES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 8.1 QAPP Development, Review, and Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 8.2 SOP Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 8.3 QAPP/SOP Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 8.4 QAPP/SOP Revisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 ASSESSMENT AND RESPONSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9.1 Assessment Techniques and Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9.2 Types of Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9.2.1 Self-Assessment of a Quality System . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9.2.2 Independent Assessment of a Quality System . . . . . . . . . . . . . . . . . . . . . 34 9.2.3 Technical Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9.2.4 Independent Technical Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 9.3 Assessor Capabilities and Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 9.4 NRMRL-Wide Assessment and Response Procedures . . . . . . . . . . . . . . . . . . . . 35 9.5 Division Assessment and Response Procedures . . . . . . . . . . . . . . . . . . . . . . . . . 36 9.6 “Stop Work” Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 9.7 Assessment Reports and Corrective Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 QUALITY IMPROVEMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 NRMRL MISSION STATEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 6.0 7.0 8.0 9.0 10.0 11.0 Appendix A vi March 27, 2002 Appendix B Appendix C Appendix D Appendix E REQUIREMENTS AND GUIDANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 QA PLANNING REQUIREMENTS (QAPR) FORMS . . . . . . . . . . . . . . . . . . C-1 NRMRL PRODUCT QA/QC VERIFICATION REPORT FORM . . . . . . . . . D-1 NRMRL MINIMUM REQUIREMENTS FOR LABORATORY RESEARCH ACTIVITIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-1 vii March 27, 2002 ABBREVIATIONS AND ACRONYMS ADP ADQ ANSI APPCD ASQ BC CMD CRADA DD DQA DQA DQI DQO EPA FTE FY GALP H&S IAG LAN LRPCD NDPD NRMP NRMRL OEI OGD OIRM ORD PCSC PE QA QAARWP QAM QAPP QC QMP QS QS QSA RMO RTP Automated Data Processing Audit of Data Quality American National Standards Institute Air Pollution Prevention & Control Division American Society for Quality Branch Chief Contracts Management Division Cooperative Research and Development Agreement Division Director Data Quality Assessment Director of Quality Assurance Data Quality Indicator Data Quality Objective US Environmental Protection Agency Full-Time Equivalent Fiscal Year Good Automated Laboratory Practices Health and Safety Interagency Agreement Local Area Network Land Remediation & Pollution Control Division National Data Processing Division National Records Management Plan National Risk Management Research Laboratory Office of Environmental Information Office of Grants and Debarment Office of Information Resources Management Office of Research & Development Personal Computer Site Coordinator Performance Evaluation Quality Assurance Quality Assurance Annual Report & Work Plan Quality Assurance Manager Quality Assurance Project Plan Quality Control Quality Management Plan Quality Staff Quality System Quality System Audit Records Management Office Research Triangle Park viii March 27, 2002 SOP SPRD STD TEP TLP TSCA TSA TTSD WSWRD Standard Operating Procedure Subsurface Protection & Remediation Division Sustainable Technology Division Technical Evaluation Panel Technical Lead Person Toxic Substances Control Act Technical System Audit Technology Transfer & Support Division Water Supply & Water Resources Division ix March 27, 2002 LIST OF FIGURES Number Figure 1. Title NRMRL Organization Chart Including QA Personnel Page 2 x March 27, 2002 1.0 Management and Organization The National Risk Management Research Laboratory (NRMRL) has six divisions (four based in Cincinnati with one branch in Edison, NJ, one division each in RTP, NC and Ada, OK). The organizational structure is depicted in Figure 1. 1.1 NRMRL Mission Statements NRMRL has established a mission statement, as has each division within its organization. Appendix A contains these mission statements. 1.2 NRMRL Organizational Policy on Quality Assurance The NRMRL policy on quality assurance (QA) is: • To ensure that NRMRL meets EPA QA requirements as defined in EPA Order 5360.1, A2 (or the most recent change to the order see http://www.epa.gov/quality/qs_docs.html); To implement a quality system that guides QA planning for individual projects and QA reviews of all activities in NRMRL; and To ensure that continuous improvement is practiced in the implementation of the NRMRL quality system. To ensure that adequate resources (including FTEs, contractor support, and travel funds) are provided to implement the NRMRL QMP across the Laboratory; • • • The purpose of this Quality Management Plan (QMP) is to describe the NRMRL quality system. The NRMRL quality system is designed to be usable and useful to project personnel, managers, and NRMRL QA staff. The quality system at NRMRL incorporates quality policy and lines of authority and responsibility for all laboratory personnel. NRMRL’s management and quality staff work to ensure the following: • standard policies and procedures are in place for the quality system elements described in the NRMRL quality system; all environmental data collection, evaluation, and use are performed in accordance with an approved planning document; and all NRMRL projects (both intramural and extramural) produce defensible data of defined quality. 1 • • March 27, 2002 N ational R isk M anagem ent R esearch Laboratory Laboratory D irector Tim O ppelt 513-569-7418 C i ndy K irchm er, x7418 Deputy D irector for M anagem ent Rotationa l Pos ition, x7391 Ruth Corn, x7920 Resources O perat ions Staff Jenn i fer Connoll y, x7963 Assoc i ate Laboratory D irectors Hugh M cK innon, x7689 Lee M u lkey, x7689 Ass i stant Laboratory D irectors Andy Ave l, x7951 Ben Blaney, x7852 Clyde Dem psey, x7842 Jon Herrm ann, x7839 Doug M cK inney, 919-541-3006 Air Poll ut i on Prevent ion and Contro l Divisi on Frank Pr i nc iotta, 919-541-2822 Kit C happe ll , x2823 Land Rem ediat i on and Poll ut i on Contro l Division Bob Olexsey, 513-569-7861 Pat W hee ler, x7861 A n n V eg a, Q A M x7635 Adm ini strat ive Operat ions Staff Dean ie G agnon, x7510 Techn ical Serv i ces Branch W ade Ponder, x2818 Nancy Adam s, QA M x5510 Em issions Character i zat ion and Prevent ion Branch Larry Jones, x7716 Atm ospher ic Protect ion Branch Bill R hodes, x2853 Indoor E nv ironm ent M anagem ent Branch M i ke O sborne, x4113 Air Po lluti on Techno l ogy Branch Bob H a ll , x2477 Treatm ent and D estruct i on Branch Laurel S taley, x7863 Rem ediat ion and C onta inm ent Branch John M art i n, x7758 Site M anagem ent Support B ranch Trish Erickson, x7406 Subsurface Protect ion and R em ediat i on D ivisi on Steve Schm elling, Act., 580-436-8511 Vacant Susta i nab le Techno l ogy Div isi on Subhas S ikdar, 513-569-7528 Jane W ilm oth, x7601 La uren D rees, Q AM x7087 Technica l and Adm ini strati ve Support S taff Roger Cosby, x8512 Steve Vandegrift, QA M x8684 Ecosystem and Subsurface Protection Branch Joe W i lliam s, x8608 Subsurface Rem ediat ion Branch Bob Pu ls, x8543 Techn ical A ss istance and Techno logy Transfer Branch Jerry Jones, x8593 Industr ial M u ltimed ia Branch Roger W ilm oth, x7509 Clean Processes Branch Vacant System s Ana lysis Branch G ordon Evans, x7684 Sustainab l e Env ironm ents Branch H er i berto C abezas, x7350 Techno l ogy Transfer and Support D i vision Dan M urray, 513-569-7522 Den ise R at liff, x7867 Tom W a gner, D Q A x7 013 Techno logy C oordinat ion S taff Teresa Harten, 513-569-7565 W ater Supply and W ater Resources D ivis ion Sa lly G ut i errez, 513-569-7683 Susan Salzl, x7862 Jim O w ens, Q A M x7235 Techno logy Transfer B ranch Lynnann Par i s, x7672 Technica l Inform at ion Branch Caro l G rove, x7362 Techn i ca l O perati ons Branch John Ire l and, x7413 Urban W atershed M anagem ent Branch Dan Su llivan, 732-321-6677 Treatm ent Techno logy Evaluat i on Branch Bob Thurnau, x7504 M i crob ial Contam inants Contro l Branch D on Reasoner, x7234 W ater Qua lity M anagem ent Branch Jim G oodr ich, x7605 Figure 1. NRMRL Organization Chart Including QA Personnel 2 March 27, 2002 1.3 QA Organization in NRMRL The QA organization in NRMRL is shown in Figure 1. The QA Managers (QAM) of the Air Pollution Prevention and Control Division (APPCD) and Subsurface Protection and Remediation Division (SPRD) are assigned to branches within their respective divisions; NRMRL’s Director of QA (DQA) is assigned to the Technology Transfer and Support Division (TTSD). The Technical Services Branch of APPCD, the Technical and Administrative Support Staff of SPRD, and TTSD are support functions, not research groups. Therefore, the required independence of the QA function is maintained. 1.4 QA Roles and Responsibilities NRMRL Laboratory Director is responsible for: • Ensuring that EPA quality policy is implemented at NRMRL by including quality responsibilities as described in this QMP in the performance standards of all NRMRL Division Directors, Branch Chiefs, QA personnel, and Technical Lead Persons; Promoting (by words, actions, and involvement) the establishment of a quality culture at NRMRL; Promoting (by words, actions, and involvement) continuous quality improvement within NRMRL; Providing facilities that are conducive to the production of data of appropriate quality to meet the needs of the research being conducted; Providing adequate resources (including full-time equivalents [FTE], contractor support, and travel) for the operation of the NRMRL quality system; Reviewing and approving the NRMRL QMP; Serving as the ultimate decision maker on unresolved quality issues; and Approving the nomination of a primary QA person for specifically identified Programs, and issuing a memorandum to recognize and identify this person’s responsibilities. • • • • • • • The Director of the Technology Transfer and Support Division (TTSD) has been designated by the NRMRL Laboratory Director as the senior management designee for ensuring the NRMRL quality system is implemented NRMRL-wide. 3 March 27, 2002 The Director of TTSD is responsible for: • Informing the Laboratory Director and other Division Directors (DD) about QA program developments; Reviewing and recommending approval of the NRMRL QMP; Approving the schedules for regular assessment of the quality system; Selecting the NRMRL Director of Quality Assurance (DQA); Serving as the NRMRL arbitrator for quality issues that cannot be resolved at lower levels; and Overseeing the activities of the DQA. • • • • • NRMRL Director of Quality Assurance is responsible for: • Reporting all unresolved EPA/Office of Research and Development (ORD) and NRMRL-specific QA issues and information to the TTSD Director; Leading the NRMRL Quality Team [consisting of all NRMRL Quality Assurance Managers (QAMs)] and serving as the facilitator for resolving quality issues between divisions; Leading quality systems audits (QSAs) of the implementation of the NRMRL quality system in each division, when appropriate, with each division being reviewed at least every three years; Coordinating the preparation and updating, as necessary, of the NRMRL QMP; Serving as the NRMRL liaison to the Agency’s Office of Environmental Information’s (OEI) Quality Staff (QS); Assisting QS in the review of policy and concept documents; Coordinating the preparation of the NRMRL QA Annual Report and Work Plan (QAARWP) and submitting it to the NRMRL Laboratory Director and the TTSD Division Director; Initiating discussions with the Laboratory Director or the Director of TTSD to correct quality problems as needed; 4 • • • • • • • March 27, 2002 • Identifying NRMRL quality training needs and assisting in the development and presentation of QA training courses; and Leading continuous quality improvement efforts within NRMRL. • NRMRL Division Directors are responsible for: • Ensuring that the NRMRL QA system is implemented across the Division by including quality responsibilities as described in this QMP in the performance standards of all Branch Chiefs, QA personnel, and Technical Lead Persons in the division; Promoting (by words, actions, and involvement) the NRMRL quality system; Promoting (by words, actions, and involvement) continuous quality improvement within their Divisions; Providing adequate resources (including FTEs, contractor support, and travel funds) to implement the NRMRL QMP across the Division; Reviewing, approving, and subsequently ensuring that the NRMRL QMP is implemented by the division and providing written review comments regarding the QMP to the DQA; Ensuring that the division QAM attends division planning meetings when proposed, new, and/or current research activities (any activity that generates or collects environmental data or uses secondary data for situations related to the environment ) are discussed; Ensuring that the QAM is aware of all research activities in the division; Resolving disagreements within the division (e.g., appropriate corrective action to an audit, QA Project Plan (QAPP), or final product review finding); and Preparing responses to findings of QSAs of the Division and submitting the responses to the DQA to ensure that all unresolved QA issues are adequately addressed. (While this responsibility may be delegated, it is the DD’s responsibility to ensure that this requirement is met.) • • • • • • • • NRMRL Branch Chiefs are responsible for: • Ensuring that the NRMRL QA system is implemented by researchers by including quality responsibilities as described in this QMP in the performance 5 March 27, 2002 standards of all QA personnel and Technical Lead Persons in the branch and by ensuring that the division QAM is aware of all research activities. In other words, the Branch chief ensures that the QA requirements identified in this QMP are implemented; the division QAM needs to be consulted regarding specific procedures used to meet these requirements; • • Promoting (by words, actions, and involvement) the NRMRL quality system; Ensuring that quality planning documents (QAPPs, Standard Operating Procedures (SOPs), etc.) are developed and implemented for all research activities occurring within their branches, i.e., for all intramural and extramural projects. Assigns branch personnel the responsibility for developing project-specific QAPPs and SOPs (QAPPs need to be prepared and approved for research activities as soon as a TLP is prepared to collect data /information to support Agency decisions or for inclusion in research products (e.g., reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data ); Keeping their Division Directors informed of Branch QA issues; Participating in and ensuring branch personnel participate in QA training sessions as appropriate; Ensuring that research products (i.e., published reports [both paper and electronic], journal articles, symposium/conference papers, and extended abstracts) are reviewed by the QAM and approved prior to publication (research products shall be subjected to QA review prior to being distributed outside of NRMRL); Resolving technical or administrative issues relating to quality within the branch including the authority to override the recommendations of the QAM (e.g., appropriate corrective action to an audit, QAPP, or final product review finding); Ensuring implementation of corrective actions; and Preparing responses to any internal branch/laboratory audits and submitting responses to the QAM to ensure that all unresolved QA issues are addressed. (While this responsibility may be delegated, it is the BC’s responsibility to meet this requirement.) • • • • • • NRMRL Technical Lead Persons (TLP) (The TLP is the person who is technically responsible for the project. For extramural contract work, the TLP is typically the 6 March 27, 2002 contracting officer’s representative [COR]; for intramural work, the TLP is typically the principal investigator [PI].) are responsible for: • Ensuring that all extramural agreements (contracts, cooperative agreements, grants, interagency agreements) include appropriate QA requirements and that the requirements are met for their projects; Ensuring development of a planning document consistent with NRMRL policy for all intramural and extramural projects that involve the collection or generation of primary and/or secondary data (Secondary data are environmental data collected from other sources, by or for EPA, that are used for purposes other than those originally intended.); Ensuring that all project participants agree to the project objectives and planned experimental approach before the QAPP is submitted for QA review; Ensuring that all environmental data collection, evaluation, and use do not proceed until there is an approved QAPP; Ensuring that the approved QAPP is implemented and that significant changes (those that may or do affect the quality of data, the scope of the project, or the successful completion of the project) to the approved QAPP are documented and approved before the change is implemented; Ensuring that research products (i.e., published reports [both paper and electronic], journal articles, symposium/conference papers, and extended abstracts) are reviewed by the Division’s QAM and approved prior to publication (research products shall be subjected to QA review prior to being distributed outside of NRMRL); Ensuring that corrective action procedures are initiated in a timely manner and issuing documentation to QA Manager of all corrective actions; Requesting or cooperating with any project-specific audits as required in Section 2.3.5; and Preparing responses to audit or review (QAPP, final report, or journal article) findings. • • • • • • • • NRMRL Division QA Managers are responsible for (while the QAMs are responsible for the following items, they may be delegated with oversight from the division QAM): 7 March 27, 2002 • Reviewing and approving QAPPs, SOPs, published reports [both paper and electronic], journal articles, symposium/conference papers, and extended abstracts; Reviewing and signing QA form for funding packages; Entering quality document tracking information into the QA tracking system; Managing QA support contracts; Attending division research project planning meetings (e.g., lab implementation plan meetings) when proposed, new, and/or current research activities are discussed; Compiling division information for the QAARWP and submitting it to the NRMRL DQA; Negotiating whenever there is disagreement to the proposed resolution to an audit or review (QAPP, final report, journal article, symposium paper, extended abstract, etc.) finding (if agreement cannot be reached, negotiations will be elevated by the QAM to the BC or DD); Reporting issues regarding NRMRL QA system noncompliance to the DQA for further action when resolution cannot be made at the division level; Performing audits as listed in Section 2.3.5 of support laboratories and projects and preparing audit reports for submission to the TLP and BC; Participating in the NRMRL DQA’s QSAs and/or internal audits of the division; Assisting in the development of QA guidance; Assisting EPA TLPs in the development of QAPPs; Serving as a QA consultant for Division projects; Assisting in the preparation of the NRMRL QMP or any revisions to the NRMRL QMP by providing information to NRMRL DQA; Providing QA training courses; Developing and revising quality system forms as needed; • • • • • • • • • • • • • • • 8 March 27, 2002 • • • • • Assisting in the development of internal QA system SOPs; Tracking corrective action as required at project level; Assisting other QAMs in support of the overall NRMRL quality system; Assisting in the review of the NRMRL QA program as needed; Serving as Quality Control (QC) Coordinator for in-house research projects when requested. (This may include assisting the TLP in the development of the QAPP; establishing appropriate project objectives, experimental design, analytical methods, sampling points, etc.; performing audits to ensure implementation of QAPP; performing some level of data validation; and reviewing the final report.); and Leading continuous quality improvement efforts within their Divisions. (Some divisions use a quality committee for assistance [advice] in this area). • NRMRL Program QA Managers NRMRL sometimes achieves mission-specific goals by establishing well-defined programs. To insure consistent implementation of the quality system, a QA Manager may be designated to have overall responsibility for QA activities associated with a program. NRMRL Program QA Managers are responsible for: • • • Assisting in the preparation of a Program-specific QMP, when requested by the Technical Program Manager; assessing implementation of the programmatic QMP; and for reporting noncompliances to the Technical Program Manager and to the NRMRL Director of Quality Assurance or the Director of TTSD when the DQA is the program’s QAM. The definition of whether a work effort is defined as a “Program” is the responsibility of management. Generally, a NRMRL Program is defined by the following guidelines: • • • The work is of sufficient size and scope to be viewed as a high-visibility effort (e.g., congressional mandates, etc.); the work is typically multi-year, multi-project level of effort; the work often crosses Laboratory Center or Division boundaries; and 9 March 27, 2002 • there is a discrete program management structure, distinct from NRMRL Branch/Division line management. 1.5 Activities, Programs and Legislative Acts Supported by the NRMRL Quality System The NRMRL quality system supports a wide variety of technical activities and programs. Internal coordination of QA/QC between programs, locations, and Divisions is performed by the Division QAMs and is supported by the NRMRL DQA. The activities, programs, legislative act, and the primary location of each are presented below: Air Pollution Prevention & Control Division (APPCD) Research Triangle Park, North Carolina Air Pollutant Modeling Alternative Energy Sources Biogenic Air Emissions Combustion-generated Pollutants Indoor Air Pollution Mobile Source Emissions Montreal Protocol North American Research Strategy for Tropospheric Ozone Pollution Prevention ETV Program Clean Air Act Toxic Substances Control Act (TSCA) Land Remediation & Pollution Control Division (LRPCD) Cincinnati, Ohio Engineering Technical Support Center Oil Spills Superfund Technical Assistance Response Team Underground Storage Tanks Sediments Program Superfund Innovative Technology Evaluation Program Clean Water Act Resource Conservation and Recovery Act Superfund Amendments and Reauthorization Act Toxic Substance Control Act 10 March 27, 2002 Subsurface Protection & Remediation Division (SPRD) Ada, Oklahoma Abiotic Remediation Bioremediation Contaminant Fate and Transport Ecosystem Restoration Groundwater Modeling Technical Assistance and Technology Transfer Underground Storage Tanks Wellhead Protection Sustainable Technology Division (STD) Cincinnati, Ohio Pollution Prevention Cost Benefit Analysis Green Chemistry Environmental Technology Verification Program Life Cycle Assessment Sustainable Environment Management Mine Waste Technology Technology Transfer & Support Division (TTSD) Cincinnati, Ohio Technical Information Technology Transfer Water Supply & Water Resources Division (WSWRD) Cincinnati, Ohio Office of Ground Water and Drinking Water programs Safe Drinking Water Act activities Clean Water Act activities Municipal and County activities State water programs 11 March 27, 2002 (Chapter 2.1 revised March 25, 2005) 2.0 Quality System and Description 2.1 Graded Approach to Research Projects Quality Assurance Project Plans (QAPPs) must be prepared to document QA/QC requirements for all research projects that involve the collection, generation, use, or reporting of environmental data (e.g. laboratory and field measurement projects, modeling projects, method development projects, etc.). A QAPP must be prepared before a research project begins. For QA planning purposes, a research project begins as soon as a TLP is prepared to collect data/information to support Agency decisions or for inclusion in research products (e.g., reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data). Before proceeding with the collection of this data/information, a QAPP applicable to the project’s QA category must be prepared. As part of the planning process for a research project, it may be necessary to perform preliminary work to gather information used to define its scope. This preliminary work must comply with the requirements specified in NRMRL’s Minimum Requirements for Laboratory Research Activities (Appendix E). The NRMRL quality system utilizes a "graded approach" for establishing appropriate requirements for QAPPs for various types of research activities. Under the graded approach, the intended use of the data dictates the required level (or category) of quality. For example, the quality controls of a fundamental research program are different from that of a regulatory compliance program because the purpose or intended use of the data is different. Once a graded approach has been applied to the components of a quality system, a graded approach is then applied to the documentation required by that system. The graded approach utilizes four (4) QA categories for research projects. The four QA categories are: Category I establishes QAPP requirements for projects involving enforcement activities, litigation, or research projects involving human subjects; Category II establishes QAPP requirements for projects supporting the development of environmental regulations or standards; Category III establishes QAPP requirements for projects involving applied research or technology evaluations; and Category IV establishes QAPP requirements for projects involving basic research or preliminary data gathering activities. For both Category I and Category II projects, NRMRL follows, in its entirety, the R-5 document for QAPP preparation developed by EPA’s Quality Staff (QS) (http://www.epa.gov/quality/qs-docs/r5-final.pdf). Guidance for Quality Assurance Project Plans, EPA QA/G-5, EPA/240/R-02/009, December 2002 may be used to help address these requirements (http://www.epa.gov/quality/qs-docs/g5-final.pdf ). If a 12 March 27, 2002 (Chapter 2.1 revised March 25, 2005) particular R-5 requirement does not apply to an individual Category I/II project, the TLP provides a brief explanation for why the specific requirement does not apply. For Category III and IV projects, a subset of applicable R-5 requirements is utilized (see Appendix B). The outlines of NRMRL QAPP Requirements for various project types are condensed from typically applicable sections of R-5 and are intended to serve as a starting point when preparing a QAPP. These lists and their format may not fit every research scenario, and QAPPs must conform to applicable sections of R-5 in a way that fully describes the research plan and appropriate QA and QC measures to ensure that the data are of adequate quality and quantity to fit their intended purpose. Some research may not neatly fall into just one of the project types listed below and may combine features of more than one subset of requirements. For such a project, the EPA TLP should consult with the division’s QA personnel to determine the most appropriate QAPP requirements. The requirements used for Category III and Category IV projects are presented in Appendix B. These include requirements for projects involving basic research, applied research, secondary data, sampling and analysis, method development, modeling, and software development/data management. (In some cases, these types of projects may be designated Category I or II; additional requirements in R-5 would then be applicable.) As guidance, the requirements in Appendix B include short descriptions of the applicable types of research. For projects of any QA category level involving design, construction, and/or operation of environmental technology, the requirements in Part C of “Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology,” ANSI/ASQC E4-1994, American Society for Quality Control, Milwaukee, WI, January 1995 and Guidance on Quality Assurance for Environmental Technology Design, Construction, and Operation EPA QA/G-11, January 2005, EPA/240/B-05/001 (http://www.epa.gov/quality/qs-docs/g11-final-05.pdf) will be followed. The required quality level (or category) and associated QAPP requirements are determined by the EPA TLP at the beginning of a project. The Division QAM should be consulted, if needed. 2.2 Quality-Affecting Activities NRMRL’s data generating research and testing activities always require QA oversight. However, NRMRL also performs other functions that may not always be attributed to project-specific efforts; certain aspects of NRMRL’s operations, such as the use of in-house data and software systems, may be inherently part of projects and the quality of these systems is often not addressed in a QAPP. Establishing quality policy for these types of activities and monitoring these activities is typically performed through the 13 March 27, 2002 (Chapter 2.1 revised March 25, 2005) development of SOPs, and in other cases, facility manuals. In addition, there are some generally accepted laboratory practices that need to be implemented to ensure that a minimum level of quality is established in each laboratory. The list of minimum requirements for laboratory research activities is presented in Appendix E. 2.3 Principal Components of the Quality System The following are the principal components of the NRMRL quality system. 2.3.1 NRMRL Quality Management Plan The NRMRL Quality Management Plan documents the NRMRL quality system and encompasses the management and technical activities necessary to plan, implement, and assess the effectiveness of QA and QC operations applied to all research and environmental programs at NRMRL. The NRMRL Laboratory Director is responsible for the development of the NRMRL QMP, although its preparation is normally delegated to the Division QAMs and the NRMRL DQA. The QMP must be approved and signed by the NRMRL Laboratory Director, Division Directors, and the DQA. The QMP is submitted to the OEI QS for review. It must be approved and signed by the QS Director and the Assistant Administrator of OEI (AA/OEI). It is reviewed annually by the NRMRL DQA and QAMs and updated as needed. The QAARWP will be used to identify changes in the QMP prior to the next scheduled revision of the document. The approved QMP is valid for five years unless there is a significant change in the organization or in the research areas emphasized. After five years or a significant change, the QMP is reviewed and revised as needed and subjected to the same approval processes described above. 2.3.2 Quality Assurance Annual Report and Work Plan (QAARWP) NRMRL assesses the effectiveness of its quality system on an annual basis. That assessment, as well as planned work activities for the coming fiscal year related to the quality system, are documented in the NRMRL Quality Assurance Annual Report and Work Plan (QAARWP). The NRMRL QAARWP is submitted to the QS at the conclusion of each fiscal year. Division-specific input for the QAARWP is prepared by the QAM for each Division. The inputs of the Divisions for the QAARWP are provided to the NRMRL DQA. The DQA prepares the NRMRL QAARWP which is submitted to QS under the signature of the NRMRL Laboratory Director. 14 March 27, 2002 (Chapter 2.1 revised March 25, 2005) 2.3.3 Extramural Quality Management Plans EPA contractors, assistance agreement recipients, and some EPA programs may be required to submit a Quality Management Plan. Again, a graded approach is used by the EPA TLP in consultation with the QAM to determine if a QMP is necessary (see Section 4). If a QMP is required, it is reviewed by the QAM and EPA TLP and approved by the EPA TLP. The NRMRL requirements for QMPs are given in Appendix B. 2.3.4 Systematic Planning NRMRL research projects use a systematic planning process that involves defining project objectives and then designing and refining a plan that meets these objectives. The graded approach is used to ensure that the level of detail is in accord with the intended data use and the resources available. 2.3.4.1 Planning Process The NRMRL TLP is responsible for using the systematic process when planning their work effort. Although not mandatory, the Data Quality Objectives Process developed by the EPA is an example of a systematic planning process. This process is described in Guidance for the Data Quality Objectives Process, EPA/QA G-4, EPA/600/R-96/055, August 2000 available at the Internet address, http://www.epa.gov/quality/. 2.3.4.2 Quality Assurance Project Plans As discussed in Sections 2.1 and 2.3.3 of this QMP, NRMRL uses a "graded approach" (i.e., project type) for the development of QAPPs. Determination of the appropriate project type is performed by the NRMRL TLP in consultation with other project participants and the Division QAM when needed. Different project types have different requirements as specified in Appendix B. The QAPP must be reviewed and approved by the Division QAM before a research project begins. NRMRL has developed QAPP requirements for each category of its graded approach (see Appendix B) and utilizes guidance documents developed by EPA’s Quality Staff (QS) to assist technical staff in addressing both NRMRL and Division-specific quality specifications e.g., the R and G series of documents developed by QS (http://www.epa.gov/quality/qa_docs.html). 2.3.4.3 Facility Manuals NRMRL facility managers, lead researchers, and QA staff develop facility manuals to assist technical staff (both EPA and contractor) in documenting the performance of routine operations. A facility is defined as a building (or portion of a building) housing a 15 March 27, 2002 (Chapter 2.1 revised March 25, 2005) set of equipment designed for a particular area of research such as a furnace to study the impact of combustion parameters on the pollutants emitted or a pilot plant to simulate a water distribution system. Facility manuals may contain the following information on a specific NRMRL facility: 1.0 Introduction 2.0 Facility Charter (purpose) 3.0 Management 4.0 Description 5.0 Equipment 6.0 Documentation 7.0 Operation 8.0 Quality Assurance 9.0 Quality Control 10.0 Data Handling 11.0 Corrective Action 12.0 Health and Safety 13.0 QA/Test Plans Appendices may contain: A Current Facility Personnel B Operating Procedures C Standard Methods D Technical Systems Audit Checklist E Performance Evaluation Audit Ranges Facility manuals are reviewed by the facility manager or lead researcher, QAM, and health and safety (H&S) personnel if H&S requirements are included in the manual; they are maintained and updated by facility managers. Formal revisions and reviews are initiated when substantial changes are made to a facility manual. A facility manual documents instrumentation and procedures, provides a historical record of facility use, standardizes procedures, provides a training aid for new employees, and facilitates preparation of QAPPs (which can reference standard sections in the facility manual, with description of the specific tests and schedules for testing). 2.3.4.4 Standard Operating Procedures and Methods NRMRL encourages the development of SOPs for those operations which have become or will become routine, including analytical procedures, sampling procedures, and instrument calibration procedures which are used for more than one project or performed by more than one person. SOPs before they are implemented are reviewed by a second person in the specific technical area and the QAM. Project-specific SOPs may also be written, and are particularly recommended to facilitate updating the QAPP or when the 16 March 27, 2002 (Chapter 2.1 revised March 25, 2005) procedure is performed by contract personnel. The purpose of the SOPs is to facilitate the uniform performance of routine procedures. When appropriate, NRMRL uses approved standard methods (e.g., SW-846, MCAWW, EPA 500 and 600 series, Standard Methods, CFR) to meet project-specific objectives. 2.3.5 Assessments Audits are scheduled for specific systems, programs, and projects based on required periodic review, the importance of or public interest in the program or project, problems in the measurement system (noted by the TLP or QAM), requests by the TLP or managers (branch chiefs, division directors), or selection by the division QAM. The audits planned by the TLP at the beginning of a project or required by the program (e.g., SITE, ETV) are specified in the QAPP or QMP. For all assessments, the lead auditor/assessor (usually either the DQA or a QAM) must ensure that auditor(s)/assessor(s) have no real or perceived conflicts of interest with the project/system/organization being audited/assessed. 2.3.5.1 Quality Systems Audits A quality systems audit (QSA) is an on-site review of the implementation of an organization's quality system as documented in the organization's approved QMP. This review is used to verify the existence of, and evaluate the adequacy of, the internal quality system. A QSA may be a self-assessment or an independent assessment. Independent QSAs of NRMRL are scheduled and conducted by personnel external to NRMRL, such as QS personnel. The NRMRL DQA schedules QSAs of the Division every three years to assess the implementation of the quality system in each Division and may enlist the help of personnel from other Divisions. In any event, the Division QAM is always invited as a member of the evaluation team to witness first hand the implementation of the QMP in their Division observed during the QSA and to respond to questions as needed. 2.3.5.2 Technical Systems Audits A technical systems audit (TSA) is a qualitative on-site evaluation of sampling and/or measurement systems. The objective of the TSA is to assess and document acceptability of all facilities, maintenance, calibration procedures, reporting requirements, sampling and analytical activities, and quality control procedures. Normally, an approved QAPP provides the basis for the TSA. Independent TSAs are most often scheduled by the TLP and conducted by the Division QAM. Assistance for the TSA is available to each QAM from other Divisions or from QA support contractors. TSAs are most useful when 17 March 27, 2002 (Chapter 2.1 revised March 25, 2005) conducted early in the life cycle of a project when corrective actions (if necessary) can be performed that will minimize any loss of data. However, a TSA can be performed any time during a project’s life cycle. The NRMRL DQA may schedule project-specific TSAs to assess the quality system for individual projects in one or more Divisions and may enlist the help of personnel in other Divisions. In any event, the Division QAM is always invited as a member of the evaluation team. TSAs are required for all Category I & II projects (both intramural and extramural). For Category III & IV projects, each division will audit two Category III /IV projects each fiscal year. At least one Category III/IV project must be an intramural project. 2.3.5.3 Audits of Data Quality An audit of data quality (ADQ) is an examination of data after they have been collected and verified by project personnel. Assessing whether the Data Quality Indicator (DQI) goals specified in the QAPP were met requires a detailed review of the recording, transferring, calculating, summarizing, and reporting of the data. An ADQ is conducted as required by specific programs or as requested by the TLP or other EPA stakeholder. ADQs may be conducted by EPA or contractor personnel. 2.3.5.4 Performance Evaluations A performance evaluation (PE) is a quantitative evaluation of a measurement system. Although each measurement in a test program could be subjected to a performance evaluation, the critical measurements (designated in the QAPP) are more commonly evaluated. An evaluation of a measurement system usually involves the measurement or analysis of a reference material of known value or composition. The value or composition of reference materials must be certified or verified prior to use, and the certification or verification must be adequately documented. Ideally, the identity of the reference material is disguised so that the operator or analyst will treat the material no differently than a test program sample. PEs are conducted as required by specific programs or projects or at the request of the TLP or other EPA stakeholder. 2.3.5.5 Laboratory Surveillances Laboratory surveillances are conducted using the Minimum Requirements for Laboratory Research Activities contained in Appendix E. Each division will conduct surveillances of ten laboratory areas per year. If a division has less than ten laboratory areas they will do all laboratory areas. 18 March 27, 2002 (Chapter 2.1 revised March 25, 2005) 2.3.6 Data Quality Assessments A Data Quality Assessment (DQA) is a scientific and statistical evaluation to determine if validated data obtained from environmental data operations are of the right type, quality, and quantity to support their intended use. The DQA process is described in Guidance for Data Quality Assessment, Practical Methods for Data Analysis, EPA QA/G-9, QA97 Update, EPA/600/R-96/084, July 2000, available at http://www.epa.gov/quality/qa_docs.html. For research projects, these assessments are normally conducted by the TLP, and they should be performed routinely on all projects. 2.3.7 QA Review of Research Products Final product reviews are conducted to determine if the final product of a research activity is clear, complete, consistent, correct, coherent, conformant, and concordant. These reviews evaluate the credibility of data, realization of project goals, comparability of data, validity of conclusions, and quality of data. Research products should be subjected to QA review prior to being distributed outside of NRMRL. A signed NRMRL Product QA/QC Verification Report form (Appendix D) must accompany the NRMRL products (reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data) submitted for administrative review and clearance if they contain environmental data. Environmental data are defined in EPA Order 5360.1 A2 as any measurements or information that describe environmental processes, location, or conditions; ecological or health effects and consequences; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as data bases or the literature. 2.3.8 Computer Tracking System Different computer tracking systems for the various elements of the quality system are in place in the various divisions. These systems track funding packages, QAPPs, QMPs, SOPs, audits, and project products. 2.3.9 Training Mandatory quality system training for all NRMRL management and technical personnel is described in Section 3.0. 19 March 27, 2002 3.0 Personnel Qualifications and Training It is the policy of NRMRL that all persons managing or directing NRMRL projects have appropriate training for their assigned work. Training needs may be identified by EPA regulation (e.g., requirements for project officer training), by those persons planning a technical activity (e.g., TLP), or by those managing persons performing a technical activity (e.g., branch chiefs or team leaders). Training programs are generally designed by the persons providing the training. The process for specifying and documenting intramural training varies with the type of training: C Technical Training - Documentation of technical training may be an appropriate degree in an area of study. Other documentation of technical training may include professional certifications (e.g., Professional Engineer or PE) or certification of specialized training (e.g., classes in computer applications or instrument operation as provided by technology vendors). Other specialized training requirements established by the project officer/principal investigator/TLP for a given NRMRL project may be described in the QAPP document or facilities manual. The QAPP is prepared by the project officer/principal investigator/TLP, reviewed by line managers, and reviewed by the Division’s QA Manager (See Section 1.4). General technical training needs for branch, facility, or team members may also be identified by NRMRL managers. Each employee is responsible for submitting his/her training records to the Division’s Human Resources Management representative who keeps a record of each NRMRL employee’s formal technical training. Project Management Training - For extramural project management, NRMRL adheres to the Agency’s requirements regarding initial and ongoing training, including requirements for training as Project Officer, Work Assignment Manager, and/or Contract Administrator. The Human Resources Management Division keeps records of project management training for NRMRL personnel. A certification that the designated EPA extramural project manager or work assignment manager has the appropriate management training is also required as part of each funding package. Quality System Training - Training in NRMRL’s Quality System policies and procedures is planned and performed by NRMRL’s QA staff. NRMRL has a Policy requiring mandatory training in NRMRL’s quality system for all NRMRL management and technical personnel. The QAMs will retain copies of the training certificates. Additional training needs are usually identified by the QA staff through interactions with NRMRL technical and managerial staff. Surveys have also been used to identify quality system training needs. Quality training is generally designed to: (1) inform NRMRL staff of new QA policies and 20 C C March 27, 2002 procedures, (2) refresh and remind NRMRL staff of current QA policies and procedures, or (3) describe services provided by the NRMRL QA staff (metrology, document review, technical assistance). C Safety Training - EPA specifies that personnel involved in laboratory or field activities receive appropriate safety training. The type and amount (hours) of training depend on the specific assignment and the nature of the potential hazards. Training requirements are specified by line managers and compliance with training requirements is monitored by Division/locality Safety Officers and other health and safety office personnel. Records Management Training - Every EPA employee has responsibility for records management. Each employee is responsible for: Creating records necessary to document their activities and actions, Filing records for safe and efficient retrieval, and Disposing of records only in accordance with agency directives and Federal regulations. To assist employees and contractors with their records management responsibilities, each location has a Records Management Officer (RMO) who will assist personnel with the implementation of the agency’s National Records Management Program (NRMP). Further training and assistance are available online at intranet.epa.gov/records/training/index.htm. C 21 March 25, 2005 4.0 Procurement of Items and Services (Extramural Actions) NRMRL and EPA policies require that parties to procurement agreements or assistance agreements shall have in place a quality system consistent with EPA requirements. Procurement agreements include contracts and simplified acquisitions and parties to these agreements include primary contractors, subcontractors and vendors. Assistance agreements include grants, cooperative agreements, and interagency agreements (IAGs). Other extramural agreements covered by this policy include Cooperative Research and Development Agreements (CRADAs). Compliance with this policy is achieved by including appropriate written requirements into solicitation, award and/or agreement documents. 4.1 Scope and Application It is NRMRL policy that any extramural action that falls into one of the following accounting and object classification structures must include a QA Review Form (QARF) , with appropriate signatures, prior to forwarding to the Contracts Management Division (CMD) or the Office of Grants and Debarment (OGD). 25.32 25.05 25.63 25.71 41.00 Programmatic Research and Development Contracts Program Contracts Programmatic Occupational Health Monitoring Programmatic - Research Interagency Agreements. Grants, Subsidies and Contributions For contracts and IAGs (Object Class 25.XX), funding actions include the initial award funding and any modifications that add additional funds. For assistance agreements (Object Class 41.XX), funding actions include the initial award funding, incremental funding (partial funding, no new work) and supplemental funding (additional funding of new work). For each funding action, NRMRL TLPs must incorporate a QARF into the funding package (see Appendix C). For CRADAs and IAGs, NRMRL TLPs must incorporate a QARF into the agreement documentation. The TLP is responsible for completing the QARF, but they have the option to consult with their QAM prior to submitting it with the funding package for the QAM’s concurrence signature. The QARF identifies the required quality documentation that shall be included in the contract clauses by the Contracting Officer (CO). There must be adequate information in the statement of work (SOW) for the offeror to develop this documentation. Attachment #1 to the SOW (see Appendix C) is used to specify the guidance to be followed and must be referenced in the SOW when applicable. Attachment #1 to the SOW is not necessary when the contractor 22 March 25, 2005 or cooperator will not be required to develop or revise any quality documentation (e.g. when they will only be implementing previously approved quality documentation). All funding actions and agreement documents are reviewed by NRMRL QAMs or their designees to ensure the appropriate requirements are included in each action. 4.2 Single-Project Effort For a single-project effort, the TLP typically requires that an offeror or applicant address NRMRL’s Quality System Specifications for Extramural Actions in their proposal (before award). The five specifications are: (1) a description of the organization’s Quality System (QS) and information regarding how this QS is documented, communicated and implemented; (2) an organizational chart showing the position of the QA function; (3) delineation of the authority and responsibilities of the QA function; (4) the background and experience of the QA personnel who will be assigned to the project; and (5) the organization’s general approach for accomplishing the QA specifications in the SOW. After-award, the TLP typically requires the awardee to prepare a quality assurance project plan (QAPP) that meets the project-specific requirements specified on the QARF and Attachment 1 to the SOW. The before-award documentation, in concert with the after-award QAPP, constitutes a joint QMP/QAPP. 4.3 Multiple-Project Effort As part of the solicitation of a large contract with multiple work assignments and large assistance agreements with more than one level of development, the TLP typically requires that an offeror or applicant submit a Quality Management Plan (QMP) (before award). The QMP needs to meet the requirements of EPA (which includes compliance with the American National Standard ANSI/ASQC E4-1994, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs - see EPA Order 5360.1) and is submitted by the offeror/applicant as part of the proposal. NRMRL has developed a set of requirements (see Section 2.0 and Appendix B) based on EPA Requirements for 23 March 25, 2005 Quality Management Plans, EPA QA/R-2, March, 2001. NRMRL’s requirements are to be addressed in QMPs submitted as part of a proposal responding to a NRMRL solicitation. Each QMP is evaluated by the Technical Evaluation Panel (TEP), if convened, to determine its acceptability (See Section 4.5). The specific evaluation procedures are determined and agreed to by the QAM, the TLP, and the CO prior to issuing the solicitation. After-award, the awardee must revise the QMP according to comments received from the TEP. For each subsequent funding action where environmental data are used, generated, or collected, the TLP typically requires the awardee to prepare a quality assurance project plan (QAPP) that meets the project-specific requirements specified on the .QARF and Attachment 1 to the SOW. 4.4 Additional Requirements for Competitive Extramural Actions Exceeding $500,000 In addition to the requirements in Section 4.1, the following requirements exist for competitive contract and assistance agreement solicitations exceeding $500,000. In accordance with Section 46.1.5.2 of the EPA Contracts Management Manual (CMM), if the potential value of the procurement exceeds $500,000, or the estimate of the percentage of costs or level-of-effort allocated to activities requiring quality requirements exceeds 15%, then the quality documentation (i.e., the Quality Management Plan or equivalent documentation) shall be included as part of the Technical Evaluation Criteria if a Technical Evaluation Panel is convened by the CO. The QAM or authorized QA designee shall : (1) assist the Project Officer with development of the Technical Evaluation Criteria, and any associated technical instructions, for the Request for Proposal, and (2) serve as a member of the Technical Evaluation Panel for the purpose of evaluating the QA aspects of the technical proposals when a Technical Evaluation Panel is convened. For assistance agreements, the QA Manager or designee shall perform a QA review of proposals being considered for award. This is in addition to the extramural and intramural reviews required by the Code of Federal Regulations. The TLP shall consult with the QAM to establish QA evaluation standards for these solicitations. Typically, QA is evaluated as part of the technical evaluation criteria (approximately 10%-20% of the total points). QARF and Attachment 1 to the SOW 24 March 25, 2005 4.5 QA Responsibilities on IAGs and CRADAs If work on a project will be performed by EPA and one or more other organizations, the responsibility for QA must be negotiated within the agreement. The TLPs in consultation with the QAMs in the various organizations must agree on and document (in the agreement and on a QARF) which organization will take the lead for QA, the specific person from the organization who will be responsible, and the QA requirements that will be adhered to during the agreement. 4.6 Evaluation of Quality Deliverables In order to ensure that extramural organizations provide quality services that satisfy EPA QA/QC requirements, NRMRL QAMs (or their designees) and TLPs review QMPs, QAPPs, SOPs, and final products and audit projects and programs. Review comments generated by the QAM (or designee) are provided to the TLP who is ultimately responsible for the quality of services provided. 4.7 Evaluation of NRMRL’s Compliance with QMP The NRMRL DQA schedules and implements reviews of QA issues pertaining to the procurement of items and services in each Division as part of the routine internal QSAs. 25 March 27, 2002 5.0 Documentation and Records QMPs, QAPPs, Facility Manuals, and SOPs are prepared, reviewed, approved, issued, used, and revised. As described in Sections 2, 7, and 8, all these documents, including revisions, must be reviewed for conformance with the quality system requirements and approved by authorized personnel before general use. The following is a discussion of NRMRL procedures for documents and records, including computer-resident records. Records shall be maintained by the PI or PO so that all NRMRL research activities can be reconstructed. 5.1 EPA Records Management Policy EPA records management policy is established by Office of Information and Resources Management and documented in the EPA Records Management Manual 2160. This procedure applies to all records, as defined under the Federal Records Act (44 U.S.C. 3101), regardless of media (including paper, microform, electronic, audiovisual, and record copies of Agency publications). All locations of NRMRL comply with EPA records management policy. The management process that ensures that records accurately reflect completed work and/or fulfill statutory and contractual requirements, including any specific record keeping requirements is defined in EPA Order 2160 and EPA Directive 2100, Chapter 10. This document defines requirements and responsibilities for record transmittal, distribution, retention, protection, preservation, traceability, disposition, and retrievability and includes the roles and responsibilities for management and staff. Following this EPA Order ensures compliance with all statutory, contractual, and assistance agreement requirements for records from environmental programs and provides adequate preservation of key records necessary to support the mission of the organization. NRMRL has a designated Records Management Officer (Sandra Croley) and each Division has a person designated as the Records Liaison. 5.2 Project Documentation and Records Documentation developed on a project-specific basis is the responsibility of the EPA Technical Lead Person for that project and is subject to EPA records management policy. The level of documentation required is dependent upon the type of project. Specific requirements are identified in the project QAPP. When a project is completed as determined by the TLP, the project file is archived. This is accomplished by the TLP in conjunction with the Division’s Records Liaison. 26 March 27, 2002 5.3 Research Laboratory-Specific Documentation and Records Documentation and records that may not be directly applicable to individual projects, (e.g., refrigerator temperature logbooks, instrument logbooks, standard preparation logbooks, SOPs, balance calibration logbooks) are maintained in the location where they are used (See Appendix E for specific requirements). All staff using the equipment or preparing standards are responsible for recording the appropriate information. When the logbooks are full or no longer needed, the logbooks or outdated SOPs are turned over to Records Liaison for appropriate retention, storage, and disposition. Personal laboratory notebooks are the responsibility of the individual. When full, they may be retained by the individual; or when no longer needed or wanted by the individual, they shall be turned over to the Records Liaison. 5.4 Software Documentation and Records NRMRL is required to maintain software quality documentation and records consistent with EPA Office of Information Resources Management (OIRM) policy. 5.5 NRMRL Quality Documentation and Records NRMRL policy is that quality system documents (e.g., the NRMRL QMP and other laboratory/division specific QA documents) developed directly by the NRMRL DQA, Division QAMs, or at the direction of the QAMs shall be controlled by NRMRL or Division QA Offices. Although the Division QAM may keep copies, records of reviews of QMPs, QAPPs, Facility Manuals, SOPs, final reports, and journal articles, internal audit reports, etc., are the responsibility of the TLP. 27 March 27, 2002 6.0 Computer Hardware and Software NRMRL follows guidance published by the Office of Information Resources Management (OIRM) to ensure that computer hardware and software used in environmental programs meets technical requirements and quality expectations. Specific guidance includes: EPA Directive 2100, Information Resources Management Policy Manual (http://www.epa.gov/irmpoli8/polman/), and EPA Directive 2182, System Design and Development Guidance (http://www.epa.gov/irmpoli8/sysdesn/). These comprehensive guidance documents address many issues regarding the use of computer systems, including purchase of computers, purchase or development of software, design of databases, records management, security, and data standards. OIRM has specified Agency-wide standards for software, and OIRM approval is required for nonstandard software. Project-specific requirements for hardware configurations and for configuration control are specified by the TLP in specific project QAPPs. Project-specific requirements and plans for software testing, validation, and documentation, whether the software is developed or purchased from an outside source, are also described by the TLP in the QAPPs for those projects. To assist in the implementation of software and hardware standards, each NRMRL location employs an ADP coordinator and a PC site coordinator (PCSC), who advise and assist NRMRL researchers in planning and implementing projects involving software development, database design and development, and other computer-based projects. Funding packages and plans for all NRMRL projects involving software development or purchase or computer purchase must undergo review and be approved by the locational ADP coordinator or PCSC, respectively, to ensure that this guidance is followed and quality is ensured. The NRMRL QA staff has also developed an abbreviated guidance on software development which is available from the QAMs, and QAPP requirements for these projects are presented in Appendix B. 28 March 27, 2002 7.0 Planning It is the policy of NRMRL that systematic planning shall occur for all NRMRL research products. Depending on the scope of research, as well as on any contractual or Agency requirements, planning documents (including QAPPs) appropriate to the scope are developed. The NRMRL DQA and Division QAMs participate in NRMRL and Divisional planning meetings as directed by senior management. NRMRL has established minimum requirements for various categories of QAPPs (see Appendix B). For projects that require original environmental data collection and/or sampling and analysis or use of secondary data, a graded approach (See Section 2) is in place, and is used for both intramural and extramural activities. The overall intent is to allow flexibility, while still meeting NRMRL’s interpretation of Agency policy for QA. Effective quality planning requires clear identification of project goals and intended use of data. The EPA TLP for the project in consultation with project participants, and division QA personnel as needed, determines the appropriate QAPP requirements to be used. It is NRMRL TLP’s responsibility to identify and involve any and all appropriate sponsoring organizations, responsible official(s), project personnel, stakeholders, scientific experts, etc. (e.g., all customers and suppliers) in the planning of the project. This is true for both intramural and extramural projects. Once the planning is complete, project documentation should include (at a minimum) a complete description of the following: C C C C C C The project’s goals, objectives, questions, and issues to be addressed; The project’s schedule, resources (including budget), milestones, and any applicable requirements (e.g., regulatory requirements, contractual requirements); The type of data needed and how the data will be used to support the project’s objectives; The quantity of data needed and the specification of performance criteria for measuring data quality; How, when, and where the data will be obtained (including existing data) and identification of any constraints on data collection; Specifications of needed QA/QC activities to assess the quality performance criteria (e.g., QC samples for both the field and laboratory, audits, technical assessments, performance evaluations, etc.); 29 March 27, 2002 C How the acquired data will be analyzed (either in the field or the laboratory), evaluated (i.e., QA review, validation, verification), and assessed against the quality performance criteria and for its intended use. Once all project planning is complete and documented, it is the NRMRL TLP’s responsibility to submit the QAPP to the designated QA Manager within each Division. The QA manager performs a review against QAPP requirements and issues documentation to indicate whether planning requirements have been met and an assessment of whether project goals can be met. The documentation from the QA manager shall include either an approval or a non approval. In the case of non approval, detailed review comments shall be provided to the TLP. Resolution of all findings shall be accomplished and documented before any research is started. 30 March 27, 2002 8.0 Implementation of Work Processes Once proper planning has been performed (as described in Section 7), it is important that the implementation of the required procedures be done. It is the responsibility of the EPA TLP to ensure that the required procedures are properly implemented. 8.1 QAPP Development, Review, and Approval The EPA TLP develops a QAPP based on NRMRL guidance (see Section 2). The QAPP must be reviewed and approved by the Division QA Manager before the project begins. A research project begins as soon as TLP is prepared to collect data/information to support Agency decisions or for inclusion in research products (e.g., reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data). QA comments are documented in a memo to the TLP, with a copy forwarded to the TLP’s first line supervisor. The format of this memo may be Division-specific. 8.2 SOP Development In some instances, procedures for sampling, analysis, or other quality activity (e.g., standards preparation) are routine and are used for more than one project. Oftentimes, it is necessary that more than one person perform a procedure. In these cases, an SOP should be written. SOPs are also recommended for special or critical operations. SOPs can make the development of QAPPs more efficient, since they can be attached or referenced. The need for the development of an SOP is determined by the EPA TLP or QAM. SOPs must be reviewed and approved by the Division QA Manager and approved by an EPA person other than the author. It is recommended that SOPs contain the following sections as appropriate: 1.0 2.0 3.0 4.0 5.0 6.0 Scope and Application Method Summary Sample Preservation, Containers, Handling, and Storage Reagents Equipment/Apparatus Procedure 31 March 27, 2002 7.0 8.0 9.0 Calculations Quality Assurance/Quality Control References 8.3 QAPP/SOP Implementation It is the responsibility of the Branch Chief to ensure that the general procedures utilized throughout the Branch are implemented during data collection activities. It is the responsibility of the EPA TLP to ensure that the required procedures specified during the planning process for a specific project are implemented during data collection activities. This is done by personally observing the procedures being performed or by requesting that the Division QA Manager perform an audit of the project activities outlined in a QAPP and/or SOP. In either case, deviations from project requirements must be documented, along with the corrective action performed. Corrective action must be performed as soon as possible to minimize any effect on data quality. If data quality is affected by any deviations, this must be discussed in any project report/paper. 8.4 QAPP/SOP Revisions During the course of a project, it may be necessary to revise QAPPs and/or SOPs. Revisions are required whenever a significant change in the plan or procedure occurs. Over time, it may also be necessary to revise QAPPs and/or SOPs to ensure they are still applicable for the work being performed. QAPPs and project-related SOPs need to be reviewed on a yearly basis for long term projects. Branch/Division SOPs need to be reviewed every 2 years. Revisions (as necessary) in the form of an addendum or fully revised document must be made as soon as a significant change is identified. Significant revisions to a QAPP/SOP must be reviewed and approved by the Division QA Manager. It must then be verified that the revised procedures are being implemented properly, as discussed in Section 8.3. Copies of the most current document must be made available to the appropriate project/branch/division personnel. All previous document versions must be archived. Responsibilities for revisions to, distribution of, and archival of QAPPs/SOPs are as follows: 32 March 27, 2002 Document QAPP Project-related SOP Branch SOP Division SOP Revision Responsibility TLP TLP Branch Chief or designee Division Director or designee Distribution of Revised QAPP/SOP TLP TLP Branch Chief or designee Division Director or designee Archival of previous versions TLP TLP Branch Chief or designee Division Director or designee 33 March 27, 2002 9.0 Assessment and Response NRMRL conducts a variety of assessments at the NRMRL-wide, Division, and projectspecific level to provide an increased understanding of the program or system being examined, and to provide a basis for improving such programs or systems. The following includes: a review of the various assessment techniques, plans for appropriate response, and the process by which management determines the assessment activities appropriate for each level. 9.1 Assessment Techniques and Practices Basic assessment categories were described in Section 2.3.5 of this QMP. In NRMRL, peer review is a separate function from QA. Program peer reviews are not performed by QA staff unless directed by the NRMRL Director or a Division Director. The NRMRL and Division QA offices respond to issues regarding data quality identified during the peer review process and provided to the QA staff. 9.2 Types of Assessments A description of the four types of assessments are provided in this section. 9.2.1 Self-Assessment of a Quality System A quality system self-assessment is a qualitative assessment of a quality system’s operations by those immediately responsible for overseeing and/or performing the work to establish whether the prevailing quality structure, policies, practices, and procedures are adequate for ensuring that the type and quality of results needed are obtained. 9.2.2 Independent Assessment of a Quality System An independent quality system assessment is a qualitative assessment of a quality system’s operations by someone other than the group performing the work to establish whether the prevailing quality structure, policy, practices, and procedures are adequate for ensuring that the type and quality of results needed are obtained. 34 March 27, 2002 9.2.3 Technical Self-Assessment A technical self-assessment is the evaluation process used by those immediately responsible for overseeing and/or performing the work to measure the performance or effectiveness of a technical system and its elements with respect to documented specifications and objectives. Such assessments may include qualitative and quantitative evaluations. 9.2.4 Independent Technical Assessment An independent technical assessment is the evaluation process used by someone other than the group performing the work to measure the performance or effectiveness of a technical system and its elements with respect to documented specifications and objectives. Such assessments may include qualitative and quantitative evaluations. 9.3 Assessor Capabilities and Authority Assessors should be thoroughly familiar with QA practices, policies, and procedures. The optimal assessment team consists of a QA professional, a person with expertise in the technical area being evaluated, and, if there are complex issues regarding experimental design or data analysis, a statistician. In many cases, a QA staff member can fill more than one role. Assessors evaluate NRMRL projects under the authority of and with the permission of the NRMRL TLP and NRMRL management. Assessors do not have the direct authority to change project procedures or to alter project goals. Their role is to provide information to the NRMRL TLP as to the best practices and to clearly point our errors observed in the implementation of the QAPP. These errors may be in technical procedures, in data archival or analysis, in project management or reporting, or in any aspect of a project that impacts the data quality. 9.4 NRMRL-Wide Assessment and Response Procedures The NRMRL DQA is responsible for planning, scheduling and conducting the following assessments. The NRMRL DQA may be assisted in the performance of assessments by Division QA Office staff. At the conclusion of each assessment, the assessment report shall include a description of the type of corrective actions required to resolve any findings of nonconformance to EPA and NRMRL policy as described in EPA and NRMRL quality documents and Orders. 35 March 27, 2002 ASSESSMENT TYPE Self-Assessment of NRMRL Quality System C C NRMRL QMP review Review of NRMRL Quality System yearly, as part of QAARWP yearly, as part of QAARWP Independent Assessment of the Application of the NRMRL Quality System in a Division C QSA of Division every 3 years Independent Technical Assessment • Technical System Audits as needed or requested by senior management as directed by senior management • QSA of Programs NOTE: The conduct of technical systems audits (TSA) will probably be performed very infrequently by the Director of Quality Assurance (DQA). TSAs conducted by the DQA intend to focus mainly on larger projects involving more than one NRMRL location. 9.5 Division Assessment and Response Procedures The Division QAM is responsible for planning, scheduling, and conducting the following assessments. The Division QAM may be assisted in the performance of assessments by QA Office staff members from any Division. At the conclusion of each assessment, the assessment shall include a description of the type of corrective actions required to resolve any findings of nonconformance to EPA and NRMRL policy as described in EPA and NRMRL quality documents and Orders. ASSESSMENT TYPE Self-Assessment of the Application of the NRMRL Quality System to a Division C Review of Division Quality System yearly, as part of QAARWP 36 March 27, 2002 Independent Technical Assessment C C C Review of Division SOPs Review of Division QAPPs all SOPs all QAPPs Final Research Products all published reports (both paper and electronic), journal articles, symposium/conference papers, and extended abstracts TSAs are conducted as described in Section 2.3.5.2. PEs and ADQs are conducted as scheduled in planning documents or as requested by management Surveillances are conducted as described in Section 2.3.5.5 C TSAs, PEs, ADQs C Surveillances 9.6 “Stop Work” Order QA staff members are often on-site and able to interact directly with project personnel. If a QA staff member observes work practices which could have serious adverse impacts on data quality, the QA staff member should promptly notify the NRMRL TLP or line manager for in-house projects. For extramural projects, if the TLP concurs that work practices are unacceptable, the TLP should notify the Contracting Officer, who may issue a Stop Work Order. 9.7 Assessment Reports and Corrective Action For TSAs, PEs, and ADQs, on-site assessments are followed up by a debriefing to the auditee and TLP (if available) at the end of the on-site portion of the evaluation. If the TLP is not present at the debriefing, then assessments are also followed up by a verbal or brief written summary to the NRMRL TLP on the first working day after the completion of the on-site portion of the audit. The general format for assessment reports is: C C C Cover page (assessment identification) Summary (brief discussion of findings, those issues that require corrective action) Assessment procedures 37 March 27, 2002 C C C Assessment results (detailed accounting of findings and other observations) Discussion (impact of findings) Appendices (checklists, protocols, other data) Draft assessment reports are generated by the assessor within two weeks of the receipt of requested audit materials. Reports are sent as drafts to the NRMRL TLP and to the auditee, for correction of factual errors and addition of any pertinent comments or responses to audit findings. Two weeks are generally allowed for this response and review. For extramural projects, after this review and correction, as necessary, the draft reports are distributed to the NRMRL TLP, the TLP’s immediate supervisor (usually a branch chief), and to the auditee. For intramural projects, the draft reports are distributed to the TLP and the TLP’s immediate supervisor. It is the responsibility of the NRMRL TLP to resolve audit findings. If findings cannot be resolved by the NRMRL TLP, the TLP’s immediate supervisor’ assistance is requested. The file on a given audit is maintained in active status until all findings are resolved and documented. Audit reports are then archived in the TLP’s project files. For surveillances, a checklist based on the requirements in Appendix E is typically used to document the results; this checklist also serves as the report to the branch chief in charge of the laboratory area. A copy of the checklist is retained by the QAM, and another copy should be given to the TLP of the area. 38 March 27, 2002 10.0 Quality Improvement All of the policies and procedures described in this manual were developed to document QA planning. Two levels of quality improvement processes combine to evaluate the effectiveness of the NRMRL quality system and ensure continual quality improvement. At the QA Program level, QSAs discussed in Section 9.4 are performed by the NRMRL Director of QA. Any problems or areas where quality improvement can be made are identified and reported to the NRMRL Laboratory Director, who is responsible for implementing the necessary corrective action. At the Project QA level, the document reviews and audits discussed in Section 9.5 are conducted to evaluate proposed procedures and their implementation. Any problems or areas where quality improvement can be made are identified and reported to the EPA Technical Lead Person, who is responsible for implementing the necessary corrective action. In addition, all project personnel are responsible for quality improvement activities with respect to their particular roles. Finally, it is the responsibility of the designated QA Manager in each NRMRL Division to monitor the quality procedures which are implemented on a day-to-day basis. This is done via audits, discussions with researchers, and involvement in the implementation of the quality system. If possible, the cause of any identified problem will be determined before corrective action is determined. The corrective action shall be planned, documented, agreed upon, and implemented to minimize the effect on program/project quality. 39 March 27, 2002 11.0 References 1. American National Standards Institute (ANSI)/American Society for Quality (ASQ) E41994, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs, 1994, ASQ, Milwaukee, WI. EPA Quality Manual for Environmental Programs, 5360 A1, May 5, 2000, USEPA Quality Staff, Office of Environmental Information (OEI), Washington, D.C. EPA Requirements for Quality Management Plans EPA QA/R-2, EPA /240/B-01/002, March 2001,OEI, Washington, D.C. EPA Requirements for Quality Assurance Project Plans QA/R-5, EPA /240/B-01/003, March 2001,OEI, Washington, D.C. System Life Cycle Management Guidance, USEPA Office of Solid Waste and Emergency Response, 1992, Washington, D.C. EPA Order 5360.1 A2, Policy and Program Requirements for the Mandatory Agencywide Quality System, USEPA, May 5, 2000, Washington, D.C. EPA Information Security Manual for Personal Computers, USEPA OIRM, December 1989, Washington, D.C. EPA Records Management Manual 2160, USEPA Management and Organization Division, 1984, Washington, D.C. Guidance for the Data Quality Objectives Process, EPA QA/G-4, EPA /600/R-96/055, August 2000, ORD, Washington, D.C. Guidance for the Data Assessment, Practical Methods for Data Analysis, EPA QA/G-9, QA00 Update, EPA /600/R-96/084, July 2000,OEI, Washington, D.C. 2. 3. 4. 5. 6. 7. 8. 9. 10. 40