HIV Drug Resistance Training - Download Now PowerPoint by 9m8BFuO

VIEWS: 15 PAGES: 68

									HIV Drug Resistance Training




         Module 11:
         Quality Control and
         Quality Assurance

                               1
                     Topics

 Quality Assurance vs. Quality Control
 Quality Assurance Before, During, and After
  Testing
 Quality Control
 EQA




                                                2
                   Objectives

 Identify requirements for QC/QA.
 Identify types of quality controls that must be in
  place and the rationale for each.
 Given individual lab needs, develop a plan for
  setting up an internal QC program.
 Identify types of quality assurance programs and
  their purpose.
 Given individual lab needs, develop a plan for
  setting up an external QA program.


                                                       3
quality assurance vs. quality
control
What is the difference between QA and QC?




                                            4
                       Terms

 Quality Control
 Quality Assurance
 Quality Assessment




                               5
                      Definitions

   Term                          Definition
Quality      The measures that must be included during each
Control      assay to verify that the test is working properly.

Quality      The overall program that ensures that the final
Assurance    results reported by the laboratory are accurate

Quality      A means to determine the quality of the results
Assessment   generated by the laboratory. It is usually an
             external evaluation of the laboratory's performance.
             Quality assessment is a challenge to the
             effectiveness of the QA and QC programs.


                                                                    6
 Quality Assurance vs. Quality Control

                  Quality Assurance           Quality Control
           Activities to ensure process   Activities to evaluate a
           are adequate for a system      product or work result
Definition to achieve its objectives

               Establish standard         Analyze known QC
                procedures for sample       sample to determine if a
                collection                  test is valid
Examples       Define criteria for        Decide if a sample is
                acceptable samples          acceptable for testing




                                                                       7
Quality Control is Part of Quality Assurance


                                                      Quality
                                                     Assurance




     "The aim of quality control is simply to ensure that the results generated
     by the test are correct. However, quality assurance is concerned with
     much more: that the right test is carried out on the right specimen, and
     that the right result and right interpretation is delivered to the right
     person at the right time.”                                                 8
                    QA or QC?

   HIV-negative plasma control
   SOP writing
   Personnel records
   Phylogenetic analysis
   DBS condition inspection
   Instrument calibration
   Proficiency testing
   New operator training
   Assay validation
   Lot release testing
                                  9
The Quality Assurance Cycle




         • Data and Lab
           Management
         • Safety
         • Customer Service




                              10
                   Discussion

   What is the difference between QA and QC?




                                                11
quality assurance before, during,
and after testing
What are some examples of errors that occur before, during, and
after testing, and what can we do about them?




                                                                  12
The Quality Assurance Cycle




         • Data and Lab
           Management
         • Safety
         • Customer Service




                              13
Pretesting




• Data and Lab
  Management
• Safety
• Customer Service




                     14
      Preventing and Detecting Errors
              Before Testing

   Lab personnel training and certification
   Sample criteria set and used
   Specimen labeling, collection and transport
   Specimen storage and processing
   Check storage and room temperature
   Lab set up
   Reagent inventory and expiration dates
   SOPs for all related procedures
   Preventive maintenance and equipment checks
   External Quality Assessment (proficiency testing)
                                                        15
During Testing




  • Data and Lab
    Management
  • Safety
  • Customer Service




                       16
      Preventing and Detecting Errors
              During Testing

   Conducting test according to SOPs
   Quality Control (QC) (in-process controls)
   Safety precautions
   Prepare reagents
   Correctly interpreting test results
   Time management and planning




                                                 17
Post-testing




 • Data and Lab
   Management
 • Safety
 • Customer Service




                      18
      Preventing and Detecting Errors
               After Testing

   Re-check patient/client identifier
   Write legibly
   Clean up and dispose of contaminated waste
   Complete quality control review
    – Positive and negative control results
    – Contamination check (phylogeny)
    – Other QA checking (e.g. SQUAT)
   Transcribing, reporting and interpreting results
   Critical review of results and interpretation
   Record Maintenance

                                                       19
SQUAT: Sequence Quality Assessment Tool

  QA tool developed for WHO by Dr. Rami Kantor
   and colleagues, Brown University
  Open-source and freely available
  To be used by genotyping labs before sending
   sequences to national HIVDR working groups for
   surveillance and monitoring activities
     – Labs should perform initial investigations and take
       corrective actions if required
    Sequences can be assessed by WHO for external
     quality assurance as part of ongoing accreditation
     process
                                                             20
                SQUAT Functions

 Sequence alignment
 HIV PR and/or RT sequence identification
 Mutation identification and generation of export
  file for uploading to database
 Flagging of problematic sequences
    –   ambiguous bases (mixtures)
    –   stop codons
    –   insertions/deletions
    –   frameshifts
    –   genetic distance anomalies
    –   atypical and ambiguous amino acids

                                                 21
Quality Assurance is Everywhere!




           • Data and Lab
             Management
           • Safety
           • Customer Service




                                   22
          Find It, Then Fix It




 Use a problem-solving team.
 Investigate root causes.
 Develop and implement appropriate corrective
  actions.
 Examine effectiveness.
 Record all actions and findings.             23
                  Discussion

 Why do errors occur?
 What are some common errors that might occur
  with HIVDR Genotyping?
 Where is QA applied in a HIVDR testing site?
 What are some steps to take before, during, and
  after testing to assure the quality of results?
 Describe the impact that errors will have on
  public health.



                                                    24
Quality Control

What do we need to do to set up a quality control program for our
genotyping?




                                                                    25
       What is quality control (QC)?

 Monitoring the test quality itself
 Ensuring that the assay is working appropriately
  and accurate test results are reported with
  confidence.
 Prevention
 Detection
 Correction




                                                     26
             QC: Two Levels/Two Types

   Levels
    – Testing of samples with known results to verify the
      procedure is working properly
    – Interpreting the presence or absence of controls.
   Types
    – Internal quality control
    – External quality control




                                                            27
                   Internal QC

   Internal QC samples with known reactivity are
    included with the following HIVDR test kits.
    – TruGene
    – ViroSeq
 For in-house assays, prepare and qualify your
  own internal QC samples
 Test the IQC samples as you would test patient
  specimens.
 Include positive and negative controls in every
  run.
 Know what to do in case of unexpected results
                                                    28
Genotyping Assay Procedures

               Patient virus

              RNA Extraction             Positive and
                                       negative controls
           cDNA synthesis (RT)

         cDNA amplification (PCR)

           Sequencing reactions        Positive control

              Electrophoresis

        Data Analysis (base calling)

Report (sequence, mutations, interpretation)          29
Guidelines for HIVDR Genotyping Internal QC
                  Policies

   Negative controls
    – HIV-negative plasma, at extraction step
    – Water, at RT step and at PCR step
        • If any negative control shows a detectable PCR product, all
          samples must be discarded
    – Investigate to locate source of contamination
   Positive controls
    – HIV-positive plasma, at extraction step
    – HIV DNA at PCR step (low copy number)
       • If positive control does not produce a PCR product, investigate
         to determine cause; handling of products from test samples
         may depend on outcome
    – HIV DNA at sequencing step
   SOPs required
                                                                        30
                     External QC

   External QC samples do not come with the test
    kit and are provided by an external source
    – Regional reference laboratory
    – Commercial supplier
    – Another laboratory that has prepared validated quality
      control in-house samples
   Test the EQC samples as you would test patient
    specimens.




                                                               31
    Internal or External Control Failure

   If QCs fail, repeat the run.
   If problems or errors occur, then corrective
    actions should be taken before results
    distribution.




                                                   32
    Planning Your Internal QC Program

   Incoming samples
   Incoming reagents and kits
   Methodology or instrument comparison




                                           33
               Incoming Samples

   Upon arrival, all incoming sample shipments
    should be evaluated for proper collection,
    transport and storage




                                                  34
         Incoming Reagents and Kits

 Evaluate all incoming reagents and kits upon
  arrival.
 Perform lot-to-lot comparisons.
    – Assay previously tested patient samples or controls on
      both the previous lot and the new lot and compare the
      results.
    – When a new lot has been evaluated and ‘passed,’ label
      container “Ready to Use.”




                                                           35
Lot-to-lot Comparison for Critical Reagents

    Lot-to-lot comparison for critical reagents:
     – For commercial kits:
        • Perform parallel run with new vs. existing kits
        • Define acceptance criteria for reagent lot to lot
          comparison
     – For home-brew assays:
        • Perform parallel run with new vs. existing lots
        • New lot of primers, RT-PCR enzymes, sequencing
          reagents
    Source of specimens
     – Positive controls
     – Previously tested samples
     – Old proficiency panels
                                                              36
    Critical Reagent Quality Control


   Extraction buffers        RNA Purification

  Enzymes (RT, DNA
    polymerase)
                                 RT-PCR
        Primers

   Reaction buffers        Sequencing reactions

Capillary matrix polymer     Electrophoresis



                                 Results

                                                  37
Methodology or Instrument Comparison

 For tests that have multiple instruments available
  in the laboratory for use, compare instruments.
 Results should be comparable.
 Compare and document results routinely
  (depending on lab throughput).




                         =                         ?
                                                       38
            Small Group Discussion

1.   What do you need to do to set up a QC program
     for genotyping? What components should be
     included?
2.   If a RT-PCR negative control becomes positive,
     what should you do?
3.   How can you perform lot-to-lot
     comparisons in your lab ?
4.   What is critical to include in a QC SOP?
5.   Once you are done with a lot-to-lot comparison
     and obtained satisfactory results, what should
     you do for the next step?
                                                  39
EQA

What is EQA and why is it important?
Which EQA Methods should we employ?
• Proficiency Testing
• On-Site Evaluation
• Re-testing




                                       40
A Systems Approach to Laboratory Quality


       Organization    Personnel   Equipment



          Stock         Quality       Data
       Management       Control    Management


           SOP,
                      Occurrence
       Documents &                 Assessment
                      Management
         Records

                                    Safety &
         Process       Specimen
                                     Waste
       Improvement    Management
                                   Management   41
    External Quality Assessment (EQA):
          Definition and Purpose

   Objective assessment of a test site’s operations
    and performance by an external agency or
    personnel
   Allows comparison of performance and results
    among different test labs
   Provides early warning for systematic problems
    associated with kits or operations
   Provides objective evidence of testing quality
   Indicates areas that need improvement
   Identifies training needs

                                                       42
        Management Responsibilities

   Determine policies for EQA (WHO, WHAT, WHEN,
    HOW)
   Assign responsibility
   Establish & maintain a system and schedule
   Receive EQA results and support corrective action
    measures
   Manage corrective action efforts
   Monitor and maintain records
   Investigate deficiencies
   Communicate outcomes

                                                    43
    Testing Personnel’s Responsibilities:
                 Overview

   Participate in the EQA program
   Take corrective actions
   Maintain EQA records
   Communicate outcomes to supervisors




                                            44
EQA Methods




              45
What is Proficiency Testing?


        Panels of specimens are sent
         to multiple test labs by
         reference laboratory/EQA
         provider
        Test labs perform tests and
         report results
        Results indicate quality of
         personnel performance and
         test lab operations
        Results are often compared
         across several testing labs
                                        46
     Proficiency Panels for HIVDR EQA

   VQA (NIH-supported, Rush University, Chicago1)
   TAQAS (TREAT Asia2)
   ENVA/QCMD3
   ANRS France
   DigitalPT/Accutest
   Acrometrix
   Others?



         1. Huang et al. J Clin Microbiol.43(8):3963-70 Aug 2005
         2. Land et al. Journal of Virological Methods 159(2): 185-193; Aug 2009
                                                                                   47
         3. Pandit et al. J Clin Virol. 43(4): 401-6; Dec 2008
What is On-site Evaluation?

            Periodic site visits to perform
             systematic assessment of lab
             practices
             – Focuses on how the lab
               monitors its operations and
               ensures testing quality
             – Provides information for
               internal process improvement




                                              48
What is On-site Evaluation? – Cont’d

                 Also referred to as audits,
                  assessments, or supervisory
                  visits
                 Learn “where we are”
                 Part of every lab quality
                  system
                 Measures gaps or deficiency
                 Collect information for:
                  – Planning & implementation
                  – Monitoring
                  – Continuous improvement
                                                49
What is Re-testing?

      The process by which a
       random selection of
       specimens are collected from
       the routine workload at the
       test lab and sent to the
       reference laboratory for
       validation
      Used to detect errors




                                  50
EQA Should Lead to
 Corrective Actions




                      51
Sample of Corrective Action Form




                                   52
           Small Group Discussion

1. What are the methods in a EQA program?
2. What type of EQA is most appropriate in our
   own situation?
3. What is the potential of internal bias in self-
   evaluation of quality?




                                                     53
How To Implement EQA



             Proficiency
               testing




     Re-
   testing


                       On-site
                      evaluation



                                   54
              On-Site Evaluation Process




    Pre-
               Entrance    Information    Outcome        Exit
 evaluation                                                        Reporting
               interview    gathering    assessment   conference
preparation




                                                                               55
On-Site Evaluation Checklist




                               56
            Tester Responsibilities:
        Ensuring a Productive Site Visit

   Before Visit:
    –   Record keeping is essential. Get organized
    –   Confirm date of visit
    –   Review written policies and procedures
    –   Conduct internal assessment in preparation of site visit
   During Visit
    – Participate in visits – cooperate
    – Ask questions of site assessors
   After Visit
    – Take corrective actions, where necessary
   Remember – visits are instructive, not punitive
                                                                   57
             On-site Evaluation:
         Pre-Evaluation Preparation

   Assign Responsibility
    – Laboratory management
    – Quality Manager
   Use Integrated team approach
   Determine who will conduct on-site evaluations
    – Select auditors with necessary skills:
       • Attention to detail
       • Ability to communicate effectively
       • Diplomacy
    – Provide appropriate training
   Schedule site visits

                                                     58
             On-site Evaluation:
         Pre-Evaluation Preparation

   Determine site visit frequency:
    – Established sites - twice per year/once per year
   For efficiency, cluster evaluations geographically
   Schedule in advance
    – Announced or unannounced visits




                                                         59
                 On-site Evaluation:
                 Entrance Interview

 The entrance interview sets the tone for the
  entire visit
 Provide overview of process in terms of what will
  be done
    –   Review of facility
    –   Record review
    –   Observation
    –   Interview with testing staff
    –   Use of proficiency panel
    –   Exit interview


                                                      60
                On-site Evaluation:
              Information Gathering

   Observe physical layout of the lab
   Evaluate testing operations
    –   Specimen collection
    –   Observation of test performance
    –   Quality control
    –   Inventory of kits
    –   Record-keeping
   Use quality systems checklist
   Conduct in an instructional, not punitive, manner.


                                                     61
    On-site Evaluation:
     Outcome Assessment


                  Evidence of
                implementation
    YES        & maintenance of         NO
                quality system?



• Conclude visit        • Ensure visit is
                          sufficient to
• No deficiencies         objectively document
                          evidence
                        • Expand visit or
                          observation
                        • Reference
                                                 62
                          deficiencies
             On-Site Evaluation:
              Exit Conference

 Review findings with supervisory and testing staff
 Make positive statements first – acknowledge
  staff cooperation and support
 Address negative findings – allow test site to
  discuss findings and provide additional
  information
 Provide instructions and timeframe for submitting
  plan for correcting problems



                                                       63
              On-site Evaluation:
                       Reporting

   Include information:
    –   Lab Name & Location
    –   Date of Visit
    –   Assessment Team Members
    –   Major Findings
    –   Recommendations for corrective actions
   Submit completed checklist and report to relevant
    authorities




                                                    64
          Minimal Package for QC/QA

 WHO-accredited DR labs should be enrolled in at
  least one EQA program
 Perform self-assessment at least once a year
 Internal/external controls must be included in
  each assay run and the results recorded.
    – Trending over time can be a very useful quality
      indicator
   Clearly define valid testing run criteria in related
    SOPs


                                                           65
                      Discussion

   What is EQA and why is it important?
   Which EQA Methods should we employ?
    – Proficiency Testing
    – On-Site Evaluation
    – Re-testing




                                           66
                     Reflection

   What are our responsibilities in preventing,
    detecting, and correcting errors before, during,
    and after testing?




                                                       67
                    Summary

 Identify requirements for QC/QA.
 Identify types of quality controls that must be in
  place and the rationale for each.
 Given individual lab needs, develop a plan for
  setting up an internal QC program.
 Identify types of quality assurance programs and
  their purpose.
 Given individual lab needs, develop a plan for
  setting up an external QA program.


                                                       68
    • Ability to communicate effectively
       • Diplomacy
    – Provide appropriate training
   Schedule site visits

                                                     58
             On-site Evaluation:
         Pre-Evaluation Preparation

   Determine site visit frequency:
    – Established sites - twice per year/once per year
   For efficiency, cluster evaluations geographically
   Schedule in advance
    – Announced or unannounced visits




                                                         59
                 On-site Evaluation:
                 Entrance Interview

 The entrance interview sets the tone for the
  entire visit
 Provide overview of process in terms of what will
  be done
    –   Review of facility
    –   Record review
    –   Observation
    –   Interview with testing staff
    –   Use of proficiency panel
    –   Exit interview


                                                      60
                On-site Evaluation:
              Information Gathering

   Observe physical layout of the lab
   Evaluate testing operations
    –   Specimen collection
    –   Observation of test performance
    –   Quality control
    –   Inventory of kits
    –   Record-keeping
   Use quality systems checklist
   Conduct in an instructional, not punitive, manner.


                                                     61
    On-site Evaluation:
     Outcome Assessment


                  Evidence of
                implementation
    YES        & maintenance of         NO
                quality system?



• Conclude visit        • Ensure visit is
                          sufficient to
• No deficiencies         objectively document
                          evidence
                        • Expand visit or
                          observation
                        • Reference
                                                 62
                          deficiencies
             On-Site Evaluation:
              Exit Conference

 Review findings with supervisory and testing staff
 Make positive statements first – acknowledge
  staff cooperation and support
 Address negative findings – allow test site to
  discuss findings and provide additional
  information
 Provide instructions and timeframe for submitting
  plan for correcting problems



                                                       63
              On-site Evaluation:
                       Reporting

   Include information:
    –   Lab Name & Location
    –   Date of Visit
    –   Assessment Team Members
    –   Major Findings
    –   Recommendations for corrective actions
   Submit completed checklist and report to relevant
    authorities




                                                    64
          Minimal Package for QC/QA

 WHO-accredited DR labs should be enrolled in at
  least one EQA program
 Perform self-assessment at least once a year
 Internal/external controls must be included in
  each assay run and the results recorded.
    – Trending over time can be a very useful quality
      indicator
   Clearly define valid testing run criteria in related
    SOPs


                                                           65
                      Discussion

   What is EQA and why is it important?
   Which EQA Methods should we employ?
    – Proficiency Testing
    – On-Site Evaluation
    – Re-testing




                                           66
                     Reflection

   What are our responsibilities in preventing,
    detecting, and correcting errors before, during,
    and after testing?




                                                       67
                    Summary

 Identify requirements for QC/QA.
 Identify types of quality controls that must be in
  place and the rationale for each.
 Given individual lab needs, develop a plan for
  setting up an internal QC program.
 Identify types of quality assurance programs and
  their purpose.
 Given individual lab needs, develop a plan for
  setting up an external QA program.


                                                       68

								
To top