Docstoc

PQS-SOP overview-b

Document Sample
PQS-SOP overview-b Powered By Docstoc
					TechNet21 draft-b




          C E
       AN


                Q
                QU
                Q
                Q
        M



                   AL
                           PQS project
     OR




                    LT
                    LIT
                    LTTY
   RF
  PE




        SAFETY




            Overview of the
   PQS Standard Operating Procedures
                                      (T.o.R 6)




                                 Andrew Garnett




                                   January 2004

Revision history:
D1: 21.01.04 – Original draft for discussion.
D2: 13.02.04 – Revised following Geneva meeting. Title changed. Issued to WHO.
TechNet21 draft: 22.03.04 – Version for release at TechNet21
30.03.04: Further revised following discussions at TechNet 21 consultation


Contents

47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc 1 of 26                       09/09/2011
TechNet21 draft-b

Contents ......................................................................................................................... 1
Glossary ......................................................................................................................... 3
1.     Introduction ............................................................................................................ 4
2.     Format of SOPs ...................................................................................................... 5
3.     Performance specifications .................................................................................... 5
3.1        SOP: Develop a performance specification ....................................................... 5
3.2        SOP: Review and revise a performance specification ....................................... 7
3.3        SOP: Withdraw a performance specification ..................................................... 7
4.     Product verification ................................................................................................ 7
4.1        SOP: Develop a product verification protocol ................................................... 9
4.2        SOP: Review and revise a product verification protocol ................................. 10
4.3        SOP: Withdraw a product verification protocol .............................................. 11
5.     Testing laboratories .............................................................................................. 11
5.1        SOP: Approve a PQS testing laboratory .......................................................... 11
5.2        SOP: Re-evaluate a PQS testing laboratory ..................................................... 12
6.     Product pre-qualification...................................................................................... 13
6.1        SOP: Review applications for product pre-qualification ................................. 14
6.2        SOP: Re-evaluate a pre-qualified product ....................................................... 15
6.3        SOP: Remove a pre-qualified product ............................................................. 16
7.     Field testing .......................................................................................................... 17
7.1        SOP: Field-test a PQS product ......................................................................... 17
8.     User feedback ....................................................................................................... 18
8.1        SOP: Obtain feedback on the performance of a PQS product ......................... 19
Annex 1: References .................................................................................................... 21
Annex 2: Format for PQS standard operating procedure ............................................. 24




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Glossary
AD            Auto-disable (syringes)
FMWG          Field Monitoring Working Group
IEC           International Electro-technical Commission
ILAC          International Laboratory Accreditation Cooperation
ISO           International Standards Organization
PQS           Performance, Quality, Safety
PQS-SG        PQS Steering Group
QA            Quality Assurance.
QC            Quality Control
SOP           Standard Operating Procedure
SWG           Specification Working Group
AD            Auto-disable (syringes)
FMWG          Field Monitoring Working Group
IEC           International Electro-technical Commission
ILAC          International Laboratory Accreditation Cooperation




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
  TechNet21 draft-b



  1.              Introduction
  Item six in the Terms of Reference for the PQS project sets out the following
  requirement:
  To develop Standard Operating Procedures (SOP) for the following:
     1. Laboratory accreditation process.
     2. Review of applications for prequalification of equipment and devices.
     3. Development/revision of product specifications.
     4. Development/revision of test procedures.
     5. Discontinuation of products.
     6. Field testing.
     7. Performance feedback.

  In the course of carrying out this task, five of these seven SOPs have been broken
  down further. In all, as shown in Figure 1, thirteen separate SOPs have been written.
  The figure also indicates the relationship between the WHO/UNICEF internal
  procedures described in the SOPs and external standards and processes which have a
  fundamental impact upon these procedures. For example, ISO/IEC 17025 and its
  related laboratory accreditation process is a pre-requisite for the pre-qualification of a
  PQS testing laboratory; accordingly it must be well understood by members of staff
  and consultants who inspect and pre-qualify these organizations. Similarly ISO
  standards, such as ISO/DIS 7886-3 for AD syringes, play a key role in the
  specification and testing of certain categories of PQS product.

  Figure 1 – Structure of PQS standard operating procedures
WHO/UNICEF                                                                PQS S TANDAR D
INTERNAL PROCEDURES                                                  OPE RATING PROCE DURES




        PE RFORMANCE                     PROD UCT                           TESTING                 PROD UCT                        FIEL D                   USE R
        SP ECIFICATION                  VERIFICATION                     L ABOR ATORY           PRE -QUALIFICATION                 TESTING                 FEEDB ACK
        PROC EDURES                     PROC EDURES                     PROC EDURES               PROC EDURES                    PROC EDURES              PROC EDURES



       Ho w to d eve lo p an d          Ho w to d eve lo p an d          How to a pp rove       Ho w to e val ua te ap ps        Ho w to fie ld te st      Ho w to m on itor
     p ub lis h a PQS pro du ct       p ub lis h a PQS pro du ct         a PQS te stin g         fo r p re -qu al ifi ca ti on   a PQS pr od uct           u se r fe ed ba ck
   p erform an ce s p ecifi cati on     ver ifi ca ti on p rotocol         la bo ra tor y          o f P QS p rod ucts                                    o n P QS p ro du cts



        Ho w to re vie w a nd           Ho w to re vie w a nd          Ho w to re -e valu ate    Ho w to re -e valu ate a
      re vise a P QS p rod uct        re vise a P QS p rod uct           a PQS te stin g          p re-q ua li fie d PQS
   p erform an ce s p ecifi cati on    ver ifi ca ti on p rotocol          la bo ra tor y                p rod uct



                                                                                                     Ho w to re m ove a
        Ho w to wi th dra w a           Ho w to wi th dra w a
                                                                                                  p re-q ua li fie d pro du ct
          PQS p rod uct                     PQS p rod uct
                                                                                                fr om th e PQS da ta ba se
   p erform an ce s p ecifi cati on     ver ifi ca ti on p rotocol




         IS O a nd o the r                 IS O a nd o the r             IS O/IE C 1 70 25                                       In du stry-s tan da rd      Use r b od y
           s tan da rd s                     s tan da rd s                a ccre di tatio n                                           p rotocol s            pro ced ure s



                                                                         IL AC si gn atory
                                                                             o r o the r
                                                                          a ccre di tatio n
RELATED                                                                        b od y
EXTERNAL PROCEDURES

  The purpose of this document is to provide an introductory overview of each of the
  thirteen SOPs shown in Figure 1.

  47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


2.     Format of SOPs
All SOPs are written in accordance with the requirements of WHO document
FCH/IVB/ATT/GEN-001: Standard Operating Procedure for writing all ATT internal
procedural documents version 1. A template for writing PQS SOPs is attached as
Annex 2.
Several of the SOPs outlined below have annexes setting out standard letters,
recording forms, standard formats for performance specifications and the like.
Several SOPs also include flowcharts designed to provide a general overview of the
procedure described in the document. Some of these flowcharts are repeated below.

3.     Performance specifications
A PQS product performance specification is a published standard which sets out the
detailed performance requirements for an immunization-related product.
A performance specification defines the functional requirements of a product and
describes the environment within which it must operate; it also describes any interface
and inter-changeability requirements. Although it should set out clear verification
criteria, it must not attempt to describe how the functional requirements are to be met.
There are three SOPs which deal with performance specification issues. These are:
    SOP No FCH/ IVB/ATT/…/00X: How to develop and publish a PQS product
     performance specification.
    SOP No FCH/ IVB/ATT/…/00X: How to review and revise a PQS product
     performance specification.
    SOP No FCH/ IVB/ATT/…/00X: How to withdraw a PQS product performance
     specification.

3.1    SOP: Develop a performance specification
A performance specification must be comprehensive, unambiguous and written in a
consistent manner, in a ‘neutral’ style; this avoids favouring products from a particular
manufacturer or from a particular country or geographical region. Wherever possible
the specification must refer to ISO or other published standards. Finally, it must
comply fully with WHO immunization policies and guidelines current at the time of
publication.
There are two approaches to product selection. The non-developmental approach
invites manufacturers to offer compliant items from their existing product range. The
developmental approach identifies selected manufacturers who are prepared to modify
an existing product or to develop a new one. A performance specification which
targets commercially available products is likely to attract the widest and most
economically competitive range of compliant devices; this non-developmental
approach must be the first choice. The developmental approach should be taken only
when market investigation has established that no existing products can meet the
design criteria and that the design criteria themselves cannot be further modified.
Figure 2 gives an overview of the specification writing process.


47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Figure 2 – Developing a performance specification


                                                         Id en ti fy
                                                           n ee d




                                                  Form ul ate d es ig n
                                                       criteri a

                                                                                                              De velo pm e nta l
                                                                                                                   ro ute
                                                   Co nd uct ma rket
                                                    in ves ti ga tio n


                                                                                           YES



                        Do an y e xis tin g                   NO            Ca n d e sig n cr ite ria
                        p rod ucts s atisfy                                 b e ch an ge d to su it
                        d es ign cri ter ia?                                a va il ab le pr od ucts ?



       No n-d eve lo pm en ta l       YES                                                   NO
               ro ute
                                                                            In vite m a nu facture rs
                                                                               to p articip ate i n
                                                                            p rod uct de velo p me nt




                                                                                  Is the re              NO
                                                                                  s ufficie nt
                                                                              m arke t i ntere st?




                                                                                YES

                                                   Write n eu tra l
                                                   p erform an ce
                                                   s pe ci fica tio n




                                               Pu bl is h p er for ma nce
                                                    s pe ci fica tio n




A new performance specification will generally be written by a Technical Specialist
(TS) working closely with the relevant Specification Working Group (SWG). The
PQS Secretariat (Secretariat) will arrange for the draft document to be peer reviewed
after which it will be approved by the PQS Steering Group (PQS-SG) and published
on the PQS website1.




1
 Refer to document PQS organizational structure (T.o.R 1) for details of the structure and role of each
of these three entities.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

3.2        SOP: Review and revise a performance specification
The PQS-SG, assisted by the Secretariat and the SWGs, will carry out an annual
technical review of all current PQS performance specifications. This review will take
account of the following factors:
     WHO and UNICEF immunization programme changes which may affect the
       status or content of a specification;
     the introduction of new or revised international standards;
     other changes in programme requirements, such as the introduction of new
       vaccines;
     comments received from testing laboratories, technical specialists and
       manufacturers which identify technical shortcomings in the specification;
     feedback reports from field monitoring activities;
     technical or policy developments which may render a specification obsolete.
If exceptional circumstances justify it2, the PQS-SG may carry out an extraordinary
technical review of an individual specification in addition to the normal annual
review.
The SWGs will evaluate the need for revised performance specifications and will send
their revision proposals to the Secretariat for formal approval by the PQS-SG, either at
its next annual technical review, or at an extraordinary technical review. After
obtaining approval from the PQS-SG, the Secretariat will commission the appropriate
TS to carry out the work. During this process, the SWG and TS will consult with the
Field Monitoring Working Group (FMWG) who will recommend any necessary
revisions to the field monitoring techniques that are described in the performance
specification. The Secretariat will arrange for the revised document to be peer
reviewed. Finally, the PQS-SG will review and approve the changes and the
Secretariat will publish the revised document on the PQS website.

3.3        SOP: Withdraw a performance specification
As soon as it becomes evident that the product type described in a performance
specification is no longer required to help meet WHO objectives, the specification
should be formally withdrawn.
The appropriate SWG will identify out-dated performance specifications that should
be withdrawn. The SWG will send the withdrawal proposal to the Secretariat for
formal approval by the PQS-SG, either at its next annual technical review, or at an
extraordinary technical review.
The PQS-SG will examine the proposal and, if satisfied of the case for withdrawal,
will direct the Secretariat accordingly. The Secretariat will then amend the PQS
website and will circulate the decision to withdraw the document.

4.         Product verification
A product must be inspected or tested before it can be added to the list of pre-qualified
products in the PQS database. The general term adopted for this process is

2
    For example, as a result of a policy change, or after the discovery of a major technical flaw.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

verification. A verification protocol3 establishes whether a specific product from a
specific manufacturer satisfies the requirements of the relevant PQS performance
specification. There are three distinct methods:
    1. Type-examination.
    2. Independent type-testing.
    3. Full Quality Assurance.
Type-examination involves the systematic inspection of product samples supplied by
the product manufacturer who wishes to be pre-qualified. The technique is checklist-
based and it is a suitable verification method for technologically simple items that are
supplied in relatively small quantities. Where the risks arising from product failure
are low it may also be a suitable method of verification for simple high volume items,
and also for more complex items that are supplied in small quantities. Finally type
examination may also be justified where quality control standards in the relevant
industry are uniformly high. Type examination can be carried out in-house by WHO
and UNICEF or the work may be delegated to an independent inspecting organization.
Alternatively a type examination certificate issued by a reputable independent body
may be an acceptable substitute.
Independent type-testing involves the physical testing of a number of product samples
against a rigorously defined test protocol. This route is justified for complex high
value products and also for simple high volume products where the risks arising from
product failure are high. Independent type testing must be carried out by an ISO/IEC
17025 accredited testing laboratory.
Full Quality Assurance is justified for complex high value, low volume products
which require an element of site-specific design as well as on-site assembly and
commissioning work; cold rooms and standby generator installations are examples
which fall into this category. Manufacturers/approved installers offering such products
cannot be pre-qualified solely on the basis of type-examination or type-testing because
every instance of the product will in some way be unique. Instead pre-qualification
must be based on a thorough assessment of technical specifications and quality control
procedures, supported by references from other customers. To achieve full quality
assurance, the manufacturer/installer must generally be ISO 9001 accredited; in most
cases the completed installation will also be inspected and approved by an
independent inspecting organization4.
There are three SOPs which deal with product verification issues. These are:
    SOP No FCH/ IVB/ATT/…/00X: How to develop and publish a PQS product
       verification protocol.
    SOP No FCH/ IVB/ATT/…/00X: How to review and revise a PQS product
       verification protocol.
    SOP No FCH/ IVB/ATT/…/00X: How to withdraw a PQS product
       verification protocol.



3
 The term ‘verification protocol’ is used in preference to ‘test procedure’ as this phrase better
encapsulates the three different types of product examination described below.
4
    See SOP How to develop and publish a PQS product performance specification Annex 1.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

4.1    SOP: Develop a product verification protocol
A PQS product verification protocol describes in detail how the performance of a
class of immunization-related products will be tested or otherwise evaluated as part of
the PQS product pre-qualification procedure.
In some cases an individual performance specification will have its own unique
verification protocol. In other cases a verification protocol will be common to a group
of specifications - for example, all absorption cycle refrigerators may use the same
verification protocol, even though there may be more than one performance
specification for this category of equipment. The appropriate SWG will agree this
basic verification protocol group.
Figure 3 gives an overview of the procedure for deciding which of the three
verification routes is appropriate for a particular product. Note that the most difficult
choice is whether to follow the type-examination or the type-testing route for complex
high volume products. For safety-critical items the default decision will generally be
to adopt type-testing, unless the quality control standards of all pre-qualifying
manufacturers is likely to be high.




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Figure 3 – Which product verification route?
                                                        Ke y:
                                       Start
                                                        QC = Q ua lity con tro l




                                  Co mp le x
                  s im ple                             com pl ex
                                  o r s im ple
                                     item ?




            L ow o r           h igh                                         h igh              L ow o r       lo w
        h igh vo lu me ?                                                                    h igh vo lu me ?




                                                                                                                   On -si te
         lo w
                                                                                                                 a ss em bl y?
                                                                                                        no




                   no                            yes                                   no
                               Sa fe ty                        Hig h QC
                              critical ?                    s tan da rd s?

                                                                                                                          yes

                                                              yes

           Pre pa re                                                                   Pre pa re                     Pre pa re
      Typ e Exa mi na tio n                                                          Typ e Te stin g           Qu ali ty As su ran ce
           p rotocol                                                                   p rotocol                     p rotocol




The PQS-SG will determine the need for a new product verification protocol and the
Secretariat will commission the appropriate Specification Working Group to carry out
the work in consultation with the Field Monitoring Working Group (FMWG).
Following approval by the PQS-SG the Secretariat will publish the approved
document.


4.2        SOP: Review and revise a product verification protocol
In conjunction with its annual technical review of PQS performance specifications the
PQS-SG will carry out a similar review of all current PQS product verification
protocols taking account of the same factors outlined in section 3.2.
If exceptional circumstances justify it, the PQS-SG may also carry out an
extraordinary technical review of an individual product verification protocol in
addition to the normal annual review.
The SWGs will evaluate the need for revised protocols and will send their revision
proposals to the Secretariat for formal approval by the PQS-SG, either at its next
annual technical review, or at an extraordinary technical review. After obtaining
approval from the PQS-SG, the Secretariat will commission the appropriate TS to
carry out the work. During this process, the SWG and TS may also consult with the
FMWG. The Secretariat will arrange for the revised document to be peer reviewed.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

Finally, the PQS-SG will review and approve the changes and the Secretariat will
publish the revised document.

4.3        SOP: Withdraw a product verification protocol
As soon as it becomes evident that a particular product verification protocol is no
longer required to help meet WHO objectives, the document should be formally
withdrawn.
The SWGs will identify out-dated protocols that should be withdrawn. Protocols are
most likely to become obsolete because the product(s) to which they refer have been
removed from the PQS database. Each SWG will send its withdrawal proposals to the
Secretariat for formal approval by the PQS-SG, either at its next annual technical
review, or at an extraordinary technical review.

5.         Testing laboratories
An accredited testing laboratory offers one or more of the three product verification
services described in section 4 of this document, namely: Type-examination,
Independent type-testing or Full Quality Assurance. To be accredited, a laboratory
must be able to demonstrate its competency by conforming to appropriate
internationally or nationally accepted standards or codes of practice and must be able
to show to a competent third-party accreditation body that it does so; usually by peer
assessment5.
All independent type-testing must be carried out by an accredited laboratory. In
addition accredited laboratories may sometimes be employed to carry out type-
examination or full quality assurance of PQS products.
There are two SOPs which deal with testing laboratory issues. These are:
    SOP No FCH/ IVB/ATT/…/00X: How to assess a PQS testing laboratory.
    SOP No FCH/ IVB/ATT/…/00X: How to re-evaluate a PQS-assessed testing
       laboratory.

5.1        SOP: Assess a PQS testing laboratory
This SOP describes how to assess whether a testing laboratory has the relevant
competencies. Laboratories that satisfy the assessment process may then be pre-
qualified by WHO to verify specific categories of PQS products. Figure 4 gives an
overview of the pre-qualification procedure.




5
    Adapted from ILAC. Testing, Quality Assurance, Certification and Accreditation. ILAC-I2:1994.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Figure 4 – Procedure for assessing a testing laboratory

                                               Start




                                           Ma ke i nitia l            Tes t l ab ora to ry m ay ap pl y to WHO
                                             con ta ct                 o r WHO m ay in vite a pp lica tio ns




                                            L ab ora to ry                no
                                         IS O/IE C 1 70 25
                                           a ccre di ted ?



                                                    yes


                                         PQS Se cretari at
                                         s en ds Ap pr ova l
                                        In fo rm ati on Pa ck




                                           L ab ora to ry
                                            re sp on ds




                                             Facili ti es
                                                                          no
                                          a nd exp er tis e
                                             s uitab le ?



                                                    yes
                                                              Ch eck ag ai ns t cu rre nt
                                                               li st at www.il ac.o rg

                                  yes     Accre di tatio n              no
       On -si te in sp ecti on                                                      On -si te in sp ecti on
                                          b od y an ILAC
             o pti on al                                                                 e ss en tia l
                                            s ign atory?




                                 yes                                    no
              Accep t                      L ab ora to ry                                                          Re je ct
            la bo ra tor y                a ccep ta bl e?                                                        la bo ra tor y




The Secretariat will identify potentially suitable testing laboratories; will send the
laboratory a pre-qualification information pack and will analyze the information
dossier that the laboratory provides in response. If the dossier is satisfactory, the
Secretariat will generally arrange for a suitably qualified WHO employee or
consultant to visit the laboratory. Based upon the visit report, the Secretariat will make
a recommendation to the PQS-SG who will then direct the Secretariat whether or not
to pre-qualify the testing laboratory.

5.2    SOP: Re-evaluate a PQS testing laboratory
This SOP describes how to carry out an annual re-evaluation of a testing laboratory
which has been pre-qualified in accordance with the procedures outlined in section 5.2
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

above. It also sets out the grounds for disqualifying a laboratory that has failed to
perform adequately and it describes the disqualification process. Figure 5 sets out the
procedure.
Figure 5 – Annual review procedure for re-evaluating a testing laboratory

                                                           Start




                                                     a nn ua l re po rt       no
                                                                                        Se nd
                                                     re ce ive d fro m
                                                                                      re mi nd er
                                                      la bo ra tor y?



                                                                   yes



                     Re co rd                             Re po rt                     Re po rt
               n on -co m pli an ce                   s ati sfacto ry?               re ce ive d?
                                        no                                   yes


                                                                   yes
                                                                                      no



                Re as se ss me nt                    Ha ve a dve rs e
                  in sp ecti on                    u se r re po rts b ee n
          no       re qu ire d?        no               re ce ive d?



                            yes                                    yes

                                                        Ca rry o ut
                                                    re as se ss me nt
                                                       in sp ecti on




                      Ma intai n             yes       In sp ecti on         no       Ca ncel
                    PQS -as se ss ed                      re po rt                 PQS -as se ss ed
                        s tatus                       fa vou rab le ?                  s tatus




Each year the Secretariat will carry out a review of the performance of each pre-
qualified testing laboratory, will review the laboratory’s own annual report, will
arrange a reassessment visit where warranted, and will identify laboratories that have
failed to perform adequately. The Secretariat will then prepare an annual review and
recommendations for the PQS-SG. Based on the report, the PQS-SG will direct the
Secretariat either to disqualify a test laboratory, to amend the list of pre-qualified
services, or to maintain its PQS-assessed status unaltered. If a decision is taken to
disqualify the laboratory, the Secretariat will notify the test laboratory.

6.     Product pre-qualification
Before a product or device can be added to the PQS database, it must be pre-qualified.
A product can only be pre-qualified if it complies with a PQS performance
specification and with the related PQS product verification protocol.

47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

There are three SOPs which deal with product pre-qualification issues. These are:
    SOP No FCH/ IVB/ATT/…/00X: How to review applications for pre-
       qualification of PQS products.
    SOP No FCH/ IVB/ATT/…/00X: How to re-evaluate a pre-qualified PQS
       product.
    SOP No FCH/ IVB/ATT/…/00X: How to remove a pre-qualified product from
       the PQS database.

The document: WHO/BCT/03.09: Procedure for assessing the acceptability, in
principle, of single-use injection devices for procurement by United Nations agencies
describes a parallel, but closely related, procedure for syringes. Although this
procedure is not currently fully harmonized with the PQS format it has been
incorporated into the SOPs described below.

6.1    SOP: Review applications for product pre-qualification
Product pre-qualification involves a linked sequence of activities. Once a potentially
suitable product has been identified the manufacturer is sent a pre-qualification
information pack containing the relevant PQS performance specification and product
verification protocol, together with WHO’s terms and conditions and other related
material. The manufacturer is asked to submit a product dossier. The dossier is then
checked; if it is satisfactory the product is verified in accordance with the procedures
set out in the relevant verification protocol, using the services of a pre-qualified
independent testing laboratory, or other agreed method. Only after this stage has been
completed satisfactorily can the product be pre-qualified. Figure 6 outlines the
procedure.




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Figure 6 – Product pre-qualification process

                                                                 Start




                                                            Si ng le -us e                    yes
                                                             in je ctio n
                                                               d evice ?


                                                            no

                                                              e ith er/or

                           Ma nu fa cture r                                                 Se cretari at in vite s
                     a pp lie s to Se cr eta ria t                                           m an ufactu rer to
                            to p re- qu ali fy                                                  p re-q ua li fy



                                                         PQS Se cretari at
                                                     s en ds pr e-q ua li ficatio n
                                                        in fo rm ati on p ack



                                                           Ma nu fa cture r
                                                         s ub mi ts pro du ct
                                                              d os sie r



                                   Typ e-                                              Qu ali ty
                               e xa m in ati on                                       Ass ura nce
                Ob tai n                                    Veri fica tio n
               s am pl es                                     ro ute?



                                                                             Typ e- te st

                                                           Co mm is si on
                                                          te st la b ora to ry



                 Foll ow                                      Foll ow                                           Foll ow         Foll ow
          Typ e- exa mi na tio n                           Typ e- te sti ng                                       QA        p roced ur e i n
               p rotocol                                     p rotocol                                         p rotocol   WHO/B CT/0 3.09




                                         no                                                 yes
                     Re je ct                                 Pro du ct                                 Pre -q ua lify
                     p rod uct                             s ati sfacto ry?                              p rod uct




The Secretariat will establish and maintain a register recording details of all
applications for product pre-qualification. The Secretariat can recommend that a
product on the register should be pre-qualified. The PQS-SG may confirm this
recommendation and will then direct the Secretariat to add the product to the PQS
database. The Secretariat will then amend the PQS website and may otherwise
circulate the decision to add the product.

6.2    SOP: Re-evaluate a pre-qualified product
Once a product has been pre-qualified it must periodically be re-evaluated to ensure
that it continues to be fit for purpose. This SOP describes the re-evaluation procedure.
Figure 7 outlines the procedure.


47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


Figure 7 – Re-evaluation process

                                                                          Start




                                                                      Si ng le -us e    yes
                                                                       in je ctio n
                                                                        d evice ?


                                                                               no



                                                        yes             Pro du ct
                                                                        wa rni ng
                                                                       re ce ive d?


                                                                               no

                                   Ca rry o ut                          Ca rry o ut                 Foll ow
                                e xtra- ord in ary                       a nn ua l                p roced ur e
                                re -eva lu ation                     re -eva lu ation         in WHO/B CT/03 .0 9




                                With dr aw           no                                 yes          Re -val ida te
                                                                   Re -eva lu ati on
                                p rod uct as                                                         p rod uct as
                                                                   s ati sfacto ry?
                                    SOP                                                               th is SOP


                  Re f. SOP No FCH/ IVB/ATT/..../0 0X:
                 Ho w to re m ove a pre -q ua lifie d p ro du ct
                        fr om th e PQS da ta ba se .




      The Secretariat will ensure that every product on the database is re-evaluated
      annually, but circumstances may also arise that require re-evaluation to take place
      more frequently. The Secretariat will pass all recommendations arising from the
      re-evaluation process to the PQS-SG which will review these recommendations
      and direct the Secretariat either to re-validate or to remove a product from the PQS
      database. If a decision is taken to remove a product, this will be done as described
      in Section 6.3.

6.3      SOP: Remove a pre-qualified product
Circumstances can arise which make it necessary to remove a pre-qualified product
from the database. This SOP identifies these circumstances and describes the removal
procedure.
A product may be removed for one of the following reasons:
    the product may be unsatisfactory – for example as a result of a change to the
      product specification, poor reliability, etc.;
    the manufacturer may be unsatisfactory – for example poor quality control,
      financial irregularity, etc.;
    if there has been a major change to the relevant performance specification or
      verification protocol, the product may no longer comply with the revised
      requirements;

47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

       WHO may withdraw the relevant performance specification or verification
        protocol;
      WHO may implement a policy change which renders the product obsolete.
If the Secretariat recommends that a product loose its pre-qualification status, the
PQS-SG may confirm this recommendation and will then direct the Secretariat to
remove the product from the PQS database.

7.         Field testing
Field-testing provides manufacturers with information to improve product design and
it can also help end-users to choose products that are best suited to their needs.
However, if test results are to be useful, there must be a completely clear
understanding of the purpose to which they will be put. This requires well-developed
evaluation techniques and protocols based on standardized criteria. The aim must be
to obtain the maximum amount of useful information on product performance at
minimum cost and with minimum disruption to the working lives of health care staff.
There is one SOP which deals with field testing: SOP No FCH/ IVB/ATT/…/00X:
How to field-test a PQS product.

7.1        SOP: Field-test a PQS product
Obtaining good quality information on product performance in the field is a challenge.
There are two generic methods that can be used to field-test PQS products – field
surveys and real-time instrumentation. In addition there is a tailor-made field-survey
based tool which has been specifically designed to evaluate the ease of use of AD
syringes6. Figure 8 indicates the strengths and weaknesses of each of these three
methods.
Figure 8 – Field-testing techniques
Method                Strengths                                Weaknesses
Field-survey           Can produce statistically reliable,     Can be time consuming and
                        quantitative results.                    expensive to administer.
                       Able to capture multi-dimensional       Dependent on skills and motivation
                        factors, including user behaviour.       of the survey team.
MSF/WHO tool           Well-researched product-specific        Does not yet ensure with certainty
for the assessment      tool.                                    ‘reproducibility over time’.
of single use          Satisfactory internal consistency.
injection devices 7    Can produce quantitative results.

Real-time              Relatively cheap to administer.         Narrowly focused on a specific
instrumentation        Provides accurate and complete           indicator.
                        quantitative records.                   Cannot directly capture user
                       Enables continuous monitoring to         behaviour.
                        take place over extended time
                        periods.




6
 See Gergonne, B., Grandesso, F., Pinoges, L., Construction and validation of a tool for the
assessment of single use injection devices under field conditions MSF Epicentre 2003.
7
    Ibid p.14.
47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

Figure 9 outlines the various steps required to develop a test protocol and to carry out
the field-test itself. These steps are described in detail in the SOP.
Figure 9 – Developing a field-test protocol and implementing a field-test
     ACTIO N BY:                                                                       ACTIO N BY:
                                                                     Start



                                                                   Pro du ct
                                                                                       FMWG
                                                                   re view



                                                              Pre pa re /re vise
                                                                                       FMWG
                                                            m od el te st p rotocol




                                                  no              Pro to col
                                                                 a pp rove d           PQS -SG
                                                               b y PQS -SG?



                                                                             yes       Se cretari at,
                                                                                       FMWG,
                                                              Ob tai n fu nd ing       Go vern me nt(s) ,
                                                              a nd con se nts          WHO/UNICEF
                                           Se e pa ra 5.4
                                                               a nd carry ou t         cou ntry o ffi ce s an d/or
                                                                  fi el d-tes t        con su ltan t(s ).
                                                                                       Pro du ct m a nu fa cture r(s )

                                                                                       FMWG
                            Re vise                              Pe er re vi ew
      Se cretari at                                                                    Ma nu fa cture r(s )
                          te st-re po rt                      fi el d-tes t r ep ort


                                   no



                          Tes t-re po rt            no          Tes t-re po rt
                        ra is es se rio us                       a pp rove d           PQS -SG
                             is su es ?                        b y PQS -SG?



                                    yes                                      yes

                          Oth er a ction                           Pu bl is h
     PQS -SG                                                                           Se cretari at
                      b efo re pu bl icatio n                    te st-re po rt




8.      User feedback
User feedback can supply valuable information on the performance of equipment
under field conditions. Much of this information may be anecdotal and much will
tend to be qualitative. Nevertheless useful quantitative data on overall equipment
reliability can be obtained from management reporting systems and also from field
inspections carried out by review teams during exercises such as programme reviews,
Effective Vaccine Store Management (EVSM) or Vaccine Management inspections.
There is one SOP which deals with user feedback: SOP No FCH/ IVB/ATT/…/00X:
How to obtain feedback on the performance of a PQS product.




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

8.1      SOP: Obtain feedback on the performance of a PQS product
This SOP outlines some techniques for obtaining user feedback. It also establishes the
administrative framework for capturing such information and for channelling it to
where it is most useful. To be effective, this process will require the active
cooperation of national programme managers as well as the assistance of technical
staff in WHO/UNICEF country and regional offices.
 Figure 10 characterizes the strengths and weaknesses of four principal methods that
can be used to obtain feedback on equipment performance in the field.
Figure 10 –Methods for obtaining feedback on product performance in the field
Method              Strengths                                 Weaknesses
Electronic           Relatively cheap to establish and        Requires internet access.
reporting             maintain.                                Responses need to be moderated and
                     System administrator can easily call      collated before ‘publication’.
                      for feedback on a specific product.      Respondents are self-selecting.
                     Continuous real-time data resource.      Requires motivated country staff.
                     Moderated and consolidated               Data likely to be largely anecdotal.
                      responses easily accessible by other
                      users.
Management           Relatively cheap to operate.             Relies on motivated country staff to
reporting            Provides data at regular time             achieve accuracy and timeliness.
                      intervals.                               Reliable data depends upon honest
                     Helps to build MoH management             reporting and a no-blame
                      strength in depth.                        management culture.
                     MoH management may carry out             Liaison needed with MoHs in order
                      analysis.                                 to collect data in a form that is useful
                     Data can be quantitative.                 to PQS.
Manufacturer’s       No cost to PQS – part of pre-            None if properly implemented.
product defect        qualification conditions
reports              Data most likely to be safety-related
                      which may lead to product
                      replacement or retrofit.
Field inspections    Targeted data collection method.         May be expensive.
                     Data generally collected and             Generally administered by agency
                      analyzed by specialists.                  staff or consultants.
                     Data can be quantitative.                One-off or infrequent data source.


Figure 11 summarizes the procedure for identifying, collecting, collating and
distributing data received from the various sources shown in Figure 10.
Figure 11 – Product feedback procedure




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

                                                                                                                ACTIO N BY:
                                             Pre pa re /m ai nta in
                                                                                                                FMWG
                                           FEEDBACK SCHE DULE



                                                Cir cu la te
                                                                                                                Se cretari at
                                           FEEDBACK SCHE DULE



                                               Ob tai n an d co ll ate
                                               FEEDBACK DATA                                                    Se cretari at
                                                       fr om :




    PQS e lectron ic               Da ta fro m                       Ma nu facture r's       WHO/UNICEF
      re po rti ng        n ati on al m an ag em en t                p rod uct de fe ct           fi el d
       s ys te m                    re po rti ng                         re po rts            in sp ecti on s




                                                                                                                Se cretari at
                                                                                 yes            Pu bl is h
                                                 Ma jor pr ob le m                                              FMWG
                                                                                          PROD UCT AL ERT
                                                  id en ti fie d?                                               SWG
                                                                                           o n P QS w eb site
                                                                                                                PQS -SG

                                                              no

                                               Co ns oli da te d ata i nto
                                               ANNUAL FEEDB ACK                                                 Se cretari at
                                                    SUMMARY




          Se nd to        In corp ora te d ata i nto                  Se nd to FMWG             Pu bl is h
      m an ufactu rer           PROD UCT                                   a nd to                 on           Se cretari at
      fo r co mm en t   RE- EVALUATION RE PORT                            SWGs                PQS we bs ite




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b



Annex 1: References
The references below include a consolidated list of the documents cited or listed in the
thirteen PQS SOPs described above.
1stNclass.com. Quality Glossary – 250+ common Quality Management acronyms and
terms. http://www.1stnclass.com/quality_glossary.htm
American National Standards Institute. http://www.nssn.org/ The National Standards
System Network provides a wide range of standards information from ANSI, other
U.S. private sector standards organizations, government agencies and international
standards organizations.
British Standards Institute. http://www.bsonline.techindex.co.uk/ Lists British and
international standards, which can be purchased on-line.
ECRI. A non-profit organization specializing in information about medical devices
and equipment http://www.ecri.org/
European Community. Guide to the implementation of directives based on the New
Approach and the Global Approach. Office for Official Publications of the European
Communities Luxembourg, 2000. ISBN 92-828-7500-8. Also at
http://europa.eu.int/comm/enterprise/newapproach/ Document outlining the New
Approach to technical harmonization and the Global Approach to conformity
assessment used within the EC to achieve the granting of a ‘CE’ mark.
Inter-Agency Procurement Services Office (IAPSO). http://www.iapso.org/supplying/
On-line catalogue.
International Laboratory Accreditation Cooperation. ILAC-12:1994 Testing, Quality
Assurance, Certification and Accreditation. www.ilac.org
International Laboratory Accreditation Cooperation. ILAC-13:1996 The Role of
Testing and Laboratory Accreditation in International Trade. www.ilac.org
International Laboratory Accreditation Cooperation. ILAC-G10:1996 Harmonised
Procedures for Surveillance and Reassessment of Accredited Laboratories.
www.ilac.org
International Laboratory Accreditation Cooperation. ILAC-G15:2001 Guidance for
Accreditation to ISO/IEC 17025. www.ilac.org
International Laboratory Accreditation Cooperation. The ILAC Mutual Recognition
Arrangement. www.ilac.org
International Standards Organization. Information on DEVCO – ISO Committee on
developing country matters. ISO, 2000-05/600. ISO, 2000.
International Standards Organization. Standardizing bodies having notified
acceptance of the WTO TBT Code of good practice for the preparation, adoption and
application of standards. ISO, 2003.
International Standards Organization. http://www.iso.org Lists ISO standards, which
can be purchased on-line.


47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b


ISO/IEC 17025:1999 General requirements for the competence of testing and
calibration laboratories. www.iso.org
Office of the Assistant Secretary of Defence for Economic Security. SD-15:
Performance Specification Guide. Washington D.C., 1995.
Office of the Assistant Secretary of Defence for Economic Security. SD-15:
Performance Specification Guide. Washington D.C., 1995.
http://www.dsp.dla.mil/documents/sds.htm
Office of the Assistant Secretary of Defence for Economic Security. SD-5: Market
Research – gathering information about commercial products and services.
Washington D.C., 1997. http://www.dsp.dla.mil/documents/sds.htm
Story, M.F., Mueller, J.L. Mace, R.L., The Universal Design File: Designing for
People of All Ages and Abilities. NC State University, Center for Universal Design.
http://www.design.ncsu.edu:8120/cud/pubs/center/books/ud_file/toc3b14.htm
Stup, R. Standard Operating Procedures – a writing guide. Pennsylvania State
University College of Agricultural Sciences Agricultural Research and Cooperative
Extension, 2001.
UNDP/World Bank/WHO-TDR. Good laboratory practice training manual. Undated
and unreferenced.
UNICEF Supply Division. http://www.supply.unicef.dk/Catalogue/ On-line catalogue.
WHO/EPI/LHIS/97.03 EPI Performance Specifications and Test Procedures - Annex
1: Testing institutions. February 1999 revision.
World Health Organization. A WHO guide to good manufacturing practice (GMP) -
Part 1: Standard operating procedures and master formulae. WHO/VSQ/97.01,
1997.
World Health Organization. SOP.015/00: Publication of specifications of products
for procurement. WHO/HTP/EDM/QSM/2001. Review date: 2003.
World Health Organization. SOP.019/00: Standard procedure for writing a Standard
Operating Procedure (SOP). WHO/HTP/EDM/QSM/2001. Review date: 2003.
World Health Organization. FCH/IVB/ATT/GEN-001: Standard Operating
Procedure for writing all ATT internal procedural documents version 1.
World Health Organization. WHO/BCT/03.09: Procedure for assessing the
acceptability, in principle, of single-use injection devices for procurement by United
Nations agencies
World Standards Services Network. (WSSN).
http://www.wssn.net/WSSN/print/listings/worldmap.html Gives links to national and
international standards organizations worldwide.
World Standards Services Network. Geographical list of Web sites in WSSN. WSSN.
http://www.wssn.net/WSSN/print/listings/worldmap.html Gives links to national and
international standards organizations worldwide.
World Trade Organization Technical Barriers to Trade (TBT). Code of good practice

47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b

for the preparation, adoption and application of standards. WTO, ????




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc09/09/2011
TechNet21 draft-b



Annex 2: Format for PQS standard operating procedure




STANDARD OPERATING PROCEDURE



SOP N° : FCH/ IVB/ATT/…/00X                                 Version n°: 01        Page: 1 of X




1st edition :
DD/MM/YYYY
                                                           TITLE:
Effective date :
DD/MM/YYYY
Revision date :
DD/MM/YYYY




                           Name                     Date                     Signature


Authorised by:             ATT Coordinator



Revewed by:                Group Leader



Checked by:


Original prepared by:



                                  ACCESS TO TECHNOLOGIES




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc 24 of 26                            09/09/2011
SOP No FCH/ IVB/ATT/…/00X    Date of issue:    Revision Date:
Version: 01                  Effective date:   Page: 25 of 26
Title:




Table of Content


1.0 Purpose


2.0 Scope


3.0 Responsibility


4.0 Documentation required


5.0 Procedure


6.0 Distribution




                                                          25 of 26
TechNet21 draft-b


                            REVISION HISTORY FORM
                      (Form number: FCH/IVB /ATT/GEN/F002)




SOP Number

Date of original version:




                                      REVISIONS
Date                          Reason                Authorized by (Signature
                                                    and Name)




47db3548-cc3e-4c79-ad69-5c31dde7aad1.doc 26 of 26                  09/09/2011

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:8
posted:9/9/2011
language:Latin
pages:26