Docstoc
EXCLUSIVE OFFER FOR DOCSTOC USERS
Try the all-new QuickBooks Online for FREE.  No credit card required.

Performance of a diagnostic test

Document Sample
Performance of a diagnostic test Powered By Docstoc
					      Performance of a diagnostic test


                                 Manuel Dehnert
                       16th EPIET Introductory Course
                           Lazareto, Menorca, Spain
                                   October 2010

Source:
Thierry Ancelle, Marta Valenciano, 2007
    What affects the performance of a test
      applied to a given population ?


1. the quality of the test itself

2. the frequency of the disease in the population




                                                    2
                        Outline

1. Performance of a test in an experimental setting
   (intrinsic characteristics)
   – sensitivity
   – specificity
   – choice of a threshold

2. Performance of a test in a population
   – predictive value of a positive test (PVP)
   – predictive value of a negative test (PVN)
   – impact of disease prevalence, sensitivity, and
     specificity on predictive values

                                                      3
 1. Performance of a test
in an experimental setting




                             4
                    Sensitivity of a test
• Ability of a test to identify correctly affected individuals
   – proportion of people testing positive
     among affected individuals

                               True patients
                              (gold standard)



         Test
                 +         True positive (TP)


                 -        False negative (FN)

                 Sensitivity (Se) = TP / ( TP + FN )
                                                                 5
             Sensitivity of a PCR
        for congenital toxoplasmosis

                                Patients with
                               toxoplasmosis
              True positive          54
Rapid test
              False negative          4
                                     58


     Sensitivity = 54 / 58 = 0.931= 93.1 %


                                                6
                   Specificity of a test
• Ability of test to identify correctly non-affected individuals
   - proportion of people testing negative
      among non-affected individuals


                           Non-affected people



         Test
                 +        False positive (FP)


                 -        True negative (TN)


                 Specificity (Sp) = TN / ( TN + FP )
                                                                   7
              Specificity of a PCR
         for congenital toxoplasmosis
                                 Individuals
                                   without
                               toxoplasmosis
              False positive          11
Rapid test
              True negative          114
                                     125


      Specificity= 114 / 125 =0.912 = 91.2 %


                                               8
              Performance of a test

                            Disease
                  Yes                  No

       +          TP                   FP
Test
       -          FN                   TN


                       TP                   TN
           Se =                 Sp =
                  TP + FN              TN + FP
                                                 9
                           Distribution of quantitative test results
                          among affected and non-affected people
                                         (ideal case)
                                                               Non affected:
                                            Threshold for
                                            positive result        Affected:
Number of people tested




                               TN                             TP



                          0         5          10             15          20
                                    Quantitative result of the test
                                                                               10
                                Distribution of quantitative results
                              among affected and non-affected people
                                          (realistic case)
                                                                     Non-affected:
                                              Threshold for
                                              positive result            Affected:
Number of people tested




                                         TN                     TP



                                          FN          FP
                          0          5                   10               15         20
                                      Quantitative result of the test
                                                                                     11
                              Effect of Decreasing the Threshold

                                                                   Non affected:
                                   Threshold for
                                   positive result                      Affected:
Number of people tested




                                            FP
                                   TN                     TP


                                     FN

                          0         5                10                  15         20
                                      Quantitative result of the test
                                                                                    12
       Effect of Decreasing the Threshold
                             Disease
                   Yes                  No

        +          TP                   FP
Test
        -
                   FN                   TN

                        TP                   TN
            Se =                 Sp =
                   TP + FN              TN + FP
                                                  13
                              Effect of Increasing the Threshold

                                                                     Non-affected:
                                                   Threshold for
                                                   positive result       Affected:
Number of people tested




                                        TN
                                                               TP
                                              FN

                                                          FP

                          0         5                10                    15        20
                                     Quantitative result of the test
                                                                                     14
       Effect of Increasing the Threshold
                             Disease
                   Yes                  No

        +          TP                   FP
Test
        -          FN                   TN


                        TP                   TN
            Se =                 Sp =
                   TP +                 TN + FP
                                                  15
            FN
  Performance of a Test and Threshold

• Sensitivity and specificity vary in opposite
  directions when changing the threshold

• The choice of a threshold is a compromise
  to best reach the objectives of the test
   – consequences of having false positives?
   – consequences of having false negatives?




                                                 16
        When false diagnosis (FP)
   is worse than missed diagnosis (FN)

• Example: Screening for congenital toxoplasmosis

  – One should minimise false positives

  – Prioritise SPECIFICITY




                                                    17
        When missed diagnosis (FN)
     is worse than false diagnosis (FP)

• Example: Testing for Helicobacter pylori infection

   – One should minimise the false negatives

   – Prioritise SENSITIVITY




                                                       18
Receiver Operating Characteristics curve
             (ROC curve)
• Representation of relationship
  between sensitivity and specificity for a test
• Simple tool to:
   – define best cut-off value of a test
   – compare performance of two tests




                                                   19
          Prevention of Blood Transfusion Malaria:
             Choice of an Indirect IF Threshold

Sensitivity (%)
100
                                        1/20 1/10
 80                      1/40
                  1/80
 60         1/160
                                                     IIF Dilutions
 40
          1/320

 20       1/640

  0
      0            20              40         60      80         100

                                1- Specificity (%)
                                                                     20
  Comparison of Performance of ELISA and CATT
   Test for Screening of Human Trypanosomiasis

Sensitivity (%)
100

 80                                              ELISA
                                                 CATT
 60

 40

 20

  0
      0           25            50          75           100
                       1- Specificity (%)
                                                          21
  Comparison of Performance of ELISA and CATT
   Test for Screening of Human Trypanosomiasis

Sensitivity (%)
100

 80                                                ELISA
                                                   CATT
 60

 40               Area under the ROC curve (AUC)

 20

  0
      0           25            50          75             100
                       1- Specificity (%)
                                                            22
2. Performance of a test in a
         population




                                23
                    Rationale

• The status healthy / sick of a patient is not known

• Tests are not perfect




                                                        24
                        Rationale

• Questions to be addressed by the clinician
   – probability that a individual with a positive test
     is really sick?
   – probability that a individual with a negative test
     is really healthy?

• Question to be addressed by the epidemiologist
   – proportion of positive tests corresponding
     to true patients?
   – proportion of negative tests corresponding
     to healthy subjects?

                                                          25
       Predictive Value of a Positive test
                     (PVP)
 • Probability that an individual testing positive
   is truly affected
    – proportion of affected people among
      those testing positive
                 Disease
              Yes       No

Test   +      TP           FP        PVP = TP/(TP+FP)



                                                     26
       Predictive Value of a Negative test
                     (PVN)
 • Probability that an individual testing negative is truly
   non-affected
    – proportion of non affected among
      those testing negative
                 Disease
              Yes       No

Test   -      FN           TN         PVN = TN/(TN+FN)



                                                          27
Predictive Value of a Positive and a Negative
                      test


             Disease
           Yes       No

       +   TP        FP     PVP = TP/(TP+FP)
Test
       -   FN        TN     PVN = TN/(TN+FN)



                                            28
                      Problem ?

• The predicted values depend on the sensitivity
  and on the specificity of the test as well as on the
  prevalence of the disease




                                                         29
       Relation between predictive values
           and sensitivity / specificity


              Disease
            Yes       No

       +     TP       FP      PVP = TP/(TP+FP)
Test
       -     FN       TN      PVN = TN/(TN+FN)



                                            30
   Step 1: Specify the prevalence (Pr) of disease



              Disease
            Yes       No

       +
Test
       -

             Pr         1-Pr
                                                    31
 Step 2: Use sensitivity (Se) to distribute test results
               among the diseased


                Disease
              Yes       No

       +     Se Pr
Test
       -   (1-Se)Pr

              Pr          1-Pr
                                                       32
 Step 3: Use specificity (Sp) to distribute test results
              among the non-diseased


                Disease
              Yes       No

       +     Se Pr     (1-Sp)(1-Pr)
Test
       -   (1-Se)Pr     Sp(1-Pr)

              Pr          1-Pr
                                                       33
Step 4: Determine the proportion testing positive and
           the proportion testing negative


               Disease
             Yes       No

       +    Se Pr     (1-Sp)(1-Pr)   Se Pr + (1-Sp)(1-Pr)
Test
       -   (1-Se)Pr    Sp(1-Pr)      (1-Se)Pr+ Sp(1-Pr)

             Pr          1-Pr
                                                     34
Step 5: Calculate PVP and PVN with appropriate
            expressions from Step 4


                     Se Pr
      PVP 
            Se Pr  (1  Sp)(1  Pr)



                   Sp(1 - Pr)
      PVN 
            Sp(1 - Pr)  (1  Se) Pr


                                                 35
Relation between predictive values
    and sensitivity / specificity

                 Se Pr
  PVP 
        Se Pr  (1  Sp)(1  Pr)

Increasing specificity  increasing PVP

               Sp(1 - Pr)
  PVN 
        Sp(1 - Pr)  (1  Se) Pr

Increasing sensitivity  increasing PVN
                                          36
Relation between predictive values
          and prevalence

                 Se Pr
  PVP 
        Se Pr  (1  Sp)(1  Pr)

Increasing prevalence  increasing PVP

               Sp(1 - Pr)
  PVN 
        Sp(1 - Pr)  (1  Se) Pr

Decreasing prevalence  increasing PVN
                                         37
                                    Se: 90%   Sp: 90%


   Prevalence: 50%

            PVP: 90%


              Ill        Not ill

       +            TP         FP
Test
                    FN         TN
       -


   Prevalence: 10%

            PVP: 50%
                                                        38
  Predictive value of a positive (PVP) and negative (PVN) test
  according to the prevalence (80% sensitivity and specificity)


                       100
Predictive value (%)




                       80
                                                             PVN
                       60

                       40

                       20
                                 PVP
                        0
                             0         25        50          75    100
                                            Prevalence (%)
                                                                         39
Example: Screening for human trypanosomiasis
               in two settings

• CATT test
  – Sensitivity = 95%
  – Specificity = 75%
• Endemic area
  – Prevalence = 20%
• Low endemic area
  – Prevalence = 0.5%

• 100,000 tests performed in each area

                                               40
Example: Screening for human trypanosomiasis
               in two settings
                           CATT test sensitivity = 95%
                           CATT test specificity = 75%
Prevalence = 20%
                     Trypanosomiasis

                     Yes         No         Total
              +     19,000     20,000      39,000
    CATT
               -     1,000     60,000      61,000
                    20,000     80,000     100,000
PVP = 48.7%
PVN = 98.4%
                                                         41
Example: Screening for human trypanosomiasis
               in two settings
                           CATT test sensitivity = 95%
                           CATT test specificity = 75%
Prevalence = 0.5%
                     Trypanosomiasis

                     Yes         No         Total
               +     475       24,875      25,350
    CATT
               -      25       74,625      74,650
                     500       99,500     100,000
PVP = 1.90%
PVN = 98.97%
                                                         42
                       Conclusions
• Sensitivity and specificity
   – intrinsic characteristics of a test
      • capacity to identify the affected
      • capacity to identify the non-affected
   – matter to laboratory specialists
   – independent from the disease prevalence

• Predictive values
   – performance of a test in real life
      • how to interpret a positive test
      • how to interpret a negative test
   – matter to clinicians and epidemiologists
   – dependent on the disease prevalence
                                                43
                   References


• Ancelle T. Statistique épidémiologique. Maloine.
  2002
• Case study: Toxoplamosis




                                                     44

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:12/12/2011
language:
pages:44