Pass-Fail Testing: Statistical Requirements and Interpretations by ProQuest


More Info
									                                              Volume 114, Number 3, May-June 2009
              Journal of Research of the National Institute of Standards and Technology

                                      [J. Res. Natl. Inst. Stand. Technol. 114, 195-199 (2009)]

                                Pass-Fail Testing:
                          Statistical Requirements and

Volume 114                                                  Number 3                                                     May-June 2009

David Gilliam, Stefan Leigh,                Performance standards for detector                    testing of any system with performance
Andrew Rukhin, and William                  systems often include requirements for                characteristics conforming to a binomial
                                            probability of detection and probability              distribution.
Strawderman                                 of false alarm at a specified level of
                                            statistical confidence. This paper reviews
National Institute of Standards             the accepted definitions of confidence
and Technology,                             level and of critical value. It describes the         Key words: binomial distribution;
Gaithersburg, MD 20899                      testing requirements for establishing                 confidence bounds; confidence coefficient;
                                            either of these probabilities at a desired            critical value; probability of detection;                      confidence level. These requirements                  probability of false alarm.                       are computable in terms of functions                      that are readily avail-able in statistical                software packages and general spreadsheet
                                            applications. The statistical interpretations         Accepted: April 27, 2009
                                            of the critical values are discussed. A table
                                            is included for illustration, and a plot is
                                            presented showing the minimum required
                                            numbers of pass-fail tests. The results
                                            given here are applicable to one-sided                Available online:

1.   Introduction                                                     less to perform binomial model based tests to determine
                                                                      estimates of this quantity. Similarly, for tests with contra-
   In evaluating the efficacy of equipment that is meant              band absent, the detection system either correctly reports
for detection of hidden contraband or dangerous sub-                  no detection, or it falsely reports the presence of contra-
stances, the instrument is often subjected to testing that            band: and the probability of a false alarm is presumed to
measures its performance against requirements set forth               remain fixed throughout the period of testing.
in protocols set by national or international standards                  For a detection system, PD or PFA can only be deter-
organizations. Performance requirements in these stan-                mined accurately by a sufficient number of trials.
dards include those for probability of detection (PD) and             However, there is a number called the confidence level
probability of false alarm (PFA) at a specified level of              (CL) that gives some sense of adequacy of the results
statistical confidence.                                               from a series of trials of a given size.
   The detection systems considered in this paper are all                CL is defined in terms of the binomial probability
assumed to behave according to a binomial distribution.               mass function, also called the binomial discrete density
Only two outcomes are considered for independent                      function, b (m; n, p),
trials with contraband present: the detection system either
                                                                                 b(m; n, p) = Pr (BIN( n, p) = m)
correctly reports detection or does not. Furthermore,                                                                                    (1)
the probability of detection must remain constant during                                            n!
To top