Docstoc

Biometric Conformance Test Methodology Standards_ Test

Document Sample
Biometric Conformance Test Methodology Standards_ Test Powered By Docstoc
					Biometric Conformance Test Methodology
     Standards, Test Architectures
            and Test Suites


              Dylan Yaga and Fernando Podio,
                 Computer Security Division
             Information Technology Laboratory
       National Institute of Standards and Technology




                                         BCC2009 – September 22, 2009
                  Overview

Conformance Testing
Conformance Testing Methodology Standards
NIST/ITL CSD Conformance Test Architectures and Test
Suites
Ongoing CTS Developments
         Need for Conformance Testing

The existence of biometric standards alone is not
enough to demonstrate that products meet the
technical requirements specified in these standards.
Conformance testing captures the technical description
of a specification and measures whether an
implementation faithfully implements the specification.
Some national and international conformance testing
methodology standards have been published. Others
are under development.
Taxonomy of Conformance Testing Standards

 Boilerplate
    Foreword, Introduction, Scope, Conformance,
    Normative References, Terms and Definitions

Key elements
    List of requirements in the base standard, test
    assertions, test report, implementation conformance
    statement

Annexes
           Hierarchy of Conformance Tests(*)
 Level 1 testing
  “Conformance testing methodology that checks field by field and byte by
  byte conformance with the specification of the Biometric Data Interchange
  Record (BDIR) as specified in the base standard, both in terms of fields
  included and the ranges of the values in those fields.”
 Level 2 testing
  “Conformance testing methodology that tests the internal consistency of
  the BDIR under test, relating values from one part or field of the BDIR to
  values from other parts or fields of the BDIR.”
Level 3 testing
  “Conformance testing methodology that tests that a BDIR produced by an
  Implementation Under Test is a faithful representation of the Input
  Biometric Data Record subject to the constraints of the parameters in the
  metadata records.”
(*): from ISO/IEC 29109-1, Conformance testing methodology for biometric interchange records
                                                 (
format – Part 1: Generalized conformance testing methodology
Conformance Testing Methodology Standards for
        Biometric Technical Interfaces
 Published American National Standards:
   ANSI INCITS 429-2008, Conformance Testing Methodology for ANSI
   INCITS 358-2002, BioAPI Specification

 American National Standards under development:
   INCITS Project 1853-D - Conformance Testing Methodology for
   Common Biometric Exchange Formats Framework (CBEFF) Data
   Structures Specified in INCITS 398-2008
Conformance Testing Methodology Standards for
        Biometric Technical Interfaces
  Published international standards:
    ISO/IEC 24709.1: 2007, Conformance Testing for BioAPI (ISO/IEC
    19784-1:2006) Part 1: Methods and Procedures

    ISO/IEC 24709.2: Conformance Testing for BioAPI (ISO/IEC 19784-
    1:2006) Test Assertions for Biometric Service Providers

  International standards under development:
    ISO/IEC 24709.3: 200x, Conformance Testing for BioAPI (ISO/IEC
    19784-1:2006 ) Part 3: Test Assertions for BioAPI Frameworks
Conformance Testing Methodology Standards for
     Biometric Data Interchange Formats
 Published American National Standards:
   ANSI INCITS 423.1-2008, Part 1: Generalized Conformance Testing
   Methodology

   ANSI INCITS 423.2-2008, Part 2: Conformance Testing
   Methodology for ANSI INCITS 378-2004, Finger Minutiae Format for
   Data Interchange

   ANSI INCITS 423.3-2008, Part 3: Conformance testing methodology
   for ANSI INCITS 377-2004 Finger pattern data interchange format

 American National Standards under development:
   ANSI INCITS 423.4-200x, Part 4: Conformance Testing
   Methodology for ANSI INCITS 381-2004, Finger Image Based
   Format for Data Interchange (completed)
Conformance Testing Methodology Standards for
Biometric Data Interchange Records as Defined in
           ISO/IEC 19794-x Standards
 Published International Standards:
    ISO/IEC 29109-1:2009, Part 1: Generalized CTM
 International Standards under development (1st generation):
    ISO/IEC 29109-2 – Part 2: Finger Minutia Data (completed)
    ISO/IEC 29109-4 – Part 4: Finger Image Data (completed)
    ISO/IEC 29109-5 – Part 5: Face Image Data
    ISO/IEC 29109-6 – Part 6: Iris image data
    ISO/IEC 29109-7 – Part 7: Signature/sign time series data
    ISO/IEC 29109-8 – Part 8: Finger pattern skeletal data
    ISO/IEC 29109-9 – Part 9: Vascular image data
    ISO/IEC 29109 -10 – Part 10: Hand geometry silhouette data
Conformance Testing Methodology Standards for
Biometric Data Interchange Records as Defined in
           ISO/IEC 19794-x Standards
 International Standards under development (Cont.):
    “Second generation” of Conformance Testing Methodology
    standards: amendments to the “second generation” of the
    biometric data interchange format standards:
  Rev. of 19794-1: 201x   Rev. of 19794-1 Amd. 1: 201x
  Rev. of 19794-2: 201x   Rev. of 19794-2 Amd. 1: 201x
         …                     …
  Rev. of 19794-9:201x    Rev. of 19794-9 Amd 1:201x

  19794-11:201x           19794-11 Amd.1: 201x (Sig./Sign processed data)
  19794-13:201x           19794-13 Amd. 1:201x (Voice data)
  19794-14:201x           19794-14 Amd. 1:201x (DNA data)
Conformance Testing Methodology Standards
            and Testing Tools
 NIST/ITL supported development of conformance testing
 methodology standards in INCITS M1 and JTC 1/SC 37 and
 promoted conformity assessment:
   M1 Standing Document 4 - Report to M1 on Issues for
   Harmonizing Conformity Assessment to Biometric Standards
    http://m1.incits.org
   SC 37 Standing Document 7 - SC 37 Conformity Assessment
   Report - www.jtc1.org – select SC 37 – Standing Documents

 NIST/ITL is developing conformance testing architectures
 and Conformance Test Suites for implementations of
 selected biometric data interchange formats, technical
 interfaces and Biometric Information Records.
 Initial Implementation of Conformance Test
                    Suites
NIST/ITL CSD Conformance Testing Methodology Standard
for ANSI INCITS 358-2002, BioAPI Specification
 This Conformance Test Suite (CTS) designed to test
 implementations of BioAPI V1.1 is publicly available at:
   http://www.itl.nist.gov/div893/biometrics/BioAPI_CTS/index.htm

 Test results derived from testing a number of commercial
 products with this CTS were cross-validated with test
 results on the same products tested by a similar tool
 independently developed by DoD BTF.
 Advanced NIST/ITL CSD Conformance Test
 Architectures (CTAs) and Test Suites (CTSs)

 Designed to test implementations of:
    Biometric data interchange formats
    Biometric Information Records (BIRs)

These test tools support the development of
biometric standards and help developers and
users verify conformance to these standards,
leading to greatly increased levels of confidence
in products.
NIST/ITL CSD Conformance Test Architectures
       (CTAs) and Test Suites (CTSs)
 Beta 1 CTA
   Released to the public 08/2008 and demonstrated at BCC
   2008 together with a CTS to test Patron Format A BIRs.
 http://www.itl.nist.gov/div893/biometrics/CBEFF_PFA_CTS/index.
   htm
 In pre-release testing of the Beta 2 CTA with four CTS
 modules that test implementations of:
   Finger minutiae data formats: INCITS 378-2004 and 2009
   (awaiting public.)
   Finger image data formats: INCITS 381-2004 and 2009
   (awaiting public.)
 Ongoing development of CTS modules for selected
 biometric data interchange formats (19794-x).
    NIST/ITL Computer Security Division
Conformance Test Architecture (CTA) - Beta 2
 Supports Conformance Test Suites to test:
    Implementations of biometric data interchange formats
    Implementations of Biometric Information Records


                          GUI



                       Controller


         SBH              BDB             SB
        Module           Module          Module
      NIST/ITL Computer Security Division
  Conformance Test Architecture (CTA) - Beta 2
Key features:
  Independent components – each can be independently developed & tested.
  Dynamically-Loaded CTS Modules – Modules automatically loaded at runtime.
  Binary data is in context – part of a Field data structure – actual length /
  expected length information.
  XML Manifests: define the format to which the binary data is expected to
  conform.
  Test Logs (XML) & Test Reports (HTML)


                        GUI                    The Architecture supports
                                                 local and web service
                                                       controllers
         Local              Web Service
        Controller           Controller
               CTA Useful Analysis Example
                                 Record Header (Fields = 9, Bytes = 21, Errors = 1)
                                       001 01_FormatIdentifier         0000: 46 4d 52 00
                                       002 02_Version                  0004: 30 33 30 00
                                       003 03_RecordLength             0008: 00 00 01 6b
                         *                    Error 1: RECLEN_NE_BYTESREAD (363 [0x16B] <> 364 [0x16C])
                                       004 04_CbeffPidOwner            0012: 00 42
                                       005 05_CbeffPidType             0014: 00 11
                                       006 06_CaptureEqmtCompliance    0016: 00
                                       007 07_CaptureEquipmentID       0017: 00 b5
                                       008 08_NumFingerViews           0019: 02
                                       009 09_Reserved                 0020: 00

                         . . .


                         Extended Data Block Length (Fields = 1, Bytes = 2, Errors = 0)
                                    229 26_ExtendedDataBlockLength 0351: 00 0b

                                 Extended Data Header (Fields = 2, Bytes = 4, Errors = 1)
                                       230 27_ExtendedDataAreaType     0353: 00 01
                                       231 28_ExtendedDataAreaLength   0355: 00 0b
                         *                    Error 1: XDALen_IS_NOT_17_0x11__QUADRANTS
Highlighted: example
of binary data used to           Ridge Count Extraction Method (Fields = 1, Bytes = 1,   Errors = 0)
                                       232 29_RidgeCountExtrnMethod   0357: 01
 generate the binary
    file to the right            Ridge Data (Fields = 3, Bytes = 3,   Errors = 0)
                                       233 30_RidgeCount_Idx_1        0358: 12
                                       234 31_RidgeCount_Idx_2        0359: 14
                                       235 32_RidgeCount              0360: 03

                                 Ridge Data (Fields = 3, Bytes = 3,   Errors = 0)
                                       236 30_RidgeCount_Idx_1        0361: 15
                                       237 31_RidgeCount_Idx_2        0362: 13
                                       238 32_RidgeCount              0363: 01
CTA Welcome Screen
      CTA Batch Test Binary Files




Specify Manifest or allow for Decode
                    CTA Decode
                                 Binary data decoded into
                                    a manifest




Raw Binary Data File
(Represented as Hexadecimal)




   One at a Time or Batch
Test Cases Increase Confidence
                    There are several test case
                    types that test suites can
                    implement. These levels
                    allow for greater confidence
                    that the module has been
                    implemented correctly. Test
                    Cases play a large role.
CTA Manifest / Test Case Builder




      Editing an Individual Test Case and the
               summary of its results
CTA Key Feature: Data Formats Unit
             Testing
“The goal of unit testing is to isolate each part of
the program and show that the individual parts are
correct.”
                                   -- Wikipedia
Good Data Must Pass      … and …
Bad Data Must Fail … As Expected
Test Types
  Data Only
  Structure Only
  Partial
  Complete
Broad Coverage in Few Test Cases
    Data-Only Test Case: Finger Views LE 176
                     (0xB0)
Test Report Displaying Results of all Fields




The Errors that were expected from this test




The Errors that were found as a result of the test, which, in this case, are the same as expected




And the Overall Result:
              NUnit-Inspired Display




Passing files indicated with a green marker
Failing files indicated with a red marker
Improperly Formed files indicated with a black marker
       CTS Module Developers’ Kit

                    GUI
                                        Included

                  Controller


        SBH         BDB         SB
       Module      Module      Module



Undergoing Development
C# 2.0 Sample Module Code
CTS Module Developers’ Kit Key Features
 Useful “Module Assistant” Library

 Powerful “Groups of Fields”, e.g.,
   A Record Consists of a Record Header + 1 or More Views
     A View Ends With “Extended Data”
        Extended Data is Zero or More of Any Combination of Any 4 XData Types
   Simplifies Both Decode and Manifest / Test Case Testing


 XML File of Fields, Groups and Errors + XSLTs
     C# Source Code and Documentation Tables are Easily
     Synchronized!!!
Developers’ Kit Key Enhancements in
           Consideration

.NET 3.0 or Higher
“Module Assistant” Library Improvements
  Automatically Handle Most Level 1 Testing and Some
  Level 2 Testing
  Automatically Handle Record Structure Validity (Via XML
  Schemas or Similar Technology)
  Map Combined Fields to Logical Individual Fields
    E.g., 2-bit “Type”, 2-bit “Reserved” and 4-bit “Count” Handled
    as Three Fields
    Easier Logic, Maintenance/Upgrades and Documentation
                     Contact Information
Dylan Yaga
NIST/ITL Computer Security Division
email: dylan.yaga@nist.gov

Fernando Podio
email: fernando.podio@nist.gov



Our sincere thanks to Mark Jerde from ID Technology Partners (NIST Contractor)
 for his contributions as the key developer of the advanced Conformance Test
  Architecture as well as the Conformance Test Suites for finger minutiae data
                             format implementations.

				
DOCUMENT INFO