Informative Standards and their Relevance

There are a number of international standards that are relevant and useful with regards to Biometrics Performance Monitoring. See also the section Useful resources for other relevant national and international guidelines.

ISO/IEC 2382-14 Information technology – Vocabulary – Part 14: Reliability, maintainability, availability
Note: defines and explains general terms in Reliability, Maintainability, and Availability

ISO/IEC 19795 (all parts) Information technology – Biometric performance testing and reporting

  • Part 1: Principles and framework
    Note: among useful and relevant parts are 1) Terms and definitions 2) General biometric system including description of components, functions, and performance measures 3) factors influencing performance 4) fundamental performance metrics
  • Part 2: Testing  methodologies for technology and scenario evaluation (ISO/IEC 19795-2:2007)
    Note: among useful and relevant parts is what specification details (system information) to collect and log about the biometric system/product. Furthermore, Biometrics Performance Monitoring, BPM, tool can be customized and used for Technology and Scenario evaluation if it enables a test to claim conformance to the set of the specified clauses in the standard.
  • Part 3: Modality-specific testing (Technical Report) (ISO/IEC TR 19795-3:2007)
    Note: 1) When designing evaluation tests, this TR’s consideration of performance influencing factors for a given modality should give fast and straightforward hints in narrowing down the selection process (inclusion/exclusion) of which parameters/metrics to monitor based on a given modality 2) The Robustness test is presented as a useful method to determine the amount of change in performance (performance sensitiveness) as a function of change of a modality-specific influential factor. This, in Biometrics Performance Monitoring, seems to be an appropriate notion to define adaptive baseline performance level as a function of (triggered by) the change of an influential factor (e.g. temperature) for a certain biometric modality.
  • Part 4: Interoperability performance testing (ISO/IEC 19795-4:2008)
    Note: 1) normative requirement of ‘Measuring component failure’ (beside transactional error rates) by introducing and defining the properties ‘failure to process’ and ‘component-level failure rate’ 2) Examples given regarding component-level failure, their root-cause, and in what phases/functions they may occur.
  • Part 5: Access control scenario and grading scheme (ISO/IEC 19795-5:2011)
    Note: 1) Centered around access control applications (and truly inspirational for other biometric application types) this standard introduces one important aspect (which is a central topic in this Best Practices) that is to enable the customer to state its performance requirements on error rates and transaction times (Grade levels and their corresponding metrics). The framework establishes a grading scheme (level of performance with statistical significance) that can be applied when setting the corresponding metrics in both Service Level Agreements and Baseline Performance Levels of BPM tools (presented in this Best Practices). 2) Similar to Part 2, logging detailed information/specification of the biometric system is required.
  • Part 6: Testing Methodologies for Operational Evaluation (ISO/IEC FCD 19795-6.2, Target publication date: 2012-06-17)
    Note: among useful and relevant parts are 1) Terms and definitions 2) The highly relevant ‘Operational system monitoring’ is presented in an annex with one particular detail: to graph performance metrics as a function of time in order to detect potential abnormalities and tendencies 3) Due to unknown ground truth (identity claim), BPM is comparable to Operational Evaluation with regards to relevant performance measures which are presented in this FCD 4) Presented ‘Sub-goals of operational testing’ are highly relevant to this Best Practices; e.g. one can forward/input the obtained performance data from pilots to BPM programs when setting the corresponding metrics (benchmark levels) in both Service Level Agreements and Baseline Performance Levels of BPM tools (presented in this Best Practices). 5) logging detailed information/specification of the biometric system and relevant environmental data 6) ‘Non-mandatory performance metrics’ presented in an annex are useful for defining and measuring additional metrics in BPM programs.
  • Part 7: Testing of on-card biometric comparison algorithms (ISO/IEC 19795-7:2011)
    Note: In realm of BPM, what is relevant here is the absence (stated as outside of scope) of methods for evaluating the performance of IC cards readers along with ruggedness or durability of the card.

ISO/IEC 29197 Introductory element — Evaluation Methodology for Environmental Influence in Biometric Systems (ISO/IEC WD 29197, Target publication date: 2013-08-11)
Note: this is highly relevant for this Best Practices when it is published as a Standard (see also notes on Part 3: Modality-specific testing).

ISO/IEC 19784 (Parts 1, 2, 4), Information technology – Biometric application programming interface
Authors’ note: we need to study these standards in more details in order to identify their relevance to this Best Practices.

  • Part 1: BioAPI specification (ISO/IEC 19784-1:2006)
    Note: Back in 2003, when Optimum Biometric Labs initiated its work with BPM, it defined all biometric-related components (and any related subsystem component) as Units. Incidentally, this is exactly how Units (BioAPI Units) are defined in this standard (hardware or software or a combination of both e.g. Sensor, Archive, Matching algorithm, Processing algorithm). This notion helps to make this Best Practices more familiar to users and system integrators as BioAPI is commonly used in the industry.

ISO/IEC DIS 2382-37 Information technology — Vocabulary — Part 37: Harmonized biometric vocabulary (ISO/IEC DIS 2382-37, Target publication date: 2013-01-24)
Note: when published, this standard should be relevant with regards to the Terms and Definitions within its scope.

SP 500-288 – Rev 0 – Draft 2 (WS-BD is a specification describing how to expose a biometric sensor to various clients via web services.)
Note: Back in 2003, when Optimum Biometric Labs initiated its work with BPM, it chose to base the interface of its BPM tool on Web Services; predicting that biometrics devices and services will eventually choose the standard as interface. Three years later, in 2006, NIST initiated the Biometric Web Services (BWS) project. Thus, this standard initiative is an invaluable support for this Best Practices as web services (WSDL) is also the interface (API) between the biometric system and Biometrics Performance Monitoring, BPM, tool.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: