A Guide on the Evaluation of Latent Fingerprint Recognition Algorithms

This reliance means that it is important to independently assess the accuracy, fairness, and deployability of the technology. The National Institute of Standards and Technology (NIST) is one of the most important organizations conducting accuracy testing of this kind. NIST develops testing programs for biometric algorithm vendors around the world to participate in.

The idea of the testing is to avoid bias in performance claims by allowing all systems to be tested in the same conditions. This enables a comparison of technologies and enables the assessment of how these technologies will work, instead of just in perfect laboratory conditions. The toughest part of biometric tests is latent fingerprints, which are often partial, distorted, or only partially visible.

What NIST Tests

NIST tests algorithms against established benchmarks using established data sets. Tests are designed to assess the efficacy, speed, and error rates of various algorithms across systems. This allows researchers to compare the performance of different algorithms head-to-head, especially when systems are presented with low-quality data or difficult data.

Latent fingerprints are of interest because they are closer to the kind of images that we see in real forensic cases, where fingerprints are collected from surfaces at crime scenes. Those images are very rarely pristine, and it is challenging to accurately measure performance. Evaluation of Latent Fingerprint Technologies is one of our projects designed to evaluate algorithms on these criteria, as recently performed in the NIST ELFT evaluation.

How Results Are Shared

At completion of testing, NIST will release formal reports detailing the performance of each submission, as well as a wide array of comparative statistics and performance metrics that can be used by system developers and sponsoring agencies to compare with other sites.

The results can also be used for historical analyses, to gauge how much technology has improved over several years or more. Other examples of evaluations and historical metrics are available, which you can discover here.

Why Transparency Matters

Transparency is a key part of the evaluation process. Without third-party testing, claims about performance are likely to be based on the vendor’s perspective, which can change based on the methodological choices or limitations of the testing environment. By providing a uniform testing framework, NIST can ensure that all products are compared equally.

This process provides an advantage for both developers and the users who buy their solutions. Vendors receive impartial information about how well their products stack up against others in the market. Agencies and other organizations that procure these technologies are able to make better-informed decisions. And it fosters improvements, as published results highlight where improvements can be made to algorithms.

Real-World Relevance

Latent fingerprint recognition is a key biometric modality employed in law enforcement, immigration, social benefits programs, and identity verification. These applications often present fingerprint recognition algorithms with partial and degraded data, and hence, the performance of these algorithms should be assessed using realistic conditions.

The aim of NIST evaluations is to subject recognition algorithms to realistic conditions, ensuring that systems deployed at operational sites are both effective in operation and trustworthy. It is in this way that these evaluations assist in the deployment of biometric technology by building user trust and quantifying the level of performance expectation that can be reasonably expected in the field.