Biometric Discrimination
Power: Is It Mostly Hype?

Background

Biometrics has been widely accepted as powerful tools in combating terrorism. However, there is recent evidence that biometric systems are not nearly as powerful as the media indicates, and certainly not as powerful as advertised by the companies making the products.

Last year's project teams investigated freeware and licensed biometric products, see Investigation of Freeware Biometric Products and Investigation of Two Licenced Biometric Products, which were presented at Research Day 2011.

Project

General Approach

This semester we will investigate whether biometric systems are truly powerful or whether they are over hyped by the media as well as by the creators of biometric products.

The project team will

We will focus on verification (not identification) for access control. When requesting information, say that we would like to verify the identity of students entering certain university locations, like a laboratory or computer classroom that needs access control to protect valuable equipment. The key information (parameters) to obtain on a user-verification biometric product are Accuracy (e.g., EER) will decrease as the population of users increases - for example, there is a huge difference between a class of 30 students (e.g., taking online tests) and the roughly 300 million population of the United States. Accuracy (e.g., EER) will also decrease as the quality of the data samples degrades - for example, it can be difficult to identify someone from a partial or smudged fingerprint. Biometric equipment manufacturers typically quote accuracy figures from controlled experiments on a small population of users and clean high resolution biometric samples.

It is anticipated that the final deliverable of this team will be an article that exposes the media and manufacturer hype relating to these products.

Errors discovered over recent years have made it apparent that biometrics is not an exact science. For example, in 2004 the FBI arrested Brandon Mayfield in connection with the Madrid terrorist attacks because fingerprints on a bag containing detonating devices, found by Spanish authorities following the Madrid commuter train bombings, were identified as belonging to him, and the fingerprints were "100% verified". However, an FBI internal review recently acknowledged serious errors in their investigation, and ensuing lawsuits have resulted in a formal apology from the U.S. government and a $2 million settlement, see Wikipedia article.

An excellent freeware face recognition system that can be downloaded and tested is FaceL:_Facile_Face_Labeling. You can also reexamine the face and fingerprint products for which we have licenses.

Frame Problem (FP) Approach

The research problem: Using the Frame Problem knowledge from the AI domain, discover a possible way to organize factors that may help improve the accuracy in contemporary Biometric Systems.

The work involves:

Background Info: Frame Problem in the AI domain hints at the identification of unknown variables in the dynamic systems. One approach to understand uncertain systems with non-perfect accuracy is by expressing these systems in terms of frame problem axioms. First identified by John McCarthy and Patrick J. Hayes in1969, frame problem can still be used as a framework to understand some specific situations when accuracy of dynamic systems with many unidentified variables is still not confirmed, as is the case in contemporary biometric systems.

Fast Agile XP Deliverables

We will use the agile methodology, particularly Extreme Programming (XP) which involves small releases and fast turnarounds in roughly two-week iterations. Many of these deliverables can be done in parallel by different members or subsets of the team.

The following is the current list of deliverables (ordered by the date initiated, initiated date marked in bold red if programming involved, deliverable modifications marked in red, completion date and related comments marked in green, pseudo-code marked in blue):

  1. 9/24 Check websites and request information from sales representatives of companies that manufacture biometric products to determine advertised accuracies of these systems. For the user-verification access control problem, we will focus on For each of the two biometrics, two cost levels, and two population sizes, we want to determine the expected system performance and an indication of the quality of the data samples required (e.g., high quality samples carefully taken, or more realistic lower quality of samples to be expected from students). Also ask the sales reps for references - that is, contact information of customers who have purchased the product. (This will allow you to contact these people to determine product problems and maybe actual product performance.) The outcome of this deliverable should be a set of tables - for example, for each of the two biometrics a table of the performance as a function of unit cost and population.
  2. 9/27 Do a literature survey on Frame Problem (FP) and how we may apply FP for biometric systems We are investigating face and fingerprint biometrics this semester, so meet with your customer Roshan Shaikh to select one of these to use for the FP approach this semester.