Power: Is It Mostly Hype?
Biometrics has been widely accepted as powerful tools in combating terrorism.
However, there is recent evidence that biometric systems are not nearly as powerful as the media indicates,
and certainly not as powerful as advertised by the companies making the products.
Last year's project teams investigated freeware and licensed biometric products, see
Investigation of Freeware Biometric Products and
Investigation of Two Licenced Biometric Products,
which were presented at Research Day 2011.
This semester we will investigate whether biometric systems are truly powerful or whether they are over hyped by the media
as well as by the creators of biometric products.
The project team will
We will focus on verification (not identification) for access control.
When requesting information, say that we would like to verify the identity of students entering certain university locations,
like a laboratory or computer classroom that needs access control to protect valuable equipment.
The key information (parameters) to obtain on a user-verification biometric product are
- investigate various sources - books, articles, internet, etc. - and summarize the information available
- check websites and request information from sales representatives of companies that manufacture biometric products to determine advertised accuracies of these systems
- later interview selected sales representatives and possibly technical experts from the companies that manufacture biometric products
to question advertised accuracies of these systems - for example, we have contacts at the companies we purchased equipment from
Accuracy (e.g., EER) will decrease as the population of users increases - for example, there is a huge difference between a class of 30 students
(e.g., taking online tests) and the roughly 300 million population of the United States.
Accuracy (e.g., EER) will also decrease as the quality of the data samples degrades -
for example, it can be difficult to identify someone from a partial or smudged fingerprint.
Biometric equipment manufacturers typically quote accuracy figures from controlled experiments on a small population of users and
clean high resolution biometric samples.
- system performance, usually the equal error rate (ERR) from the Receiver Operating Characteristic (ROC) curve
- the number of users handled
- the quality of the data samples required
It is anticipated that the final deliverable of this team will be an article
that exposes the media and manufacturer hype relating to these products.
Errors discovered over recent years have made it apparent that biometrics is not an exact science.
For example, in 2004 the FBI arrested Brandon Mayfield in connection with the Madrid terrorist attacks because
fingerprints on a bag containing detonating devices, found by Spanish authorities following the Madrid commuter train bombings,
were identified as belonging to him, and the fingerprints were "100% verified".
However, an FBI internal review recently acknowledged serious errors in their investigation,
and ensuing lawsuits have resulted in a formal apology from the U.S. government and a $2 million settlement,
see Wikipedia article.
An excellent freeware face recognition system that can be downloaded and tested is
You can also reexamine the face and fingerprint products for which we have licenses.
Frame Problem (FP) Approach
The research problem:
Using the Frame Problem knowledge from the AI domain,
discover a possible way to organize factors that may help improve the accuracy in contemporary Biometric Systems.
The work involves:
- Understanding the FP and some of its available solutions
- Creating a working taxonomy for common Biometric Systems
- Testing the available Biometric Systems using a computational model with FP
- Identifying a relationship between factors for accuracy of existing Biometric Systems
Frame Problem in the AI domain hints at the identification of unknown variables in the dynamic systems.
One approach to understand uncertain systems with non-perfect accuracy is by expressing these systems in terms of frame problem axioms.
First identified by John McCarthy and Patrick J. Hayes in1969,
frame problem can still be used as a framework to understand some specific situations
when accuracy of dynamic systems with many unidentified variables is still not confirmed,
as is the case in contemporary biometric systems.
Fast Agile XP Deliverables
We will use the agile methodology,
particularly Extreme Programming (XP) which involves small releases and fast turnarounds in roughly two-week iterations.
Many of these deliverables can be done in parallel by different members or subsets of the team.
The following is the current list of deliverables
(ordered by the date initiated,
initiated date marked in bold red if programming involved,
deliverable modifications marked in red,
completion date and related comments marked in green,
pseudo-code marked in blue):
Check websites and request information from sales representatives of companies that manufacture biometric products
to determine advertised accuracies of these systems.
For the user-verification access control problem, we will focus on
For each of the two biometrics, two cost levels, and two population sizes, we want to determine the expected system performance
and an indication of the quality of the data samples required
(e.g., high quality samples carefully taken, or more realistic lower quality of samples to be expected from students).
Also ask the sales reps for references - that is, contact information of customers who have purchased the product.
(This will allow you to contact these people to determine product problems and maybe actual product performance.)
The outcome of this deliverable should be a set of tables -
for example, for each of the two biometrics a table of the performance as a function of unit cost and population.
- two biometrics - face and fingerprint - but also ask about alternatives like voice, etc.
- two cost levels: low (up to $5,000) and high (no limit) per license or unit
- handling two population sizes: low (up to 100 students, like several sections of a course)
and high (up to 10,000 students, like the total student population of the university).
Do a literature survey on Frame Problem (FP) and how we may apply FP for biometric systems
We are investigating face and fingerprint biometrics this semester,
so meet with your customer Roshan Shaikh to select one of these to use for the FP approach this semester.