Keystroke Biometric: Test-Taker Setup and Data Collection

Keystroke Biometric Background: read & understand this section

Last semester's project worked to develop procedures and supporting code to perform actual test-taker authentication experiments, see Fall 2009 Technical Report, and this semester's project will continue that effort.

Project

Some programming might be necessary.

Test-Taker Experiments

This semester we will continue to develop a test-taker authentication application that uses the current authentication system.

Your first task is to review what was done last semester and determine whether it is sufficient to run experiments.

Data Collection

Continue to take new data, this semester from all members of the keystroke teams.

Fast Agile XP Deliverables

We will use the agile methodology, particularly Extreme Programming (XP) which involves small releases and fast turnarounds in roughly two-week iterations. Some of these deliverables might be done in parallel by different members or subsets of the team. The following is the current list of deliverables (ordered by the date initiated, deliverable modifications marked in red, deliverable date marked in bold red if programming involved, completion date and related comments marked in green, pseudo-code marked in blue):
  1. 2/1- Keystroke Deliverable 1 Instructions
  2. 2/1- Keystroke Deliverable 2 Instructions
  3. 2/1- Keystroke Deliverable 3 Instructions
  4. 2/3- Data Collection
    For experimental purposes we need keystroke data samples over time at two-week intervals. Each team member is to record five keystroke samples, in alternate weeks over a nine week interval, with data sets collected in Week 4 (Feb 11), Week 6 (Feb 25), Week 8 (Mar 11), Week 10 (Mar 25), and Week 11 (Apr 8, after Spring Break). Thus, each team member will record a total of 25 data samples (5 samples at each of 5 recording times). These data are to be collected using the existing data collection method. Obtain details about using the existing data collection method from your customer Robert Zack. Your team (Team 4: Test-Taker Setup & Data Collection) is in charge of this operation and should:
  5. 2/9- Raw Data Inventory
    Conduct an inventory of the keystroke raw data files. A user, or subject, is a person entering data. A set of samples is a group of samples by the same user produced at roughly the same time, usually at one sitting but certainly within a few days. Let Setn be a set of n samples, for example Set5 is a set of 5 samples. We are interested in counting sample sets. A large quantity of data was collected in the 2006 time frame, called early data. More recently, since 2008, we have been collecting data over time for longitudinal studies, called recent data, mainly sequences of sets at two week intervals. If 3 sets of 5 samples each were recorded at 2-week intervals, we would call these sets Set5-Week1, Set5-Week3, Set5-Week5. Data are taken with regard to two independent variables: keyboard type (desktop keyboard or laptop keyboard), and input mode (copy task or free text input). This results in four keyboard-mode quadrants: Desktop Copy, Laptop Copy, Desktop Free, and Laptop Free. Because our current interest is authenticating online test takers, we are mainly interested in the Free-text mode. Separately count the early data and the recent data sets. For the Desktop Free and Laptop Free quadrants, list the users by name and specify the sets.
  6. 2/17- Modify the feature extractor program (DELIVERABLE CANCELLED, given to Team 5)
    Background:
    The current keystroke biometric system was developed for running experiments and is not appropriate for deployment in an actual test-taker authentication application. The feature extraction program removes outliers and standardizes the feature measurements into the range 0-1. These procedures perform best on large quantities of data involving many subjects because we want the feature data for an individual subject to be unique (within the various 0-1 feature ranges) relative to that of other subjects. It is not reasonable, therefore, to extract features on a small amount of data, such as a single sample from a user needing authentication, without mixing it with data from many other subjects. However, it would be possible, though awkward and extremely inefficient, to take a sample from an individual user, mix it with many samples from other users (such as the large training sample data), extract features from the large mixture of samples, separate the one-user feature vector resulting from the mixture, and finally submit that feature vector to the classifier for authentication. The important thing is to use standardization values from a large body of data and to use the same values for both training and testing. For this reason, for experimental purposes we usually run the feature extractor on all the data and then split the data into training and testing data.
    Modification:
    To verify one student at a time we will create an efficient authentication process to compare a single new sample against one or more enrollment samples. Feature extraction for the training sample will be as before on a large quantity of data over many users. The change is that the feature extractor will output the standardization x-min and x-max values for each feature. Then, when computing the feature values for a student's new data, these recorded x-min and x-max values will be read in and used to perform the standardization. In summary, the revised feature extractor will have the capability of either inputting and then using, or computing and then outputting, a file of x-min and x-max values.
    System Operation:
    Extract features on a large body of training data and output the x-min/x-max file. Input the x-min/x-max file when extracting features on a sample from a user to be authenticated, and send the resulting feature file together with an enrollment feature file to the classifier for authentication.
  7. 3/12 Updated 4/7- Last semester's team developed a procedure for a professor to create test questions which would then be presented to online student test-takers inside the Java applet in stealth mode -- that is, so they would not realize their keystrokes were being captured. Your task is to check that procedure to determine whether it works, and if it does, inform your customer John Stewart so he can use it with his students. You should also comment on the procedure's elegance, ease of use, etc., and indicate how it can be improved. Finally, if time permits, implement the suggested improvements.
    Work closely with John Stewart to ensure successful completion of simulated student testing of the application, but using actual students in a normal course environment. (You must determine where the data is being collected so we can find it.) The test, which at this point will be a batch-processed simulated test, will involve the following.
    • Testing Data: collect a total of 5 data samples (simulating both enrollment and testing) from each of 10-30 students from one of John's courses (actual students in a course environment).
    • Training Data: use an existing set of data such as the 36-subject data (the 18+18 used for training and testing in deliverables 1-3).
    • Run a weak-training (different users for training and testing) experiment
      • Run feature extraction on all the data, both training and testing, then separate the output feature vector file into two files for training and testing
      • Run the BAS authentication program on the training and test files to obtain performance results. With 5 data samples per subject, this simulates 10 (5*4/2, all the possible pairs) authentication tests per subject to give us a reasonable number for obtaining performance statistics.
  8. 3/25-4/3 DELIVERABLE CANCELLED, completed by Robert Zack.
    We need the following ASAP so all three keystroke teams can run the experiment described in the next deliverable. Find 18 subjects who have entered 10 keystroke data samples (all freetext on same machine type of desktop or laptop), preferably 5 at one time and the 5 at a later time (like two weeks later). Create two raw-data files: the first file contains the 5 earlier-entered data samples from each of the 18 subjects, and the second contains the 5 later-entered data samples from each of the 18 subjects. Now, concatenate the two files to obtain one file with the earlier-entered data coming earlier in the file.
  9. 3/25-4/3 DELIVERABLE CANCELLED, completed by Robert Zack.
    Run the feature extraction program on the raw data file from the previous deliverable to obtain a feature file, manually split the feature file into a training file (18 subjects, first 5 feature samples from each subject) and a test file (18 subjects, second 5 feature samples from each subject), and run BAS on these training and test files to obtain results similar to the common deliverables at the beginning of the semester.
    Note:
    The experiment at the beginning of the semester used different subjects for training and testing (we call that "weak" training) and this experiment uses the same subjects (but different data samples) for training and testing (we call this "strong" training).
    [Team 5 should run this experiment with both the old and the improved (real 239 features) feature extraction program.]