This study is based on Kirkpatrick's (1996) four level evaluation model. The study assessed the correlation between and among three levels of data that resulted from evaluation processes used in the U.S. Air Force technical training. The three levels of evaluation included trainee reaction (Level 1), test scores (Level 2), and job performance (Level 3). Level 1 data was obtained from the results of a 20 item survey that employed a 5-point Likert scale rating. Written test scores were used for Level 2 data. The Level 3 data was collected from supervisors of new graduates using a 5-point Likert scale survey. The study was conducted on an existing database of Air Force technical training graduates. The subjects were trainees that graduated since the process of collecting and storing Levels 1 and 2 data in computerized database began. All subjects for this study graduated between March 1997 and January 1999. A total of 188 graduates from five Air Force specialties were included. Thirty-four cases were from a single course in the aircrew protection specialty area; 12 were from a single course in the munitions and weapons specialty area; and 142 were from three separate courses in the manned aerospace maintenance specialty …

The problem under investigation in this research is the development of a general approach that will establish a students knowledge level so that the student's learning can be optimized by beginning it at the most effective point. In preparation for this study, an active test with an acceptable CT3 homogeneity index was found. Two computer programs, RightOrder and MathTest, were written in Visual Basic. The latter administers the test, producing a file or responses that serves as input for the former, which performs the calculations and matrix manipulation necessary to determine the CT3 of a set of test items and construct a difficulty strata scale. The test was administered twice to the same population, the first time in the original item order. In the second administration, one item from each successive level of difficulty, beginning with the easiest, was given until the respondent answered incorrectly. Then all the remaining items were presented in order of difficulty, beginning with the easiest. The three hypothesis of this study are (a) the difficulty strata scale generated from the computerized retest, using a z-score to be determined as critical value, is congruent with that derived from the analysis done on the data of the …