VIEWS: 5,074 PAGES: 6 POSTED ON: 7/3/2011
ENLISTMENT SCREENING TEST (EST) AND COMPUTERIZED ADAPTIVE SCREENING TEST (CAST) by Jacobina Skinner, Ph.D. Operational Technologies Corporation San Antonio, TX 2007 The military services use screening tests to reduce enlistment processing costs. Recruiters administer the screening tests locally, identify applicants who likely will meet service mental qualifications, and arrange for them to travel to central Military Entrance Processing Stations (MEPS) for additional testing. Transportation and boarding costs are avoided for applicants whose probability of meeting entrance standards is extremely low. The traditional use of screening tests by recruiters in all military services has been to predict the likelihood an applicant will meet or exceed the minimum Armed Forces Qualification Test (AFQT) score required for enlistment eligibility. The AFQT is a composite score derived from the Armed Services Vocational Aptitude Battery (ASVAB). For many years, recruiters have relied on the Enlistment Screening Test (EST) and/or the Computerized Adaptive Screening Test (CAST) for this purpose. Enlistment Screening Test (EST) The Enlistment Screening Test (EST) is a paper-and-pencil instrument that is hand-scored by recruiters. Updated versions of the EST were prepared when new ASVAB forms were published with changes in subtest content (Morton, Houston, & Bayroff, 1957; Jensen & Valentine, 1976; Mathews & Ree, 1981, 1982). As the former lead agency for ASVAB development, the Air Force Human Resources Laboratory (AFHRL) developed parallel versions of the EST for ASVAB Forms 5 and 6 (Jensen & Valentine, 1976). The screening tests consisted of 30 items in each of the Word Knowledge, Arithmetic Reasoning, and Space Perception content areas. The tests were developed using data on applicants tested at a national representative sample of recruiting offices. Test statistics demonstrated that applicants’ scores on the EST were highly correlated with their later performance on the AFQT (r = .71). These ESTs became obsolete with the implementation of ASVAB Forms 8, 9, 10 which did not contain Space Perception content in the AFQT composite. The replacement ESTs were forms 81a and 81b, which were also developed by AFHRL (Mathews & Ree, 1981, 1982). These parallel ESTs were shorter than previous versions, containing 48 items total across the Arithmetic Reasoning, Word Knowledge, and Paragraph Comprehension subtests which also appeared in the AFQT composite. Items were selected to maximize measurement reliability at ability levels where most selection decisions on applicants were made. Data obtained on applicants at about 300 recruiting stations nationwide showed their AFQT percentile scores were well predicted by EST 81a and 81b. The correlation between EST 81a and AFQT was .83. Forms 81a and 81b were used by recruiters for all military service branches for nearly 10 years. The forms were in the field during the time that initial research efforts were being made to develop a computer-adaptive screening test – the CAST. The EST 81a and 81b provided the AFQT predictive accuracy standard against which early versions of the CAST were compared. In the late 1980s the Marine Corps judged that for their applicant screening needs the EST 81a and 81b were obsolete and contracted with the Center for Naval Analyses (CNA) to design replacements (Divgi, 1990a, b). The development had two stages. In the first stage, overlength forms, containing about 50 percent more items than needed in the final forms, were constructed using items from discontinued versions of the ASVAB. In the second stage, data on the overlength forms were used to select items for the final forms. The goal was two parallel screening tests, each containing 65 items total across Word Knowledge, Arithmetic Reasoning, and Math Knowledge content areas. The time limit for test completion by applicants was 45 minutes. These ESTs, called Forms A and B, were first constructed for the Marine Corps using data only on USMC applicants. As work for the Marine Corps was nearing completion in 1988, the Defense Advisory Committee (DAC) on Military Personnel Testing and the other services expressed interest in expanding CNA’s effort to construct a joint-service screening test. Consequently, the Navy and Air Force collected additional data on the overlength forms. Item selection for the final forms was based on the correlation of the item score with the AFQT. Expectancy tables were produced which, for any given EST score, provided the probability of exceeding AFQT percentile cutoffs of interest to the services (Divgi, 1990b). The joint service EST Forms A and B, along with the expectancy tables for AFQT, were printed and distributed to all services in February 1989. These forms are currently authorized for use by Air Force recruiters (AFPT Catalog, 1 June 2006). Air Force recruiters are also using a computerized Enlistment Screening Test (Version 1.0), which was developed by Bill Hanson, Tallahassee, FL. The test is DOS- based and consists of four parts: Word Knowledge (18 items), Arithmetic Reasoning (15 items), Paragraph Comprehension (8 items), and Math Knowledge (13 items). Total administration time is 39 minutes. No report was located on this test. Information from recruiters suggests the test may have been developed for the Navy. Computerized Adaptive Screening Test (CAST) In the early 1980s, the Army funded the Navy Personnel Research and Development Center (NPRDC) and the Army Research Institute (ARI) to construct a computer-based screening test called the CAST (Knapp & Pliske, 1986; Sands, Gade, & Knapp, 1997). The test items used were ones that had been calibrated in a related research effort to develop a computerized adaptive version of the ASVAB. The purpose of CAST was a quicker and easier screening test for recruiters to administer than the paper-and-pencil EST 81a and 81b, which required hand-scoring and hand-conversion to the AFQT metric. The pool of items for the early CAST was 78 Word Knowledge and 225 Arithmetic Reasoning multiple choice items. The stopping rule for computer-adaptive test administration was 13 Word Knowledge items and 5 Arithmetic Reasoning items. The tests are called adaptive, because computer software is used to tailor test difficulty to the examinee’s ability by selecting items one at a time, contingent on the applicant’s performance. Adaptive tests typically achieve the measurement precision of conventional, non-adaptive tests with half the number of items. The CAST score was a weighted combination of item ability measures that resulted in an estimate of the applicant’s AFQT percentile score. Three validation studies were completed. In the first one, a sample of 312 Army applicants were tested in Los Angeles, and a correlation between optimally weighted CAST subtest scores and the AFQT scores was .85. In a second data collection for Army applicants in the Midwest, a correlation estimate of .80 was obtained. In the third effort, data were collected from a national sample of 60 Army recruiting stations in 1985, and the simple bivariate correlation between CAST and AFQT scores was .79. After correction for restriction in range, the correlation was .83 (Knapp & Pliske, 1986; Pliske, Gade, & Johnson, 1984; Sands & Rafacz, 1983). The validation efforts revealed that CAST predicted AFQT at least as accurately as the EST, was more efficient to use requiring less than 15 minutes to administer, reduced the administrative burden on recruiters, and was less susceptible to test compromise. The Army implemented the CAST in 1984 using the Joint Optical Information Network (JOIN) (Sands, Gade, & Bryan, 1982; Sands & Rafacz, 1983; Johnson, Pliske, Weltin, & Frieman, 1984). CAST underwent several revisions to improve its psychometric properties, as well as modifications for use on a succession of microcomputer models (McBride & Cooper, 1999). Through CAST Version 4, the test was used only by Army recruiters. Version 5 was sponsored by the Joint Recruiting Information Support System (JRISS) for use by recruiters of all of the armed services. Several upgrades were made to Version 5 to prepare the test for joint service use. The accuracy of prediction of AFQT scores was improved by adjusting the test length. Even with the resultant increase in test administration time, most examinees can complete CAST Version 5 in about 25 minutes. The length of the Word Knowledge test was set to 15 items, and the length of the Arithmetic Reasoning test to 7 to 12 items. A variable test length for Arithmetic Reasoning allowed improved prediction in critical score ranges for the AFQT. Unlike the previous version of CAST, Version 5 has a time limit on the Arithmetic Reasoning section (25 minutes), which was set in response to recruiters’ comments that some examinees had very long test times, largely due to slow work on the math items. The test item bank contains 257 Word Knowledge items and 245 Arithmetic Reasoning items. A random item selection process is used to select the next item for examinees from among a set of five items. This change was made to reduce the number of incidents of repeating sequences of the same test questions, a problem in Version 4. Changes were also made in the software, system requirements, user interface, and the score report prepared for recruiters. The User’s Guide and source code were published by McBride and Cooper (1997). CAST Version 5 is authorized for use by Air Force recruiters (AFPT Catalog, 1 June 2006). Other Screening Tests The services have used other screening tests for decisions about various applicant groups. An early example was the Women’s Enlistment Screening Test (WEST) used by the Air Force. Its purpose was to obtain pre-enlistment information on women prior to testing on the final selection tests (McReynolds, 1961; Dieterly, 1969). The Army developed the Pre-enlistment Recruiting Test (PERT) to estimate aptitude area composite scores for qualifying for particular Military Occupational Specialties (MOS) (Kass, Weltin, Seeley, & Wing, 1981). In the Navy the Hispanic Enlistment Screening Test (HEST) was constructed in Spanish as an aid for identifying Hispanic youths whose enlistment scores would likely be acceptable after remedial English training (Mathews & French, 1985). References AFPT Catalog (1 Jun 2006). Air Force Personnel Tests Catalog. Randolph AFB, TX: HQ AFPC/DPPPWT. Dieterly, D.L. (1969). The future of the Air Force military personnel testing system. Proceedings of the 11th Annual Conference of the Military Testing Association (p. 164-168). New York, NY. Divgi, D.R. (1990a). Development of overlength forms for a new Enlistment Screening Test (CRM 90-119, ADA 235732). Alexandria, VA: Center for Naval Analyses. Divgi, D.R. (1990b). Construction of final forms for a new Enlistment Screening Test (CRM 90-120, ADA 235405). Alexandria, VA: Center for Naval Analyses. Jensen, H.E., & Valentine, L.D., Jr. (1976). Development of the Enlistment Screening Test – EST Forms 5 and 6 (AFHRL-TR-76-42). Lackland AFB, TX: Personnel Research Division, Air Force Human Resources Laboratory. Johnson, R.M., Pliske, R.M., Weltin, M.M., & Frieman, S.R. (1984). Joint Optical Information Network (JOIN) research at the Army Research Institute. Proceedings of the 26th Annual Conference of the Military Testing Association (p. 107-111). Munich, Federal Republic of Germany. Kass, R.A., Weltin, M., Seeley, L., & Wing,H. (1981). Enlistment screening test to predict Army aptitude composite scores. Proceedings of the 23rd Annual Conference of the Military Testing Association (p. 613-627). Arlington, VA. Knapp, D.J., & Pliske, R.M. (1986). An update on the Computerized Adaptive Screening Test (CAST). Proceedings of the 28th Annual Conference of the Military Testing Assocation (p. 1-6). New London, CT. Mathews, J.J. & French, C.M. (1985). Relationship of an experimental Hispanic enlistment screening test to AFQT. Proceedings of the 27th Annual Conference of the Military Testing Association (p. 809-814). San Diego, CA. Mathews, J.J., & Ree, M.J. (1982). Enlistment Screening Test Forms 81a and 81b: Development and calibration (AFHRL-TR-81-54). Brooks AFB, TX: Manpower and Personnel Research Division, Air Force Human Resources Laboratory. Mathews, J.J., & Ree, M.J. (1981). Development and calibration of Enlistment Screening Test (EST) forms 81a and 81b. Proceedings of the 23rd Annual Conference of the Military Testing Association (p. 778-787). Arlington, VA. McBride, J.R., & Cooper, R.R. (1999). Modification of the Computer Adaptive Screening Test (CAST) for use by recruiters in all military services (ARI Research Note 99-25, DTIC 1990 0427 026). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. (Also published as HumRRO final technical report, FR-WATSD-97-24, September 1997). McReynolds, J. (1961). Development of screening and selection tests for women (ASD- TN-61-54, AD0266865). Lackland AFB, TX: Aeronautical Systems Division, Air Force Systems Command. Morton, M.A., Houston, T.J., & Bayroff, A.G. (1957). Development of Enlistment Screening Test, Forms 3 and 4 (ARI Technical Research Report 1102). Alexandria, VA: Army Research Institute. Pliske, R.M., Gade, P.A., & Johnson, R.M. (1984). Cross-validation of the Computerized Adaptive Screening Test (CAST). Proceedings of the Annual Conference of the Military Testing Association (p. 335-338). Munich, Federal Republic of Germany. Sands, W.A., Gade, P.A., & Bryan, J.D. (1982). Research and development for the JOIN system. Proceedings of the 24th Annual Conference of the Military Testing Association (p. 599-604). San Antonio, TX. Sands, W.A., & Rafacz, B.A. (1983). Field test evaluation of the Computerized Adaptive Screening Test (CAST). Proceedings of the 25th Annual Conference of the Military Testing Association (p. 112-117. Gulf Shores, AL. Sands, W.A., Gade, P.A., & Knapp, D.J. (1997). The Computerized Adaptive Screening Test. In Sands, W.A., Waters, B.K., & McBride, J.R. (Eds.) Computerized adaptive testing: From inquiry to operation. Washington, D.C.: American Psychological Association. (Also published as HumRRO FR-EADD-96-26.) Bibliography Bayroff, A.G., Thomas, J.A., & Kehr, C.J. (1959). Evaluation of EST for predicting AFQT performance (ARI Technical Research Report 1114). Alexandria, VA: Army Research Institute. Knapp, D.J. (1987). National cross-validation of the Computerized Adaptive Screening Test (CAST) (ARI-TR-768). Alexandria, VA: U.S. Army Research for the Behavioral and Social Sciences. Knapp, D.J., & Pliske, R.M. (1986). Preliminary report on a national cross-validation of the Computerized Adaptive Screening Test (CAST) (ARI Research Report No. 1430). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. Moreno, K.E., Wetzel, C.D., McBride, J.R., & Weiss, D.J. (1983). Relationship between corresponding Armed Services Vocational Aptitude Battery (ASVAB) and computerized adaptive testing (CAT) subtests (NPRDC Report. No. 83-27, NTIS No. ADA 131683). San Diego, CA: Navy Personnel Research and Development Center. Park, P.K., & Dunn, M.L. 1996). Compatibility evaluation and research on the Computerized Adaptive Screening Test (CAST). Final report: User and programming guide. Washington, D.C.: American Institutes for Research (NTIS No. AD-A293 112). Pliske, R.M., Gade, P.A., & Johnson, R.M. (1984). Cross-validation of the Computerized Adaptive Screening Test (CAST) (ARI Research Report No. 1372). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. Sands, W.A., & Gade, P.A. (1983). An application of computerized adaptive testing in the U.S. Army recruiting. Journal of Computer-Based Instruction, 10, 87-89. Segall, D.O., Moreno, K.E., Bloxom, B.M., & Hetter, R.D. (1997). Psychometric procedures for administering CAT-ASVAB. In Sands, W.A., Water, B.K., & McBride, J.R. (Eds.) Computerized adaptive testing: From inquiry to operation. Washington, D.C.: American Psychological Association. Wise, L.L., McHenry, J.J., Chia, W.J., Szenas, P.L., & McBride, J.R. (1989). Refinement of the Computerized Adaptive Screening Test (Final Report, Contract No. MDA903- 86-C-0373). Washington, D.C. American Institutes for Research.
Pages to are hidden for
"ENLISTMENT SCREENING TEST _EST_ AND"Please download to view full document