ENLISTMENT SCREENING TEST _EST_ AND by chenmeixiu

VIEWS: 5,074 PAGES: 6

									               ENLISTMENT SCREENING TEST (EST) AND
            COMPUTERIZED ADAPTIVE SCREENING TEST (CAST)
                         by Jacobina Skinner, Ph.D.
                    Operational Technologies Corporation
                              San Antonio, TX
                                    2007

        The military services use screening tests to reduce enlistment processing costs.
Recruiters administer the screening tests locally, identify applicants who likely will meet
service mental qualifications, and arrange for them to travel to central Military Entrance
Processing Stations (MEPS) for additional testing. Transportation and boarding costs are
avoided for applicants whose probability of meeting entrance standards is extremely low.

       The traditional use of screening tests by recruiters in all military services has been
to predict the likelihood an applicant will meet or exceed the minimum Armed Forces
Qualification Test (AFQT) score required for enlistment eligibility. The AFQT is a
composite score derived from the Armed Services Vocational Aptitude Battery
(ASVAB). For many years, recruiters have relied on the Enlistment Screening Test
(EST) and/or the Computerized Adaptive Screening Test (CAST) for this purpose.

Enlistment Screening Test (EST)

         The Enlistment Screening Test (EST) is a paper-and-pencil instrument that is
hand-scored by recruiters. Updated versions of the EST were prepared when new
ASVAB forms were published with changes in subtest content (Morton, Houston, &
Bayroff, 1957; Jensen & Valentine, 1976; Mathews & Ree, 1981, 1982). As the former
lead agency for ASVAB development, the Air Force Human Resources Laboratory
(AFHRL) developed parallel versions of the EST for ASVAB Forms 5 and 6 (Jensen &
Valentine, 1976). The screening tests consisted of 30 items in each of the Word
Knowledge, Arithmetic Reasoning, and Space Perception content areas. The tests were
developed using data on applicants tested at a national representative sample of recruiting
offices. Test statistics demonstrated that applicants’ scores on the EST were highly
correlated with their later performance on the AFQT (r = .71). These ESTs became
obsolete with the implementation of ASVAB Forms 8, 9, 10 which did not contain Space
Perception content in the AFQT composite.

        The replacement ESTs were forms 81a and 81b, which were also developed by
AFHRL (Mathews & Ree, 1981, 1982). These parallel ESTs were shorter than previous
versions, containing 48 items total across the Arithmetic Reasoning, Word Knowledge,
and Paragraph Comprehension subtests which also appeared in the AFQT composite.
Items were selected to maximize measurement reliability at ability levels where most
selection decisions on applicants were made. Data obtained on applicants at about 300
recruiting stations nationwide showed their AFQT percentile scores were well predicted
by EST 81a and 81b. The correlation between EST 81a and AFQT was .83. Forms 81a
and 81b were used by recruiters for all military service branches for nearly 10 years. The
forms were in the field during the time that initial research efforts were being made to
develop a computer-adaptive screening test – the CAST. The EST 81a and 81b provided
the AFQT predictive accuracy standard against which early versions of the CAST were
compared.

         In the late 1980s the Marine Corps judged that for their applicant screening needs
the EST 81a and 81b were obsolete and contracted with the Center for Naval Analyses
(CNA) to design replacements (Divgi, 1990a, b). The development had two stages. In
the first stage, overlength forms, containing about 50 percent more items than needed in
the final forms, were constructed using items from discontinued versions of the ASVAB.
In the second stage, data on the overlength forms were used to select items for the final
forms. The goal was two parallel screening tests, each containing 65 items total across
Word Knowledge, Arithmetic Reasoning, and Math Knowledge content areas. The time
limit for test completion by applicants was 45 minutes.

        These ESTs, called Forms A and B, were first constructed for the Marine Corps
using data only on USMC applicants. As work for the Marine Corps was nearing
completion in 1988, the Defense Advisory Committee (DAC) on Military Personnel
Testing and the other services expressed interest in expanding CNA’s effort to construct a
joint-service screening test. Consequently, the Navy and Air Force collected additional
data on the overlength forms. Item selection for the final forms was based on the
correlation of the item score with the AFQT. Expectancy tables were produced which,
for any given EST score, provided the probability of exceeding AFQT percentile cutoffs
of interest to the services (Divgi, 1990b). The joint service EST Forms A and B, along
with the expectancy tables for AFQT, were printed and distributed to all services in
February 1989. These forms are currently authorized for use by Air Force recruiters
(AFPT Catalog, 1 June 2006).

        Air Force recruiters are also using a computerized Enlistment Screening Test
(Version 1.0), which was developed by Bill Hanson, Tallahassee, FL. The test is DOS-
based and consists of four parts: Word Knowledge (18 items), Arithmetic Reasoning (15
items), Paragraph Comprehension (8 items), and Math Knowledge (13 items). Total
administration time is 39 minutes. No report was located on this test. Information from
recruiters suggests the test may have been developed for the Navy.

Computerized Adaptive Screening Test (CAST)

       In the early 1980s, the Army funded the Navy Personnel Research and
Development Center (NPRDC) and the Army Research Institute (ARI) to construct a
computer-based screening test called the CAST (Knapp & Pliske, 1986; Sands, Gade, &
Knapp, 1997). The test items used were ones that had been calibrated in a related
research effort to develop a computerized adaptive version of the ASVAB. The purpose
of CAST was a quicker and easier screening test for recruiters to administer than the
paper-and-pencil EST 81a and 81b, which required hand-scoring and hand-conversion to
the AFQT metric.
        The pool of items for the early CAST was 78 Word Knowledge and 225
Arithmetic Reasoning multiple choice items. The stopping rule for computer-adaptive
test administration was 13 Word Knowledge items and 5 Arithmetic Reasoning items.
The tests are called adaptive, because computer software is used to tailor test difficulty to
the examinee’s ability by selecting items one at a time, contingent on the applicant’s
performance.      Adaptive tests typically achieve the measurement precision of
conventional, non-adaptive tests with half the number of items. The CAST score was a
weighted combination of item ability measures that resulted in an estimate of the
applicant’s AFQT percentile score. Three validation studies were completed. In the first
one, a sample of 312 Army applicants were tested in Los Angeles, and a correlation
between optimally weighted CAST subtest scores and the AFQT scores was .85. In a
second data collection for Army applicants in the Midwest, a correlation estimate of .80
was obtained. In the third effort, data were collected from a national sample of 60 Army
recruiting stations in 1985, and the simple bivariate correlation between CAST and
AFQT scores was .79. After correction for restriction in range, the correlation was .83
(Knapp & Pliske, 1986; Pliske, Gade, & Johnson, 1984; Sands & Rafacz, 1983). The
validation efforts revealed that CAST predicted AFQT at least as accurately as the EST,
was more efficient to use requiring less than 15 minutes to administer, reduced the
administrative burden on recruiters, and was less susceptible to test compromise. The
Army implemented the CAST in 1984 using the Joint Optical Information Network
(JOIN) (Sands, Gade, & Bryan, 1982; Sands & Rafacz, 1983; Johnson, Pliske, Weltin, &
Frieman, 1984).

        CAST underwent several revisions to improve its psychometric properties, as well
as modifications for use on a succession of microcomputer models (McBride & Cooper,
1999). Through CAST Version 4, the test was used only by Army recruiters. Version 5
was sponsored by the Joint Recruiting Information Support System (JRISS) for use by
recruiters of all of the armed services.

        Several upgrades were made to Version 5 to prepare the test for joint service use.
The accuracy of prediction of AFQT scores was improved by adjusting the test length.
Even with the resultant increase in test administration time, most examinees can complete
CAST Version 5 in about 25 minutes. The length of the Word Knowledge test was set to
15 items, and the length of the Arithmetic Reasoning test to 7 to 12 items. A variable test
length for Arithmetic Reasoning allowed improved prediction in critical score ranges for
the AFQT. Unlike the previous version of CAST, Version 5 has a time limit on the
Arithmetic Reasoning section (25 minutes), which was set in response to recruiters’
comments that some examinees had very long test times, largely due to slow work on the
math items. The test item bank contains 257 Word Knowledge items and 245 Arithmetic
Reasoning items. A random item selection process is used to select the next item for
examinees from among a set of five items. This change was made to reduce the number
of incidents of repeating sequences of the same test questions, a problem in Version 4.
Changes were also made in the software, system requirements, user interface, and the
score report prepared for recruiters. The User’s Guide and source code were published
by McBride and Cooper (1997). CAST Version 5 is authorized for use by Air Force
recruiters (AFPT Catalog, 1 June 2006).
Other Screening Tests

        The services have used other screening tests for decisions about various applicant
groups. An early example was the Women’s Enlistment Screening Test (WEST) used by
the Air Force. Its purpose was to obtain pre-enlistment information on women prior to
testing on the final selection tests (McReynolds, 1961; Dieterly, 1969). The Army
developed the Pre-enlistment Recruiting Test (PERT) to estimate aptitude area composite
scores for qualifying for particular Military Occupational Specialties (MOS) (Kass,
Weltin, Seeley, & Wing, 1981). In the Navy the Hispanic Enlistment Screening Test
(HEST) was constructed in Spanish as an aid for identifying Hispanic youths whose
enlistment scores would likely be acceptable after remedial English training (Mathews &
French, 1985).

                                       References

AFPT Catalog (1 Jun 2006). Air Force Personnel Tests Catalog. Randolph AFB, TX:
   HQ AFPC/DPPPWT.

Dieterly, D.L. (1969). The future of the Air Force military personnel testing system.
    Proceedings of the 11th Annual Conference of the Military Testing Association (p.
    164-168). New York, NY.

Divgi, D.R. (1990a). Development of overlength forms for a new Enlistment Screening
   Test (CRM 90-119, ADA 235732). Alexandria, VA: Center for Naval Analyses.

Divgi, D.R. (1990b). Construction of final forms for a new Enlistment Screening Test
   (CRM 90-120, ADA 235405). Alexandria, VA: Center for Naval Analyses.

Jensen, H.E., & Valentine, L.D., Jr. (1976). Development of the Enlistment Screening
    Test – EST Forms 5 and 6 (AFHRL-TR-76-42). Lackland AFB, TX: Personnel
    Research Division, Air Force Human Resources Laboratory.

Johnson, R.M., Pliske, R.M., Weltin, M.M., & Frieman, S.R. (1984). Joint Optical
    Information Network (JOIN) research at the Army Research Institute. Proceedings
    of the 26th Annual Conference of the Military Testing Association (p. 107-111).
    Munich, Federal Republic of Germany.

Kass, R.A., Weltin, M., Seeley, L., & Wing,H. (1981). Enlistment screening test to
   predict Army aptitude composite scores. Proceedings of the 23rd Annual Conference
   of the Military Testing Association (p. 613-627). Arlington, VA.

Knapp, D.J., & Pliske, R.M. (1986). An update on the Computerized Adaptive Screening
   Test (CAST). Proceedings of the 28th Annual Conference of the Military Testing
   Assocation (p. 1-6). New London, CT.
Mathews, J.J. & French, C.M. (1985). Relationship of an experimental Hispanic
   enlistment screening test to AFQT. Proceedings of the 27th Annual Conference of the
   Military Testing Association (p. 809-814). San Diego, CA.

Mathews, J.J., & Ree, M.J. (1982). Enlistment Screening Test Forms 81a and 81b:
   Development and calibration (AFHRL-TR-81-54). Brooks AFB, TX: Manpower
   and Personnel Research Division, Air Force Human Resources Laboratory.

Mathews, J.J., & Ree, M.J. (1981). Development and calibration of Enlistment Screening
   Test (EST) forms 81a and 81b. Proceedings of the 23rd Annual Conference of the
   Military Testing Association (p. 778-787). Arlington, VA.

McBride, J.R., & Cooper, R.R. (1999). Modification of the Computer Adaptive
   Screening Test (CAST) for use by recruiters in all military services (ARI Research
   Note 99-25, DTIC 1990 0427 026). Alexandria, VA: U. S. Army Research Institute
   for the Behavioral and Social Sciences. (Also published as HumRRO final technical
   report, FR-WATSD-97-24, September 1997).

McReynolds, J. (1961). Development of screening and selection tests for women (ASD-
   TN-61-54, AD0266865). Lackland AFB, TX: Aeronautical Systems Division, Air
   Force Systems Command.

Morton, M.A., Houston, T.J., & Bayroff, A.G. (1957). Development of Enlistment
   Screening Test, Forms 3 and 4 (ARI Technical Research Report 1102). Alexandria,
   VA: Army Research Institute.

Pliske, R.M., Gade, P.A., & Johnson, R.M. (1984). Cross-validation of the Computerized
    Adaptive Screening Test (CAST). Proceedings of the Annual Conference of the
    Military Testing Association (p. 335-338). Munich, Federal Republic of Germany.

Sands, W.A., Gade, P.A., & Bryan, J.D. (1982). Research and development for the
   JOIN system. Proceedings of the 24th Annual Conference of the Military Testing
   Association (p. 599-604). San Antonio, TX.

Sands, W.A., & Rafacz, B.A. (1983). Field test evaluation of the Computerized Adaptive
   Screening Test (CAST). Proceedings of the 25th Annual Conference of the Military
   Testing Association (p. 112-117. Gulf Shores, AL.

Sands, W.A., Gade, P.A., & Knapp, D.J. (1997). The Computerized Adaptive Screening
   Test. In Sands, W.A., Waters, B.K., & McBride, J.R. (Eds.) Computerized adaptive
   testing: From inquiry to operation. Washington, D.C.: American Psychological
   Association. (Also published as HumRRO FR-EADD-96-26.)


                                    Bibliography
Bayroff, A.G., Thomas, J.A., & Kehr, C.J. (1959). Evaluation of EST for predicting
   AFQT performance (ARI Technical Research Report 1114). Alexandria, VA: Army
   Research Institute.

Knapp, D.J. (1987). National cross-validation of the Computerized Adaptive Screening
   Test (CAST) (ARI-TR-768). Alexandria, VA: U.S. Army Research for the
   Behavioral and Social Sciences.

Knapp, D.J., & Pliske, R.M. (1986). Preliminary report on a national cross-validation of
   the Computerized Adaptive Screening Test (CAST) (ARI Research Report No. 1430).
   Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
   Sciences.

Moreno, K.E., Wetzel, C.D., McBride, J.R., & Weiss, D.J. (1983). Relationship between
   corresponding Armed Services Vocational Aptitude Battery (ASVAB) and
   computerized adaptive testing (CAT) subtests (NPRDC Report. No. 83-27, NTIS No.
   ADA 131683). San Diego, CA: Navy Personnel Research and Development Center.

Park, P.K., & Dunn, M.L. 1996). Compatibility evaluation and research on the
    Computerized Adaptive Screening Test (CAST). Final report: User and programming
    guide. Washington, D.C.: American Institutes for Research (NTIS No. AD-A293
    112).

Pliske, R.M., Gade, P.A., & Johnson, R.M. (1984).       Cross-validation of the
    Computerized Adaptive Screening Test (CAST) (ARI Research Report No. 1372).
    Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
    Sciences.

Sands, W.A., & Gade, P.A. (1983). An application of computerized adaptive testing in
   the U.S. Army recruiting. Journal of Computer-Based Instruction, 10, 87-89.

Segall, D.O., Moreno, K.E., Bloxom, B.M., & Hetter, R.D. (1997). Psychometric
   procedures for administering CAT-ASVAB. In Sands, W.A., Water, B.K., &
   McBride, J.R. (Eds.) Computerized adaptive testing: From inquiry to operation.
   Washington, D.C.: American Psychological Association.

Wise, L.L., McHenry, J.J., Chia, W.J., Szenas, P.L., & McBride, J.R. (1989). Refinement
   of the Computerized Adaptive Screening Test (Final Report, Contract No. MDA903-
   86-C-0373). Washington, D.C. American Institutes for Research.

								
To top