TEST REVIEW: The Vienna Test System Evaluation of tests from the Vienna Test System for Work Psychologist Use within Jobcentre Plus Published by: Schuhfried, Vienna. Layfayette Instrument Europe (Test Distributors for Schufried VTS), 4, Park Road, Sileby, Leics. LE12 7TJ Reviewed by Giselle Parker and Nicola Oliver North West Work Psychology Service, Jobcentre Plus, 52 Blackhouse Street, Bolton, BL1 1SX ABSTRACT This article reviews a number of tests selected from the Vienna “With the advancement Test System (VTS). With the advancement in technology proposed in the Jobcentre Plus paper, entitled The Vision1 in technology... (Department for Work and Pensions, 2003), its customers will customers will increasingly access resources via computer-based procedures. The rationale behind this paper was to evaluate tests for increasingly access possible addition to the work psychologist2 (WP) test battery. resources via computer- The findings of the feasibility and utility study can be used to based procedures.” inform a business case for further purchase of these tests. INTRODUCTION sessions are presented clearly to allow the The VTS is developed by Schufried Test customer to understand how the test is to be Instruments and is widely used throughout completed. Advantages work psychologists have Europe in a number of settings. It consists of found over using the traditional pen and paper powerful basic software, individual computer tests are numerous. Some of these are: the test tests and of additional input devices that can results are automatically computed, therefore, be added as required. The battery contains analysis of performance can be made quickly and over 60 computer-aided tests, the majority of accurately; there is greater face validation; which are tools designed for clinical or research test manuals are built into the system, time purposes. The tests are wide-ranging and measurements are down to milliseconds; include assessments of Intelligence, personality, and a database can be established for special ability, attitude/interest and clinical statistical analysis. tests. There is also a facility to design and This article reviews the relevance of a further customise tests according to the needs of the twelve psychometric tests, featured within the customer group. VTS and builds on the research by Whiteside Work psychologists working in the North West (2002) and Whiteside et al (2003). Region of Jobcentre Plus already make use of the The following tests were selected from the VTS as part of their employment assessments3 overall test battery as the authors considered with customers. These current tests largely focus these to be the most related to the work of the on the assessment of memory, attention, work psychologist in Jobcentre Plus. Lafayette concentration, fatigue and distractibility. Generally Instruments4 provided the tests for a trial period customers using the VTS have experienced no and were evaluated by the authors only. There significant difficulties even if they have little was no customer input to the evaluation; the existing computer skills. The input devices are research was entirely based on the authors’ ‘user friendly’ and the instructions and practice practical experience of using the tests: Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 169 TEST REVIEW: The Vienna Test System • Two Hand Co-ordination £360.86 and the N-test Alpha test is £423.23. • Double Labyrinth Test Price reductions are offered, in that licences are • Sensomotor Co-ordination Test available at lower cost through licence discounts • Determination Test when five systems or more are purchased. • Adaptive Spatial Ability Test Additional start-up costs include the price of the • General Interest Structure Test hardware, laptop or desktop computer and • Raven’s SPM (Standard Progressive Matrices) printer (prices averaging around £1,000 (Raven, 1976) depending on the retailer). • Corsi-Block-Tapping Test • Gestalt Perception Test PURPOSE OF THE SELECTED TESTS • Intelligence Structure Analysis • N-Test-Alpha The twelve tests selected assess different aspects • Perseveration Test of motor and cognitive functions from memory, manual dexterity, basic numerical skills, FORMAT distractibility to general cognitive ability. The specification required for the VTS is a 1. TWO-HAND CO-ORDINATION (2HAND) desktop or laptop with Pentium compatible CPU and Windows 98. Brief test description: This test assesses visual- All the tests selected for evaluation are motor co-ordination (eye-hand and hand-hand co- designed to be self-administered. They fall into ordination). The user is required to use two hands two categories of tests that allow the customer to move a symbol along a set track on a screen. to work at their own comfortable pace or to a The layout of the track allows for a comparison pre-determined period of time. Each test contains between the co-ordination performance for a practice section which allows the individual to simple and more demanding tasks. The duration acquaint themselves with the use of a mouse and of this test is approximately ten minutes. PC. The tests appear user friendly, relatively short to administer and each test contains a Possible applications for WP use: This test computerised manual, presenting theoretical and could be used to assess for suitability of jobs that practical information for the administrator. For require a high degree of dual hand co-ordination / each test a selection of norms are presented. dexterity, (e.g. fork-lift truck driving, crane Although all the tests in the VTS can be operator), for customers with conditions where worked with a mouse and standard keyboard co-ordination may be affected (e.g. dyspraxia, some of the tests are designed to be run with cerebral palsy, multiple sclerosis, repetitive strain additional equipment. Extra devices consist of injury) or customers that wish to consider their panels containing coloured and numbered input aptitude for new or alternative job options. button sensors, a hook-up for a foot pedal and light pens allowing answers to be input through Input devices required: Input panel a touch screen monitor. 2. DOUBLE LABYRINTH TEST (B19) COST OF MATERIALS Brief test description: This test assesses visual- All tests come with a seven-year licence and motor co-ordination. The speed of this test is prices are competitive compared to pencil and fixed, therefore measuring eye-hand co-ordination paper tests. For example, the Ravens SPM is ability in tasks of a pre-determined speed. Within priced at £663.80, the Corsi Blocks test is this test the user is required to retain two circles 170 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 TEST REVIEW: The Vienna Test System on a moving track by means of two control 4. DETERMINATION TEST (DT) knobs. The duration of this test is approximately five minutes. Brief test description: This test measures stress tolerance, attention deficits and reaction speed Possible applications for WP use: This test in the presence of rapidly changing and could be used to assess for suitability of jobs that continuous optical and acoustical stimuli. The require a high degree of co-ordination / dexterity user is required to press the appropriate button where accuracy under a pre-determined speed of on the control panel to record the correct working is needed, e.g. assembly line work. reaction. The duration of this test is between 6 - The only other measure of this type within the 15 minutes. existing WP test battery is the (Valpar5) Simulated Assembly Test (Valpar, 1986). The Simulated Possible applications for WP use: There is little Assembly Test can provide a useful measure of demand to measure the constructs of this test speed of hand-eye co-ordination although the within a WP assessment (typically applications for norm population is not current and industry this test are motor aptitude diagnosis and standard measures have been updated. The B19 neuropsychology). Attention deficits can be better test could be used as an alternative to the assessed through use of other Vienna tests already Simulated Assembly Test where flexibility of in use by the North West Work Psychology team administration is required and where a such as ‘cognitrone’ and ‘continuous attention’. computerised format is preferred. Therefore, this particular test would add little value to the WP test battery that already contains Input devices required: Input panel tests of attention. 3. SENSOMOTOR CO-ORDINATION TEST Input devices required: Input panel and (SMK) foot pedal Brief test description: This test assesses the co- 5. ADAPTIVE SPATIAL ABILITY TEST (A3DW) ordination of eye-hand, hand-eye, or eye-hand- foot. The user is required to manoeuvre a circular Brief test description: This test assesses segment that moves independently about a spatial perception. The user is presented with three-dimensional room. The duration of this test 3-D blocks and asked to identify an identical is from 10-20 minutes depending on which block presented from a different dimension / version is selected. angle. The duration of this test is between 10- 30 minutes. Possible applications for WP use: This test is extremely difficult to perform and measures Possible applications for WP use: A test visual-motor co-ordination to a very high level of already in use (the Differential Aptitude Test accuracy and detail. Therefore, other more ‘Space Relations’ sub-test (PsychCorp, 1995)) fundamental measures of visual-motor co- providing the same assessment of spatial ordination (e.g. the 2HAND co-ordination test perception is rarely used, as there is typically little mentioned earlier) are likely to be suitable for the demand to measure this construct within a WP occupational psychologist’s use. assessment. Therefore, another test measuring the same construct would be superfluous, Input devices required: Input panel adding little value to the WP test battery. Input devices required: Keyboard / Light pen (optional) Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 171 TEST REVIEW: The Vienna Test System 6. GENERAL INTEREST STRUCTURE TEST 8. CORSI-BLOCK-TAPPING TEST (CORSI) (AIST) Brief test description: This test assesses the Brief test description: This is a differential test capacity of the visual-spatial subsystem within used to determine careers interests, measuring the short-term memory. The screen shows nine congruence between a person and their irregularly placed dice. A pointer moves from environment. The duration of this test is one dice to another in a sequence that increases approximately 20 minutes. in number with each new item group. The user is required to reproduce the order the dice were Possible applications for WP use: Tests already selected. The duration of this test is around ten in use include the computer-based Adult minutes. Directions (CASCAiD, n.d.) and paper-based Occupational Interest Inventory (SHL, n.d.). These Possible applications for WP use: Although provide a more detailed measure of career there is already a test of short-term visual-spatial interest along with national salary estimates and memory within the existing WP test battery, it is entry routes. On Adult Directions, customers can part of an in-depth test of memory. This test also access up-to-date careers based information. could be used when the WP would like to make Therefore, this particular test would add little a short initial assessment of visual-spatial value to the WP test battery. memory, for example, used as an initial screening test to assess if further investigation Input devices required: Keyboard / Light into memory is required. Additionally, a pen (optional) computerised version of this test allows for easier administration as the test administrator is 7. RAVEN’S STANDARD PROGRESSIVE not required to memorise the sequence of MATRICES (SPM) numbers to be assessed (required when using the Wechsler Memory Scales (Wechsler, 1998)), Brief test description: This test provides a non- which can be somewhat difficult when the verbal assessment of general intelligence. The user subject reaches seven or more digits. is required to select the correct response to best complete the pattern (or sequence of patterns) Test Limitations: Only tests forward sequence. presented. The duration of this test is from 10-30 Typically a further assessment of reverse minutes depending on which version is selected. sequencing skills is sought to gain an in-depth analysis of working memory function. Possible applications for WP use: The pencil Additionally, the Corsi only presents a raw score, and paper versions of this test are used on a which does not allow for a norm group frequent basis within the WP assessment process. comparison to be made. Where a computerised format is preferred (i.e. where additional assessment of basic computer Input devices required: Keyboard / Light skills / competence is sought for customers that pen (optional) wish to consider their aptitude for computer related roles) this could make a contribution to 9. GESTALT PERCEPTION TEST (GESTA) the established WP test battery. Brief test description: This test makes an Input devices required: Keyboard / Light assessment of cognitive style of field pen (optional) (in)dependence by identifying a specific shape integrated into a pattern. Field dependant people are influenced by their perception of 172 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 TEST REVIEW: The Vienna Test System surrounding stimuli so they always perceive taking Possible applications for WP use: The only into account the surrounding ‘field’. Field other measure of general intelligence within the independent people are more likely to think laterally current WP test battery is the WAIS III. Although ‘outside of the box’. The duration of this test is this test may be considered as an alternative to around 20 minutes. the WAIS III test, the norms may not be as robust as for the WAIS III. Additional isolated Possible applications for WP use: The closest measures of intelligence are not usually sought assessment of lateral thinking within the current within a typical WP assessment; therefore, its WP test battery is within the Wechsler Adult uses in this context may be limited. Intelligence Scale Third Edition (WAIS III) (Wechsler, 1997) – the similarities subtest. This test could be Input devices required: Keyboard used to make an isolated assessment of lateral thinking, which may provide useful information 11. N-TEST-ALPHA (NTA) when used with customers where lateral thinking may be affected, e.g. Asperger’s and General Brief test description: This looks at basic Learning Difficulties. Information collected can be numerical ability by assessing the four used to assess suitability for certain occupations functions of addition, subtraction, (where there may be a high degree of problem multiplication and division. The duration of solving / creative input required) and / or identify this test is six minutes. appropriate support. This test could possibly make a valuable Possible applications for WP use: This test contribution to the established WP test battery. could simply replace the pencil and paper However, there may be a limited demand to version of numerical functioning where a measure the construct on this test within computerised format is preferred, i.e. where everyday practice. additional assessment of basic computer skills / competence is sought for customers that wish Input devices required: Keyboard to consider their aptitude for numerical data input, or similar roles. 10. INTELLIGENCE STRUCTURE ANALYSIS (ISA) Input devices required: Keyboard Brief test description: The aim of this test is to make a differentiated assessment of general 12. PERSEVERATION TEST (PERSEV) abilities at a medium abstraction level. The test consists of nine item groups: Brief test description: This test provides an • Completing sentences assessment of perseveration tendency, i.e. the • Finding similarities unwarranted repetition in cognitive processes • Inferring relationships and ways of behaviour. In other words, • Generic terms (verbal intelligence functions) resistance to change and rigidity taking the • Practical calculations place of variability, flexibility and adaptability. • Continuing numerical series Within this test, nine big circles are displayed • Recognising dice and the task involves pressing the circles with a • Assembling figures light pen to the ‘beat’ of various tones. The • Remembering product names duration of this test is about five minutes. The duration of this test is around 55-110 minutes depending on which version of the test is Possible applications for WP use: This test selected. could assess the potential to do very repetitive jobs, e.g. assembly line work, especially in Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 173 TEST REVIEW: The Vienna Test System customers with possible attentional deficits, e.g. • Practice tests ensure client familiarity Attention Deficit disorder (ADD) and General • Test manuals are built into the system Learning Difficulties. • The tests are relatively inexpensive compared to pencil and paper tests Input devices required: Input panel, light pen • Manual storing, re-ordering and general & foot pedal paperwork is alleviated • Comprehension is ensured by means of ADVANTAGES AND BENEFITS OF tracked instruction USING THE VTS • Time measurements are down to milliseconds • There is a quick, error-free evaluation The use of computerised tests within an immediately following administration assessment situation does have several • A database can be established for advantages: statistical analysis • Tests within the VTS can replace the more • Systems can be networked antiquated Valpar test equipment, whose • All data is protected by password access norms have remained unchanged since • Print out reports are easily generated. the 1970’s Additional results and automatic reporting can • There is greater face validation for the be configured into the system. There is also customer taking the test room for additional comments. • A detailed print out can be accessed immediately Table 1: Summary - evaluation of tests for workstation psychology assessments Tests Inclusion into existing Feasibility for WP use WP test battery 1. Two-Hand Co-ordination Could replace Minnesota Test (Lafayette, 1995), which is seldom used due to (2HAND) √ its low face validity, its high norm comparisons and portability issues. Could be used as an alternative to the Simulated Assembly Test where 2. Double Labyrinth Test (B19) √ flexibility of administration is required and where a computerised format is preferred. 3. Sensomotor Coordination Measures visuo-motor co-ordination beyond the level required for Test (SMK) X WP assessments. There is little demand to measure the constructs of this test within a 4. Determination Test (DT) X WP assessment. 5. Adaptive Spatial Ability There is little demand to measure the constructs of this test within a Test (A3DW) X WP assessment. 6. General Interest Structure There are tests already in use i.e. Adult Directions / Occupational Interest Test (AIST) X Inventory, that provide a more detailed measure of career interest. 7. Raven’s Standard Could be used as an alternative to the pen and paper version where a Progressive Matrices (SPM) √ computerised format is preferred. 8. Corsi-Block-Tapping Test This test could be used when the WP would like to make an initial assessment of (Corsi) √ visual-spatial memory to assess if further investigation into memory is required. This test could be used to make an isolated assessment of lateral thinking, 9. Gestalt Perception Test (GESTA) √ which may provide useful information when used with customers where lateral thinking may be affected, i.e. Asperger’s and General Learning Difficulties. 10. Intelligence Structure Isolated measures of intelligence are not usually sought within a typical WP Analysis (ISA) X assessment therefore its use in this context may be limited. Could be used as an alternative to the pencil and paper version of the four 11. N-Test-Alpha (NTA) √ rules maths test. 12. Perseveration Test Could be used as an alternative or in combination with the Simulated (PERSEV) √ Assembly test, (Valpar, 1986). 174 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 TEST REVIEW: The Vienna Test System LIMITATIONS OF USING THE VTS RESTRICTIONS ON USAGE The majority of the tests identified may The tests are designed for use by ‘suitably qualified necessitate a fixed desktop computer as the professionals’ including Occupational, Clinical and control panel required to operate these tests is Educational Psychologists. No particular level of somewhat bulky6 to transport, making the use qualification is quoted by the publishers, however of these tests in a peripatetic mode unviable. the authors believe that a Level A BPS Certificate in Also several of these tests require using a Competence should be the appropriate baseline light pen, which would need to be done in for usage (Schuhfried, Test publisher for the VTS, conjunction with a desktop monitor. also gives training and certification to ensure that the correct level of competency has been reached). RELIABILITY CONCLUSION Depending on the test form and comparative samples, co-efficients for ‘internal consistency’ In conclusion, some of the tests reviewed may (Cronbach’s Alpha) have been found between make a significant contribution to the work 0.77 and 0.96. psychologist’s existing test battery. For example, the tests assessing visual-motor co-ordination VALIDITY (2HAND / B19 / PERSEV) could replace the more antiquated Valpar test equipment. Also the A large number of studies for individual validity constructs assessed by the Raven’s Standard concepts are available in the corresponding test Progressive Matrices, the Corsi-Block Tapping manuals. Criterion-related validity, construct validity, Test and the N-Test-Alpha are relevant to the WP content validity, discriminant validity and extreme- assessment and therefore could be considered to group validity are all addressed in great detail, the incorporate the benefits and advantages of results of which indicate that the tests are valid using a computerised format. measurement tools. However, the utility of some of these tests, in a WP assessment, may be restricted to use NORMS within an assessment site. For example, the tests identified above (2HAND / B19 / PERSEV) would The norm groups vary according to the particular require the use of the additional input devices, tests, in general, sample sizes are quoted between i.e. input panel, foot pedal, and light pen, which 110 and 2820. Norm groups are taken from all would make it impossible to use these tests in a over Europe, including Great Britain. Once again peripatetic mode. Although the option of using dependent on the test, differing occupational and a light pen, on the Raven’s Standard Progressive educational standards are noted, along with Matrices, the Corsi-Block Tapping Test and the general samples and injury comparisons of samples N-Test-Alpha, would also not be viable in a of customers with acquired brain injury. peripatetic mode (as this type of input requires Compared to the typical norm groups used in the use of a desktop monitor), in theory it may the current WP test battery, a number of the norm be possible to load these specific tests remotely groups provided for comparison in the VTS, such onto a desktop computer at a remote location. as traffic psychology clients, mine workers and employees from South Africa, could be considered unrepresentative of the customers typically assessed by the WP within Jobcentre Plus. Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 175 TEST REVIEW: The Vienna Test System REFERENCES: CASCAiD (n.d.) Adult Directions. Wechsler D. (1998) Wechsler Memory Scales. Loughborough: CASCAiD. San Antonio,TX: The Psychology Corporation. Wechsler D. (1997) Wechsler Adult Intelligence Department for Work and Pensions. (2003) Scale - Third Edition. San Antonio, TX: The Jobcentre Plus vision 2002-06. London: DWP. Psychology Corporation. Lafayette (1995) Minnesota Manual Dexterity Whiteside, A. (2002) A synopsis of the Vienna Test. Lafayette, IN: Lafayette Instruments. Test System: A computer aided psychological diagnosis. Journal of Occupational Psychology PsychCorp (1995) Differential Aptitude Employment and Disability. 5(1), 41-50. Test ‘Space Relations’ Subtest. Antonio, TX: PsycCorp. Whiteside, A., Parker, G., & Snodgrass, R. (2003). A review of selected tests from the Raven, J.C. (1976). Standard Progressive Vienna Test System. Selection and Development Matrices. Oxford: Oxford Psychologists Press. Review, 19(4), 7-11 SHL (n.d.) General Occupational Interest Inventory. Thames Ditton: SHL. Valpar (1986) Vapar component work sample series: Simulated Assembly Test. Tucson, AZ: Valpar Corporation. 1 This is a document outlining plans for Jobcentre Plus development up to 2006. 2 The title of ‘work’ psychologist is given to occupational psychologists within Jobcentre Plus. 3 The purpose of the employment assessment typically involves assessing how a person’s disability or health condition is likely to impact in work. The outcome is to suggest suitable support, aids or adaptations to help an individual sustain or retain work. 4 Lafayette Instruments are the UK distributor of the VTS 5 Valpar refers to a series of work sample tests designed to assess practical skills used in industry. 6 Each control unit is approximately 22 inches in length, 14 inches in diameter and 8lbs in weight 176 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006
"DWP JNL SPRING 06"