Docstoc

DWP JNL SPRING 06

Document Sample
DWP JNL SPRING 06 Powered By Docstoc
					                                      TEST REVIEW: The Vienna Test System

Evaluation of tests from the Vienna Test System for Work Psychologist
Use within Jobcentre Plus

Published by: Schuhfried, Vienna. Layfayette Instrument Europe (Test Distributors for Schufried VTS),
4, Park Road, Sileby, Leics. LE12 7TJ

Reviewed by Giselle Parker and Nicola Oliver
North West Work Psychology Service, Jobcentre Plus, 52 Blackhouse Street, Bolton, BL1 1SX


ABSTRACT
This article reviews a number of tests selected from the Vienna               “With the advancement
Test System (VTS). With the advancement in technology
proposed in the Jobcentre Plus paper, entitled The Vision1                    in technology...
(Department for Work and Pensions, 2003), its customers will                  customers will
increasingly access resources via computer-based procedures.
The rationale behind this paper was to evaluate tests for
                                                                              increasingly access
possible addition to the work psychologist2 (WP) test battery.                resources via computer-
The findings of the feasibility and utility study can be used to              based procedures.”
inform a business case for further purchase of these tests.


INTRODUCTION
                                                                sessions are presented clearly to allow the
The VTS is developed by Schufried Test                          customer to understand how the test is to be
Instruments and is widely used throughout                       completed. Advantages work psychologists have
Europe in a number of settings. It consists of                  found over using the traditional pen and paper
powerful basic software, individual computer                    tests are numerous. Some of these are: the test
tests and of additional input devices that can                  results are automatically computed, therefore,
be added as required. The battery contains                      analysis of performance can be made quickly and
over 60 computer-aided tests, the majority of                   accurately; there is greater face validation;
which are tools designed for clinical or research               test manuals are built into the system, time
purposes. The tests are wide-ranging and                        measurements are down to milliseconds;
include assessments of Intelligence, personality,               and a database can be established for
special ability, attitude/interest and clinical                 statistical analysis.
tests. There is also a facility to design and                      This article reviews the relevance of a further
customise tests according to the needs of the                   twelve psychometric tests, featured within the
customer group.                                                 VTS and builds on the research by Whiteside
   Work psychologists working in the North West                 (2002) and Whiteside et al (2003).
Region of Jobcentre Plus already make use of the                   The following tests were selected from the
VTS as part of their employment assessments3                    overall test battery as the authors considered
with customers. These current tests largely focus               these to be the most related to the work of the
on the assessment of memory, attention,                         work psychologist in Jobcentre Plus. Lafayette
concentration, fatigue and distractibility. Generally           Instruments4 provided the tests for a trial period
customers using the VTS have experienced no                     and were evaluated by the authors only. There
significant difficulties even if they have little               was no customer input to the evaluation; the
existing computer skills. The input devices are                 research was entirely based on the authors’
‘user friendly’ and the instructions and practice               practical experience of using the tests:

                                   Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 169
             TEST REVIEW: The Vienna Test System


•   Two Hand Co-ordination                                             £360.86 and the N-test Alpha test is £423.23.
•   Double Labyrinth Test                                              Price reductions are offered, in that licences are
•   Sensomotor Co-ordination Test                                      available at lower cost through licence discounts
•   Determination Test                                                 when five systems or more are purchased.
•   Adaptive Spatial Ability Test                                      Additional start-up costs include the price of the
•   General Interest Structure Test                                    hardware, laptop or desktop computer and
•   Raven’s SPM (Standard Progressive Matrices)                        printer (prices averaging around £1,000
    (Raven, 1976)                                                      depending on the retailer).
•   Corsi-Block-Tapping Test
•   Gestalt Perception Test                                            PURPOSE OF THE SELECTED TESTS
•   Intelligence Structure Analysis
•   N-Test-Alpha                                                       The twelve tests selected assess different aspects
•   Perseveration Test                                                 of motor and cognitive functions from memory,
                                                                       manual dexterity, basic numerical skills,
FORMAT                                                                 distractibility to general cognitive ability.

The specification required for the VTS is a                            1. TWO-HAND CO-ORDINATION (2HAND)
desktop or laptop with Pentium compatible CPU
and Windows 98.                                                        Brief test description: This test assesses visual-
   All the tests selected for evaluation are                           motor co-ordination (eye-hand and hand-hand co-
designed to be self-administered. They fall into                       ordination). The user is required to use two hands
two categories of tests that allow the customer                        to move a symbol along a set track on a screen.
to work at their own comfortable pace or to a                          The layout of the track allows for a comparison
pre-determined period of time. Each test contains                      between the co-ordination performance for
a practice section which allows the individual to                      simple and more demanding tasks. The duration
acquaint themselves with the use of a mouse and                        of this test is approximately ten minutes.
PC. The tests appear user friendly, relatively short
to administer and each test contains a                                 Possible applications for WP use: This test
computerised manual, presenting theoretical and                        could be used to assess for suitability of jobs that
practical information for the administrator. For                       require a high degree of dual hand co-ordination /
each test a selection of norms are presented.                          dexterity, (e.g. fork-lift truck driving, crane
   Although all the tests in the VTS can be                            operator), for customers with conditions where
worked with a mouse and standard keyboard                              co-ordination may be affected (e.g. dyspraxia,
some of the tests are designed to be run with                          cerebral palsy, multiple sclerosis, repetitive strain
additional equipment. Extra devices consist of                         injury) or customers that wish to consider their
panels containing coloured and numbered input                          aptitude for new or alternative job options.
button sensors, a hook-up for a foot pedal and
light pens allowing answers to be input through                        Input devices required: Input panel
a touch screen monitor.
                                                                       2. DOUBLE LABYRINTH TEST (B19)
COST OF MATERIALS
                                                                       Brief test description: This test assesses visual-
All tests come with a seven-year licence and                           motor co-ordination. The speed of this test is
prices are competitive compared to pencil and                          fixed, therefore measuring eye-hand co-ordination
paper tests. For example, the Ravens SPM is                            ability in tasks of a pre-determined speed. Within
priced at £663.80, the Corsi Blocks test is                            this test the user is required to retain two circles


170 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006
                                        TEST REVIEW: The Vienna Test System


on a moving track by means of two control                       4. DETERMINATION TEST (DT)
knobs. The duration of this test is approximately
five minutes.                                                   Brief test description: This test measures stress
                                                                tolerance, attention deficits and reaction speed
Possible applications for WP use: This test                     in the presence of rapidly changing and
could be used to assess for suitability of jobs that            continuous optical and acoustical stimuli. The
require a high degree of co-ordination / dexterity              user is required to press the appropriate button
where accuracy under a pre-determined speed of                  on the control panel to record the correct
working is needed, e.g. assembly line work.                     reaction. The duration of this test is between 6 -
   The only other measure of this type within the               15 minutes.
existing WP test battery is the (Valpar5) Simulated
Assembly Test (Valpar, 1986). The Simulated                     Possible applications for WP use: There is little
Assembly Test can provide a useful measure of                   demand to measure the constructs of this test
speed of hand-eye co-ordination although the                    within a WP assessment (typically applications for
norm population is not current and industry                     this test are motor aptitude diagnosis and
standard measures have been updated. The B19                    neuropsychology). Attention deficits can be better
test could be used as an alternative to the                     assessed through use of other Vienna tests already
Simulated Assembly Test where flexibility of                    in use by the North West Work Psychology team
administration is required and where a                          such as ‘cognitrone’ and ‘continuous attention’.
computerised format is preferred.                               Therefore, this particular test would add little
                                                                value to the WP test battery that already contains
Input devices required: Input panel                             tests of attention.

3. SENSOMOTOR CO-ORDINATION TEST                                Input devices required: Input panel and
(SMK)                                                           foot pedal

Brief test description: This test assesses the co-              5. ADAPTIVE SPATIAL ABILITY TEST (A3DW)
ordination of eye-hand, hand-eye, or eye-hand-
foot. The user is required to manoeuvre a circular              Brief test description: This test assesses
segment that moves independently about a                        spatial perception. The user is presented with
three-dimensional room. The duration of this test               3-D blocks and asked to identify an identical
is from 10-20 minutes depending on which                        block presented from a different dimension /
version is selected.                                            angle. The duration of this test is between 10-
                                                                30 minutes.
Possible applications for WP use: This test is
extremely difficult to perform and measures                     Possible applications for WP use: A test
visual-motor co-ordination to a very high level of              already in use (the Differential Aptitude Test
accuracy and detail. Therefore, other more                      ‘Space Relations’ sub-test (PsychCorp, 1995))
fundamental measures of visual-motor co-                        providing the same assessment of spatial
ordination (e.g. the 2HAND co-ordination test                   perception is rarely used, as there is typically little
mentioned earlier) are likely to be suitable for the            demand to measure this construct within a WP
occupational psychologist’s use.                                assessment. Therefore, another test measuring
                                                                the same construct would be superfluous,
Input devices required: Input panel                             adding little value to the WP test battery.

                                                                Input devices required: Keyboard / Light
                                                                pen (optional)

                                   Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 171
           TEST REVIEW: The Vienna Test System

6. GENERAL INTEREST STRUCTURE TEST                                     8. CORSI-BLOCK-TAPPING TEST (CORSI)
(AIST)
                                                                       Brief test description: This test assesses the
Brief test description: This is a differential test                    capacity of the visual-spatial subsystem within
used to determine careers interests, measuring                         the short-term memory. The screen shows nine
congruence between a person and their                                  irregularly placed dice. A pointer moves from
environment. The duration of this test is                              one dice to another in a sequence that increases
approximately 20 minutes.                                              in number with each new item group. The user
                                                                       is required to reproduce the order the dice were
Possible applications for WP use: Tests already                        selected. The duration of this test is around ten
in use include the computer-based Adult                                minutes.
Directions (CASCAiD, n.d.) and paper-based
Occupational Interest Inventory (SHL, n.d.). These                     Possible applications for WP use: Although
provide a more detailed measure of career                              there is already a test of short-term visual-spatial
interest along with national salary estimates and                      memory within the existing WP test battery, it is
entry routes. On Adult Directions, customers can                       part of an in-depth test of memory. This test
also access up-to-date careers based information.                      could be used when the WP would like to make
Therefore, this particular test would add little                       a short initial assessment of visual-spatial
value to the WP test battery.                                          memory, for example, used as an initial
                                                                       screening test to assess if further investigation
Input devices required: Keyboard / Light                               into memory is required. Additionally, a
pen (optional)                                                         computerised version of this test allows for
                                                                       easier administration as the test administrator is
7. RAVEN’S STANDARD PROGRESSIVE                                        not required to memorise the sequence of
MATRICES (SPM)                                                         numbers to be assessed (required when using
                                                                       the Wechsler Memory Scales (Wechsler, 1998)),
Brief test description: This test provides a non-                      which can be somewhat difficult when the
verbal assessment of general intelligence. The user                    subject reaches seven or more digits.
is required to select the correct response to best
complete the pattern (or sequence of patterns)                         Test Limitations: Only tests forward sequence.
presented. The duration of this test is from 10-30                     Typically a further assessment of reverse
minutes depending on which version is selected.                        sequencing skills is sought to gain an in-depth
                                                                       analysis of working memory function.
Possible applications for WP use: The pencil                           Additionally, the Corsi only presents a raw score,
and paper versions of this test are used on a                          which does not allow for a norm group
frequent basis within the WP assessment process.                       comparison to be made.
Where a computerised format is preferred (i.e.
where additional assessment of basic computer                          Input devices required: Keyboard / Light
skills / competence is sought for customers that                       pen (optional)
wish to consider their aptitude for computer
related roles) this could make a contribution to                       9. GESTALT PERCEPTION TEST (GESTA)
the established WP test battery.
                                                                       Brief test description: This test makes an
Input devices required: Keyboard / Light                               assessment of cognitive style of field
pen (optional)                                                         (in)dependence by identifying a specific shape
                                                                       integrated into a pattern. Field dependant
                                                                       people are influenced by their perception of


172 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006
                                         TEST REVIEW: The Vienna Test System

surrounding stimuli so they always perceive taking               Possible applications for WP use: The only
into account the surrounding ‘field’. Field                      other measure of general intelligence within the
independent people are more likely to think laterally            current WP test battery is the WAIS III. Although
‘outside of the box’. The duration of this test is               this test may be considered as an alternative to
around 20 minutes.                                               the WAIS III test, the norms may not be as
                                                                 robust as for the WAIS III. Additional isolated
Possible applications for WP use: The closest                    measures of intelligence are not usually sought
assessment of lateral thinking within the current                within a typical WP assessment; therefore, its
WP test battery is within the Wechsler Adult                     uses in this context may be limited.
Intelligence Scale Third Edition (WAIS III) (Wechsler,
1997) – the similarities subtest. This test could be             Input devices required: Keyboard
used to make an isolated assessment of lateral
thinking, which may provide useful information                   11. N-TEST-ALPHA (NTA)
when used with customers where lateral thinking
may be affected, e.g. Asperger’s and General                     Brief test description: This looks at basic
Learning Difficulties. Information collected can be              numerical ability by assessing the four
used to assess suitability for certain occupations               functions of addition, subtraction,
(where there may be a high degree of problem                     multiplication and division. The duration of
solving / creative input required) and / or identify             this test is six minutes.
appropriate support.
   This test could possibly make a valuable                      Possible applications for WP use: This test
contribution to the established WP test battery.                 could simply replace the pencil and paper
However, there may be a limited demand to                        version of numerical functioning where a
measure the construct on this test within                        computerised format is preferred, i.e. where
everyday practice.                                               additional assessment of basic computer skills /
                                                                 competence is sought for customers that wish
Input devices required: Keyboard                                 to consider their aptitude for numerical data
                                                                 input, or similar roles.
10. INTELLIGENCE STRUCTURE ANALYSIS (ISA)
                                                                 Input devices required: Keyboard
Brief test description: The aim of this test is to
make a differentiated assessment of general                      12. PERSEVERATION TEST (PERSEV)
abilities at a medium abstraction level. The test
consists of nine item groups:                                    Brief test description: This test provides an
• Completing sentences                                           assessment of perseveration tendency, i.e. the
• Finding similarities                                           unwarranted repetition in cognitive processes
• Inferring relationships                                        and ways of behaviour. In other words,
• Generic terms (verbal intelligence functions)                  resistance to change and rigidity taking the
• Practical calculations                                         place of variability, flexibility and adaptability.
• Continuing numerical series                                    Within this test, nine big circles are displayed
• Recognising dice                                               and the task involves pressing the circles with a
• Assembling figures                                             light pen to the ‘beat’ of various tones. The
• Remembering product names                                      duration of this test is about five minutes.
   The duration of this test is around 55-110
minutes depending on which version of the test is                Possible applications for WP use: This test
selected.                                                        could assess the potential to do very repetitive
                                                                 jobs, e.g. assembly line work, especially in


                                    Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 173
             TEST REVIEW: The Vienna Test System

customers with possible attentional deficits, e.g.                       • Practice tests ensure client familiarity
Attention Deficit disorder (ADD) and General                             • Test manuals are built into the system
Learning Difficulties.                                                   • The tests are relatively inexpensive compared
                                                                           to pencil and paper tests
Input devices required: Input panel, light pen                           • Manual storing, re-ordering and general
& foot pedal                                                               paperwork is alleviated
                                                                         • Comprehension is ensured by means of
ADVANTAGES AND BENEFITS OF                                                 tracked instruction
USING THE VTS                                                            • Time measurements are down to milliseconds
                                                                         • There is a quick, error-free evaluation
The use of computerised tests within an                                    immediately following administration
assessment situation does have several                                   • A database can be established for
advantages:                                                                statistical analysis
• Tests within the VTS can replace the more                              • Systems can be networked
   antiquated Valpar test equipment, whose                               • All data is protected by password access
   norms have remained unchanged since                                   • Print out reports are easily generated.
   the 1970’s                                                              Additional results and automatic reporting can
• There is greater face validation for the                                 be configured into the system. There is also
   customer taking the test                                                room for additional comments.
• A detailed print out can be
   accessed immediately

Table 1: Summary - evaluation of tests for workstation psychology assessments

Tests                           Inclusion into existing   Feasibility for WP use
                                WP test battery
1. Two-Hand Co-ordination                                 Could replace Minnesota Test (Lafayette, 1995), which is seldom used due to
   (2HAND)
                                           √              its low face validity, its high norm comparisons and portability issues.
                                                           Could be used as an alternative to the Simulated Assembly Test where
2. Double Labyrinth Test
   (B19)
                                           √              flexibility of administration is required and where a computerised format
                                                          is preferred.
3. Sensomotor Coordination                                Measures visuo-motor co-ordination beyond the level required for
   Test (SMK)
                                           X              WP assessments.
                                                          There is little demand to measure the constructs of this test within a
4. Determination Test (DT)                 X               WP assessment.
5. Adaptive Spatial Ability                               There is little demand to measure the constructs of this test within a
   Test (A3DW)
                                           X              WP assessment.
6. General Interest Structure                             There are tests already in use i.e. Adult Directions / Occupational Interest
   Test (AIST)
                                           X              Inventory, that provide a more detailed measure of career interest.
7. Raven’s Standard                                       Could be used as an alternative to the pen and paper version where a
   Progressive Matrices (SPM)
                                           √              computerised format is preferred.
 8. Corsi-Block-Tapping Test                              This test could be used when the WP would like to make an initial assessment of
    (Corsi)
                                           √              visual-spatial memory to assess if further investigation into memory is required.
                                                          This test could be used to make an isolated assessment of lateral thinking,
 9. Gestalt Perception Test
    (GESTA)
                                           √              which may provide useful information when used with customers where lateral
                                                          thinking may be affected, i.e. Asperger’s and General Learning Difficulties.
10. Intelligence Structure                                Isolated measures of intelligence are not usually sought within a typical WP
    Analysis (ISA)
                                           X              assessment therefore its use in this context may be limited.
                                                          Could be used as an alternative to the pencil and paper version of the four
11. N-Test-Alpha (NTA)                     √              rules maths test.
12. Perseveration Test                                    Could be used as an alternative or in combination with the Simulated
    (PERSEV)
                                           √              Assembly test, (Valpar, 1986).


174 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006
                                           TEST REVIEW: The Vienna Test System

LIMITATIONS OF USING THE VTS                                      RESTRICTIONS ON USAGE
The majority of the tests identified may                          The tests are designed for use by ‘suitably qualified
necessitate a fixed desktop computer as the                       professionals’ including Occupational, Clinical and
control panel required to operate these tests is                  Educational Psychologists. No particular level of
somewhat bulky6 to transport, making the use                      qualification is quoted by the publishers, however
of these tests in a peripatetic mode unviable.                    the authors believe that a Level A BPS Certificate in
   Also several of these tests require using a                    Competence should be the appropriate baseline
light pen, which would need to be done in                         for usage (Schuhfried, Test publisher for the VTS,
conjunction with a desktop monitor.                               also gives training and certification to ensure that
                                                                  the correct level of competency has been reached).
RELIABILITY
                                                                  CONCLUSION
Depending on the test form and comparative
samples, co-efficients for ‘internal consistency’                 In conclusion, some of the tests reviewed may
(Cronbach’s Alpha) have been found between                        make a significant contribution to the work
0.77 and 0.96.                                                    psychologist’s existing test battery. For example,
                                                                  the tests assessing visual-motor co-ordination
VALIDITY                                                          (2HAND / B19 / PERSEV) could replace the more
                                                                  antiquated Valpar test equipment. Also the
A large number of studies for individual validity                 constructs assessed by the Raven’s Standard
concepts are available in the corresponding test                  Progressive Matrices, the Corsi-Block Tapping
manuals. Criterion-related validity, construct validity,          Test and the N-Test-Alpha are relevant to the WP
content validity, discriminant validity and extreme-              assessment and therefore could be considered to
group validity are all addressed in great detail, the             incorporate the benefits and advantages of
results of which indicate that the tests are valid                using a computerised format.
measurement tools.                                                    However, the utility of some of these tests, in
                                                                  a WP assessment, may be restricted to use
NORMS                                                             within an assessment site. For example, the tests
                                                                  identified above (2HAND / B19 / PERSEV) would
The norm groups vary according to the particular                  require the use of the additional input devices,
tests, in general, sample sizes are quoted between                i.e. input panel, foot pedal, and light pen, which
110 and 2820. Norm groups are taken from all                      would make it impossible to use these tests in a
over Europe, including Great Britain. Once again                  peripatetic mode. Although the option of using
dependent on the test, differing occupational and                 a light pen, on the Raven’s Standard Progressive
educational standards are noted, along with                       Matrices, the Corsi-Block Tapping Test and the
general samples and injury comparisons of samples                 N-Test-Alpha, would also not be viable in a
of customers with acquired brain injury.                          peripatetic mode (as this type of input requires
   Compared to the typical norm groups used in                    the use of a desktop monitor), in theory it may
the current WP test battery, a number of the norm                 be possible to load these specific tests remotely
groups provided for comparison in the VTS, such                   onto a desktop computer at a remote location.
as traffic psychology clients, mine workers and
employees from South Africa, could be considered
unrepresentative of the customers typically
assessed by the WP within Jobcentre Plus.




                                     Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006 ■ 175
            TEST REVIEW: The Vienna Test System


REFERENCES:
CASCAiD (n.d.) Adult Directions.                                       Wechsler D. (1998) Wechsler Memory Scales.
Loughborough: CASCAiD.                                                 San Antonio,TX: The Psychology Corporation.
                                                                       Wechsler D. (1997) Wechsler Adult Intelligence
Department for Work and Pensions. (2003)                               Scale - Third Edition. San Antonio, TX: The
Jobcentre Plus vision 2002-06. London: DWP.                            Psychology Corporation.

Lafayette (1995) Minnesota Manual Dexterity                            Whiteside, A. (2002) A synopsis of the Vienna
Test. Lafayette, IN: Lafayette Instruments.                            Test System: A computer aided psychological
                                                                       diagnosis. Journal of Occupational Psychology
PsychCorp (1995) Differential Aptitude                                 Employment and Disability. 5(1), 41-50.
Test ‘Space Relations’ Subtest. Antonio,
TX: PsycCorp.                                                          Whiteside, A., Parker, G., & Snodgrass, R.
                                                                       (2003). A review of selected tests from the
Raven, J.C. (1976). Standard Progressive                               Vienna Test System. Selection and Development
Matrices. Oxford: Oxford Psychologists Press.                          Review, 19(4), 7-11

SHL (n.d.) General Occupational Interest
Inventory. Thames Ditton: SHL.

Valpar (1986) Vapar component work sample
series: Simulated Assembly Test. Tucson, AZ:
Valpar Corporation.




 1
   This is a document outlining plans for Jobcentre Plus development up to 2006.
 2
   The title of ‘work’ psychologist is given to occupational psychologists within Jobcentre Plus.
 3
   The purpose of the employment assessment typically involves assessing how a person’s disability or health condition is
 likely to impact in work. The outcome is to suggest suitable support, aids or adaptations to help an individual sustain or
 retain work.
 4
   Lafayette Instruments are the UK distributor of the VTS
 5
   Valpar refers to a series of work sample tests designed to assess practical skills used in industry.
 6
   Each control unit is approximately 22 inches in length, 14 inches in diameter and 8lbs in weight




176 ■ Journal of Occupational Psychology, Employment and Disability, Vol 8, No.1, Spring 2006

				
DOCUMENT INFO
Shared By:
Categories:
Tags: SPRING
Stats:
views:43
posted:3/15/2010
language:English
pages:8
Description: DWP JNL SPRING 06