PSYCHOLOGICAL ASSESSMENT

Document Sample
PSYCHOLOGICAL ASSESSMENT Powered By Docstoc
					LECTURES 2008


                      Norms
   • Relative position
   • Instruments measuring same variable
    Differ in content
    Different scale units
    Composition of standardisation sample
   • National norms; specific norms; local norms
   • Fixed reference group
   • Item response theory

                                                   1
LECTURES 2008


                Definition of norms
   • Normative sample from population
   • Many characteristics are assumed to be normally
     distributed in the population
   • Standard normal distribution mean = 0, standard
     deviation = 1
   • Raw scores as such, not very informative – they can be
     transformed to norms to determine a respondent‟s
     „relative position‟
   • “A norm is a measurement against which the individual‟s
     raw score is evaluated so that the individual‟s position
     relative to that of the normative sample can be
     determined.”

                                                            2
LECTURES 2008



                  Types of norms
   • Developmental scales – some characteristics increase
     progressively with age and experience (vocabulary, job
     experience)
   • Mental age scales; determine basal age = highest age at
     and below which all items were passed; mental age =
     basal age + additional months of credit eared at higher
     age levels
   • Grade equivalents: educational achievement often
     interpreted i.t.o. grade equivalents – e.g. grade 10 math
     equivalent
   • Scale units not necessarily equal; represent median
     performance with overlapping between grades

                                                             3
LECTURES 2008



                    Percentiles
   • Refers to the percentage of people in a
     normative standardization sample who received
     scores lower than a specific raw score e.g. if an
     individual obtained a raw score of 32 and this
     score corresponds with a percentile score of 45
     (i.e. 32 has a percentile rank of 45), it means
     45% of the normative population received raw
     scores below 32
   • Disadvantages: inequality of scale units; ordinal
     level measures; percentile ranks of different
     variables and different populations not directly
     comparable

                                                         4
LECTURES 2008



                 Standards scores
   • z-scores = (X – Mean)/SD; positive z-scores – above
     average performance, negative below average –
     advantages: interval level measurement; frequency
     distribution of z-scores same as raw scores on which
     they based – disadvantages: half z-scores negative;
     limited ranges (about -3 to +3)
   • Linearly transformed standard scores – to eliminate
     disadvantages of z-scores – multiply z by constant A (to
     compensate for limited range) and add constant B (to
     eliminate negative scores): transformed ziT = ziA + B –
     disadvantages: do not look like z-scores and statistically
     more complex, less useful

                                                                  5
LECTURES 2008



       Normalised standard scores
   • Standard scores transformed to fit normal distribution – if attribute
     normal in population – advantage: assessment scores correspond to
     normal distribution; disadvantages same as z-scores and
     normalised scores change form of original raw score distribution
    McCall‟s T score: T = zN10 + 50
    Stanine scale: ranges from 1 to 9, mean = 5, SD = 1.96 –
     advantages: scale units equal; reflects individual‟s position relative
     to normative sample; performance in rank order is evident;
     comparable across groups; allow statistical manipulation – only 9
     scale units
    Sten scale: 10 scale units, mean = 5.5, SD = 2 – advantages and
     disadvantages similar to stanines
    Deviation IQ scale – used intelligence scales – mean = 100, SD =
     15 – easy to understand – used for age levels above 18 – not
     directly comparable to scales with different means, SDs


                                                                          6
LECTURES 2008

        Setting standards and cut-off
                   scores
   • Norm score gives individual‟s position relative to the
     norm group
   • Instead of comparison with a norm group, can compare
     with a standard/criterion score – e.g. 50% to pass an
     exam; x% to get a driver‟s licence
   • Cut-off scores on criterion determined by complex
     process – issues: legal, ethical, professional,
     psychometric, accepted policies
   • Can use expectancy tables (based on past experience,
     psychometric properties of measure) – advantage:
     provide way of interpreting relationship between
     assessment and probable success on criterion

                                                              7
LECTURES 2008



        Item response theory (IRT)
   • Also called latent trait models
   • Item analysis based on the probability that a
     person with a specified ability („latent trait‟)
     succeeds on an item of specified difficulty.
   • No implication that such latent traits/underlying
     abilities exist in a physical/physiological sense
     nor that it can cause behaviour – latent traits are
     statistical constructs derived mathematically
     from empirically observed relations among
     responses to test items
   • Models used to establish „sample-free‟ scale of
     measurement
                                                       8
LECTURES 2008



      Fairness and adverse impact
   • Four interpretations of fairness relate to equality
     i.t.o. being selected on the basis of one‟s
     predicted criterion performance (irrespective of
     group membership), equality in the outcomes of
     testing for different groups, equitable treatment
     in the testing process, and equality of
     opportunity to learn the material contained in the
     test.
   • Adverse impact when it discriminates against
     specific group so that members of the group
     have a smaller probability than other groups of
     being selected
                                                       9
LECTURES 2008


     Difference between bias and fairness
   • Predictive bias a statistical concept – can
     be addressed with statistical procedures or
     fixing instrument
   • Fairness in sense of equality in testing
     outcomes seen as socio-political issue that
     refers to proportions of the various
     subgroups selected (in USA coined term
     “adverse impact” which refers to
     disproportionate rate of selection of –
     usually minority - subgroups)               10
LECTURES 2008


                    Interpretation
   • If intended interpretation of assessment is shown to be
     valid and free of bias, and assessees have been treated
     fairly in assessment process, fairness has prevailed,
     regardless of whether instrument yields equal group
     results
   • Thus, if different selection rates are due solely to an
     unequal distribution of the construct in the relevant
     subgroups (i.e. the groups actually differ), and if the
     construct has a strong relationship with job performance,
     differential subgroup results do not imply unfairness – in
     SA may be a problem?


                                                              11
LECTURES 2008


                Assignment 2
   • Case study: Plant Manager at Dynamo
   (on website)

   • Due date: 25 August




                                           12