Assessment and Testing Skills For Recruiters

Title INTRODUCTION TO ASSESSMENTS For Recruiters and Recruitment Managers INDEX Index Part I- Introduction to Merittrac Part II- Meaning and Purpose of Assessments Part III- Assessment Tests for Recruitment Part IV- Challenges in using Assessment Tests Part II PART II: Meaning and Purpose of Assessments MEANING OF ASSESSMENT Introduction to Assessments MEASUREMENT EVALUATIONS ASSESSMENTS  Assigning numerical or letter grade values to traits measured  Use of standard instruments  On-going process of measuring progress against expected outcomes  Summative and / or final measurement of competences against established criteria • E.g. Measurement of the temperature of the room by using a thermometer  Relative and not absolute  Formative  Diagnostic  Specific  Usually Goal-Directed • Summative • Final  Cannot draw Inferences • Holistic • Usually Content-Driven BENEFITS OF ASSESSMENT Benefits of Assessment On the Personal Front Higher Work Motivation Low Absenteeism & Turnover Better Person Job Fit High Quality of Work Performance High Job Satisfaction & Commitment BENEFITS OF ASSESSMENT Benefits of Assessment On the Operational Front Accurate Selection  Rejection of potential bad recruit Fair Selection  Fair & standardized selection across geographies Operational Impact Consistent Selection  Same metric across geography across time Efficient Selection  Reduction in time, cost and effort RISK OF A WRONG HIRE Risk of A Wrong Hire Potential Pitfalls due to lack of Assessments:  Incur costs of a bad recruit estimated at Rs.3,00,000 - Advertising cost - Sourcing cost - Interview cost - Training cost - Induction & administrative costs - Compensation  Lowered productivity in the organization  Loss of knowledge  Incur costs of rehiring  Other significant opportunity costs EXERCISE I Exercise I Exercise:You be the Consultant Questions for Discussion: 1. What does your organizational recruitment engine look like?Do you have assessment tests as a filter? 2. 5000 people have responded to your recruitment advertisement from 8 different cities. How do you make a hiring decision? 3. Are other assessment solutions like interviews, background checks effective enough to provide a finer discrimination between higher ability candidates? Case Study Exploration:Hiring Success hinges on Assessment Tests Appended alongside is the Case Study Part III PART III: Assessments Tests for Recruitment ASSESSMENT TESTS Assessment Tests Gamut of Assessment Tests Test Types Domain Tests Cognitive Abilities Tests Communication Skills Tests Personality Profiling Question Types Performance Tests Multiple Choice Questions Free Response Tests Test Levels Basic Standard Advanced Mode of Administration Online Paper Pen DOMAIN TESTS Domain Tests Also known as:  Job Knowledge Tests Purpose  Assesses technical or professional expertise and knowledge required for specific jobs or professions Format  Multiple choice questions or essay type items  Produce valid inferences for hiring decisions Pros  Viewed positively by test takers who see the close relationship between the test and the job  Depends heavily on behavior demonstration than simply assessing characteristics  Less likely to differ in results due to bias than other types of tests DOMAIN TESTS Domain Tests contd… Some of the Domain Test Offerings: IT Domain Tests • Programming Languages BPO Domain Tests • PC Hardware • Wireless LAN Concepts Others • Basic Accounts • Operating Systems • Insurance Fundamentals • Databases • Networking Concepts • Computer Science • Enterprise Applications • Web Development Tools • ERP • Windows Client User • PC Skills • CISCO Internetworking • Electrical Engineering • Electronics Engineering • Mechanical Engineering COGNITIVE ABILITIES TESTS Abilities Tests Also known as:  Mental Aptitude tests Purpose  Assess current ability and future potential for different work skills Format  Multiple choice questions or essay type items  Produce valid inferences for hiring decisions Pros  Viewed positively by test takers who see the close relationship between the test and the job  Depends heavily on behavior demonstration than simply assessing characteristics  Not influenced by test taker attempts to impression manage or fake responses COGNITIVE ABILITIES TESTS Abilities Tests contd… Some of the Abilities Test Offerings: Verbal Ability  Assesses the ability to use language correctly and appropriately Analytical Ability  Assesses the ability to analyse data and situations, to see cause and effect and to use this to make effective decisions Quantitative Ability  Assesses the use and reasoning of the numbers or mathematical concepts Attention to Detail  Assesses an applicant’s ability to accurately identify and isolate similar/minute details COMMUNICATION SKILLS TESTS Communication Skills Tests Purpose Assesses the language level of the candidate: spoken understanding, reading comprehension, grammar and written expression Format Paper Pencil or Performance Tests Produce valid inferences for hiring decisions Pros Viewed positively by test takers who see the close relationship between the test and the job  Depends heavily on behavior demonstration than simply assessing characteristics COMMUNICATION SKILLS TESTS Communication Skills Tests contd… Some of the Communication Test Offerings: Voice Clarity Spoken English Test Assess a test taker’s spoken English skills Accent Aspects Fluency Grammar Contextual Relevance Written English Test Assesses a test taker’s written English skills Word Usage Aspects Spelling & Punctuation PERSONALITY TESTS Personality Tests Purpose Assesses attitudes and personality dimensions Format Via Paper Pencil or Computerized Measures Pros Produce valid inferences for a number of organizational outcomes (e.g., performance, inventory shrinkage difficulties in dealing with supervision) Reduce business costs by identifying individuals who are less likely to be absent, or engage in other counterproductive behavior EXERCISE II Exercise II Exercise:You be the Consultant Please attempt the test administered to you. Please score your responses by referring to the scoring key provided to you. 1. How did you score on the dimensions of Warmth, Influence and Sensitivity? If your total score was:   3 or below, you are relatively low on these dimensions 4-7, you are average on these dimensions  8 or above, you are high on these dimensions 2. Suppose you were hiring employees for the following jobs, would you prefer people who are high or low on Warmth, Influence and Sensitivity for each position:  Software Tester  Human Resource Manager  Counter Sales Executive  Accountant Part IV PART IV: Challenges in using Assessment Tests CHALLENGES IN THE USE OF ASSESSMENT TESTS Challenges in the use of Assessment Tests Logical Choice  Map the correct set of tests to role-required competencies Reliability  Consistency of Test Results Validity  Accuracy of the inferences made based on test results CHALLENGES IN THE USE OF ASSESSMENT TESTS Challenges in the use of Assessment Tests contd… Test Interpretation  Test Scores meaningless when interpreted in isolation  Data to aid in appropriate test score interpretation Scoring Options  Raw Scores  Standardized scores and Percentiles Test Credentials  Research and development in creating the Test LOGICAL CHOICE OF TESTS Logical Choice of Tests Mapping the correct set of tests to the role-specific competencies Step 1 Task Step 2 Understand the role-specific tasks & competencies through a Functional Specification Document Written English Map the relevant tests to assess competencies Analytical Ability Test Illustration I: Role of a Software Developer Following are the basic Tasks: Requirement Specification Software Unit Designing Unit Testing and Debugging Business Communication Technology Skills Test Software Developer Attention to Detail Test Coding Rationale Learning Ability Test LOGICAL CHOICE OF TESTS Logical Choice of Tests contd.. Mapping the correct set of tests to the role-specific competencies Task Illustration II-: Role of a Tech Support Agent(Voice) Following are some of the basic Tasks: Provide front line queue support for customers Written English Learning Spoken Ability English Test Handle & enter all calls into call tracking database Listen attentively and respond effectively to customer complaints PC Skills Test Tech Support Agent Attention to Detail Test Listening Skills Test EXERCISE III Exercise III Action Assignment: Logical Choice of Tests Step 1- Understand the role-specific tasks & competencies through a Functional Specification Document Step 2- Map the relevant tests to assess competencies through a Test Matrix VALIDITY OF A TEST Validity Food for thought: Why do I need to know about the validity of a test? An examinee gets a score on the SAT. How confident are you to draw conclusions based on the test score without the scores being a direct measure of the verbal and mathematical ability, such as degrees centigrade are a direct measure of the heat of an object. Why Validity?  To provide solid rationale for decision-making based on test What is Validity?  A judgment of the degree to which test assesses what it purports to assess  Through correlation coefficient How to assess Validity?  The value of the correlation coefficient can take on any value between -1 and +1 VALIDITY OF A TEST Validity contd.. Types of Validity: Content Validity  Match between test questions and the content to assess  Comes from content experts Criterion Validity  Match between a test score and some desired outcome  Comes from corelating test scores to the criterion  Degree to which a test assesses the underlying theoretical construct Construct Validity Example: Algebra test with test questions phrased in long and complex reading passages, inadvertently assesses reading skills instead of factual knowledge of basic algebra Comes from correlating two tests that measure closely related constructs RELIABILITY OF A TEST Reliability Food for thought: Does the test give you the same results every time it is used? An examinee gets a score on the test. How confident are you to draw conclusions based on the test score without the test giving you the same results every time it is used? Why Reliability?  To provide solid rationale for decision-making based on test What is Reliability?  Is the degree of consistency of the the measure How to assess Reliability?  Through reliability coefficient  The value of the reliability coefficient can take on any value between 1 and +1 RELIABILITY OF A TEST Reliability contd.. Types of Reliability: Single Measure Measured at one testing session Homogeneity (alpha)  Split half reliability Different forms of the Measure Measured at two testing sessions Test-Retest reliability TEST INTERPRETATION Test Interpretation Mean  The average score of a group of test takers Median  The middle of a distribution that cuts the group into two halves Mode  The most frequently occurring value in the data set TEST INTERPRETATION Test Interpretation contd… Standard Deviation  Measure of the variance from the mean in a group of test takers  A measure of the "spread" of scores between test takers  Standard deviation of test scores that would have been obtained from a single test taker had that he been tested multiple times Standard Error of Measurement  A measure of the "spread" of scores within a test taker had the test taker been tested repeatedly  Each test taker need not be tested repeatedly on a test in order to estimate the SEM  SEM can be computed from the test scores of a population of test takers on a single test TEST INTERPRETATION An Illustration: For A Sample Test Measures Mean Median SD SEM Reliability Sample Test 17.33 18 3.67 1.9 0.72 Test Interpretation: An illustration Ideal Value 15.62 NA 3.75 1.9 0.65 T e s t Verbal Ability 300 250 T a 200 k 150 e r 100 s 50 0 Scores Data sample: 2370 Findings  The mean value for the sample test is higher than the ideal value, which illustrated by the high scores achieved by the test takers  SEM value is within an acceptable range; if a test taker scores 14 in this test, the true score range would be 14±2, where 2 is the SEM  Reliability value is higher than the acceptable range (0.6 to 0.65) for 25 items Ideal Mean is a mean that should be obtained for a given set of Items if the test is designed correctly EXERCISE IV Exercise IV Exercise IV:Test Interpretation Exercise- Given are three Tests. Interpret the test specifications and choose the best Test based on the Test Specifications. SCORING OPTIONS Scoring Options contd… Food for thought: Candidate A gets raw scores of 50 on the verbal ability test and 50 on the numerical ability test. Is it technically correct to compare his raw scores on the two tests and with that of other test takers? Verbal Ability 15 Numerical Ability Step 1: Consider the Mean •In one case, A’s test score is 10 points above the mean 10 •In the other case, A’s test score is 10 points below the mean •Thus, we have to interpret A’s score to the average performance of the group/class 0 5 0 20 40 60 SCORES 80 100 Mean = 40 Mean 60 SCORING OPTIONS Scoring Options contd.. Step 2: Consider the SD Verbal Ability 30 20 •Both distributions have the same mean (40), but different standard deviations (10 vs. 20) •Thus, we have to interpret A’s score in relation to the variability in the test scores 10 15 25 Numerical Ability 0 0 5 20 40 60 SCORES 80 100 SCORING OPTIONS Standard (Z) Scores • Scoring Options contd.. In short, we would like to be able to express a person’s score with respect to both (a) the mean of the group and (b) the variability of the scores – how far a person is from the mean = X - M – variability = SD Standard score or Z i = ( Xi - M) SD SCORING OPTIONS Scoring Options Some of the different Score Offerings: Raw Scores  The number of questions a test taker gets right on a test  Little or no meaning- converted to other types of scores Percentile Scores  Percentage of test takers in the sample who gained a score at the same level or below that the test-taker  Percentile ranks range from 1 to 99 Standardized Scores  Standard scores take into account Mean and Standard Deviation  Allow for comparison between scores of test takers and scores of a single test taker on different tests TEST CREDENTIALS Test Credentials Theory/Experience on which the test was based Estimates of Validity & Reliability Test Credentials Process used to develop the test Ongoing Research & Refinement of the test Thank You