Executive Summary

Document Sample
Executive Summary Powered By Docstoc

    ACT WorkKeys
    Certified Job Profile Report

    Prepared for Center for Continuing
    Education at
    Macomb Community College

    July 30, 2008

                              Table of Contents

Executive Summary ...................................................................................... 3

Recommendations ........................................................................................ 4

Job Profiling Process .................................................................................... 5

Final Task List................................................................................................ 8

Skill Analysis Documentation .................................................................... 11

Reading For Information ............................................................................. 11
Locating Information
Applied Mathematics

Subject Matter Expert Demographics ........................................................ 17

WorkKeys Terminology .............................................................................. 19

WorkKeys and Content Validation ............................................................. 22

                      Executive Summary

This occupational job profile report presents the results of an ACT WorkKeys® job
analysis for the position of Certified Nurse Assistant (CNA). The profile was conducted
for the Center of Continuing Education Office of Macomb Community College by ACT
authorized lead profiler Ann Stanton of Macomb Community College, Shelley Kaye of
Oakland Community College, and Cindy Leyrer of Ingham Intermediate School District.
Ann, Shelley, and Cindy bring over 25 years of combined job profiling experience as
ACT authorized profilers.

An occupational profile differs from a traditional ACT Job Profile by soliciting input using
subject matter experts or SMEs from several companies instead of from one single
company. An occupational profile is also conducted for the purpose of (a) identifying
the skills and skill levels for entry and for success in completing a new or existing
training program, (b) to evaluate the effectiveness of a program by comparing the pre-
and post- assessment scores of trainees to the program’s profile requirements, or to (c)
document the entry and exit skill levels of a program as part of the initial development
process of the training program. This occupational profile was intended to address not
only (a), (b) and (c) but to document the tasks associated with the job of CNA and for
it’s usefulness to the development of the training program for Certified Nurse

The results of this profiling project and a review of its findings can (a) help guide the
selection of students into the training program and (b) encourage skill development for
those applicants whose skills currently do not match the requirements for entry.
Results from this occupational profile process can be found on Table 1: Skill Level
Requirements for CNA.


One recommendation remains before using the skill levels established by the profile to
set expectations for potential and current students:

   For admission to the program, it is recommended that WorkKeys Reading for
    Information, Locating Information, and Applied Mathematics assessments be
    administered to all students. This recommendation allows for a match between the
    student’s current skill level and those of the profile. Possible resources to address
    skill development and remediation needs when scores are deficient are KeyTrain
    and Plato courseware. WorkKeys assessments should be used in addition to
    measures currently used for selection: a student’s GPA, interviews with the student,
    background checks, etc.

                      Job Profiling Process

WorkKeys job profiling is conducted by profilers who have been trained and authorized
by ACT WorkKeys Industrial/Organizational Psychologists. They follow an established
process that is designed to systematically develop accurate profiles through a task
analysis that is used to select the tasks most critical to a job, and a skill analysis that is
used to identify the skills and skill levels required at the entry level and for effective
performance on that job.

On June 30th, 2008, from 8:00am until 5:00pm, Ann, Shelley, and Cindy met at
Macomb Community College’s MTEC facility in Warren with seven subject matter
experts (SMEs), individuals who actually perform the job of CNA at different
companies. The SMEs’ who participated in this process represented four employers,
Brookdale Senior Living, Bortz HealthCare, Henry Ford Macomb Hospital, and Relevar
Home Care. These facilities offer Long Term Basic or Assisted Living and Long Term
Skilled Care, along with in-patient health care services or in-home Health Care
Services. The SMEs work in the positions of CNA, Patient Care Assistants (PCA,),
Nurse Techs, and Resident Assistants.

Task Analysis
Prior to the beginning of the profile, a review of the learning objectives for the training
program using the Clinical Skills Test Checklist for state certification exam for nurse
assistant was completed. This checklist provided 23 content areas, including but not
limited to; Hand Washing, Indirect Care, Pulse and Respirations, Personal Care,
Transfer, Range of Motion, Blood Pressure and so on. General company information
was collected along with job descriptions and basic workplace materials such as
incident reports, medication logs, shift reports, weekly visit reports, Care Plans, etc.
Research using the on-line Occupational Information Network (O*NET) and the
Department of Labor’s on-line resources for high demand jobs was referenced in
educating the profilers on tasks, tools and technology, knowledge, skills, abilities, work,
activities, etc. of the CNA. ACT also served as a resource for identifying occupations
the same or similar to a nurse assistant for whom a job or occupational profile was
conducted. These combined resources assisted in the development of an Initial Task
List, which the SMEs reviewed, changed, edited, added, and deleted so that the Final
Task List reflected an accumulative, agreed upon list of tasks the SMEs perform on a
regular basis.

Once the task list was finalized, the SMEs rated each task, providing a criticality rating
by evaluating each task in terms of its importance and the time they spend on that task
relative to the time they spend on other tasks (Importance rating and Relative Time
Spent rating). The mean Importance rating for each task was multiplied by the mean
Relative Time Spent rating for each task to produce the Criticality rating for each task.
These ratings represent aggregate information rather than information reached by the
consensus of the SMEs. The criticality information was used to sort the task
statements, placing the most critical tasks at the beginning of the list. The groups
confirmed that the tasks on their respective lists were critical to the job. The Final Task
List can be found in the Final Task List section of this report.

Skill Analysis
The SME group completed a skill analysis to identify the WorkKeys skills required for
the job as CNA. WorkKeys Reading for Information, Locating Information, and Applied
Mathematics skills were reviewed one at a time and the SMEs finished the analysis for
one skill before going on to the next. The profiler gave each SME a copy of the skill
definition, read the definition aloud, and answered any SME questions. Once the
SMEs understood the definition of a WorkKeys skill and had determined its relevance
to the tasks on the Final Task list, the group identified the tasks on the list that require
each of the skills and specified how individuals use the skill to achieve the objective of
each task.

    The SMEs also identified the appropriate skill levels and determined which level
     was needed for program entry and completion (or program exit). Entry was defined
     as the students’ first day in the program, before they gained program-specific
     knowledge from training or experience. The SMEs agreed that skill levels for entry
     and for exit were the same. These results are shown in Table 1. Finally, the SMEs
     ranked the skills indicating Reading for Information as the most critical skill, with
     Locating Information as second, followed by Applied Mathematics as the least
     critical to tasks requiring the skill for the position of CNA. A summary of the SMEs’
     discussion of each skill can be found in the Skill Analysis Documentation section of
     this report.

Table 1. Skill Level Requirements for CNA

    WorkKeys Skill           Skill Level        Entry                Exit
                             Range              Performance          Performance

            Reading for
                                      3-7               4                     4
            Locating                  3–6
                                                        4                     4
                                      3–7               3                     3

                              Final Task List

The Final Task Lists for the CNAs are shown in the table below. An “X” mark in a skill
column means that, according to the SMEs in the profile session, the task on that row
requires that skill. The names of the WorkKeys skills have been abbreviated to save
space, as follows: Reading for Information (RI), Locating Information (LI), and Applied
Mathematics (AM). Tasks are presented in order, from those most critical to job
performance to those least critical.

                                                                  RFI       LI       AM

 1.  Alerts nurse to all critical values such as blood            X         X         X
     pressure, temperature, glucose levels, pulse,
     respiratory rates and oxygen level.
 2. Transfers patients using proper body mechanics and                      X
     lifting equipment (i.e. Hoyers, gait belts, draw sheets,
     sliding boards) alone or with assistance.
 3. Bathes, shaves, grooms, dresses, providing assistance                   X
     in personal hygiene and prepares patient for surgery,
     treatment, or examination.
 4. Turns and repositions patients, alone or with                 X         X
     assistance, to prevent bedsores and promote
     circulation paying attention to pressure points.
 5. Actively listens giving full attention to what other people
     are saying taking time to understand points being
     made, asking questions as appropriate, and not
 6. Follows all documented federal, state, JCAHO, OSHA,           X         X
     HIPPA, and patient rights and privacy with standards
     and regulations on health and safety.
 7. Measures vital signs, such as pulse rate, temperature,        X         X         X
     blood pressure, weight, and height, and records
     information on patients' charts and/or in computer.
 8. Measures, records, and reports patient intake and             X         X         X
     output, (urination, feces, drainage tubes, suction
     collection) noting abnormality in texture and/or color.
 9. Observes, documents and reports verbally and by               X         X
     charting observations or critical changes in physical and
     mental status to medical staff.
 10. Answers signal lights, bells, or intercom system to
     determine patients' needs.
 11. Assists patient with range of motion (ROM) such as:          X         X         X
     walking, exercising, moving in and out of bed,
     automobiles, wheelchairs, and up and down steps.
                                                               RFI   LI       AM

12. Documents patient’s daily activities in writing or         X     X        X
    electronically on a regular scheduled basis.
13. Monitoring fluid intake and observes dehydration status          X        X
    such as dry mouth, chapped lips, sunken eyes, and
    skin elasticity.
14. Attends in-service training or computer generated          X              X
    lecture for the purpose of reviewing and receiving
    information such as elderly diseases, infection control,
    awareness of MSDS and must pass subject matter
    tests or takes additional training.
15. Documents, records, enters, transcribes, stores, or        X              X
    maintains information in written or electronic form.
16. Meets with oncoming shift to pass on information and       X
    determine assignments.
17. Collects samples, (i.e. urine, blood, sputum, feces)             X
    labels, dates, packages samples for transfer/lab pick
18. Follows standard emergency codes such as code blue,        X     X
    black, pink according to CNA rules and responsibilities.
19. Prepares, serves and collects food trays and feeds                        X
    patients requiring help according to prescribed dietary
20. Straightens and organizes patient's room and personal
    belongings and changes bed linens daily or as needed.
21. Reads and understands workplace information such as        X     X        X
    fire safety, Material Safety Data Sheet information,
    hazards, codes, memos, etc.
22. Cleans Foley tubes according to sterile technique.               X
23. Assists nurse by attending to patient as needed while
    nurse performs procedures such as catherazation,
    applying dressing, starting IVs, and suctioning.
24. Entertains patient, reads aloud, and plays cards or        X
    other games with patient, keeping them mentally
    healthy and alert.
25. Checks supply levels and communicates with                                X
    necessary staff for ordering.
26. Finger sticks patient and tests blood sugar levels using         X        X
    strip method then records in computer and/or patient
27. Monitors performance and processes of self, other                X
    individuals, or organizations to make improvements or
    take corrective action using meeting format, suggestion
    box, or other methods.
                                                                RFI   LI        AM

28. Observes and reports (verbally, in writing, or by calling   X     X
    State hotline) patient abuse and neglect by staff and or
    family members following awareness guidelines.
29. Provides post-mortem care by washing, cleaning and                X
    positioning body, removing tubes and Foleys, and
    tagging and putting in body bag for transfer.
30. Accompanies ambulatory patients outside home,
    serving as guide, companion, and aide.
31. Drapes patients for examinations and treatments, and              X
    remains with patients, performing such duties as
    holding instruments and adjusting lights.
32. Draws blood by intravenous or fingers stick methods               X
    and sends to the lab.
33. Cleans and sterilizes equipment such as IV poles,                 X
    beds, table tops, or bedside commodes.
34. Massages patient and applies preparations and
    treatments, such as liniment, ice packs or alcohol rubs
    and lotions.
35. Provides general light housekeeping duties such as                X
    changing bed linens, laundry including washing and
    ironing, cooking, washing dishes, vacuuming and
    cleans patient’s quarters.
36. Runs errands for in-patients facilities such as going to
    utility area for equipment and supplies.
37. Directs visitors and answers telephone.
38. Performs bladder scans and records information in
    computer system and reports to nurse.
39. Purchases, prepares, and serves food for patient and              X         X
    other members of family, following special prescribed
40. Communicates by email or phone with home office on          X
    daily or weekly basis to receive schedule, patient status
    or changes for home health care visits.
41. Travels to several households daily to provide health
    care to patients.
42. Performs variety of miscellaneous duties as requested,
    such as shopping, banking, doctor appointments,
    personal care needs and running errands.
43. Prepares treatment rooms for examination of patients
    ensuring room is presentable and ready for use.

             Skill Analysis Documentation

The WorkKeys skills are presented in order, from those that are the most critical to
performance in the training program to those that are the least critical.

Reading for Information
The WorkKeys Reading for Information skill is an employee’s skill in reading and
understanding work-related written text in order to do a job. The written text includes
memos, letters, directions, notices, bulletins, policies, and regulations. Reading for
Information materials do not include information that is presented graphically such as
charts, forms, or blueprints. There are five levels of difficulty ranging from Level 3 the
least complex, to Level 7 the most complex. After presenting the SMEs with definitions
of Level 3, Level 4 and Level 5, the SMEs indicated the Reading for Information skill
required for the job of CNA is Level 4.

These are the characteristics of Level 4 reading materials:

      Identify important details that may not be clearly stated. Reading passages are
       slightly more complex, and contain a number of details and important details
       may not be clearly stated or be less obvious.
      Use the reading material to figure out the meaning of words that are not defined
       for them.
      Passages describe policies and procedures with a variety of factors that must be
       considered in order to decide on appropriate action or behavior.
      Reading materials contain information that requires one to choose what to do
       when several changing conditions call for a different action. Examples include
       reading vital signs on a patient or when a patient’s glucose level changes, they
       may need to respond with a change of course or action with appropriate

      The material read may be written in a straightforward manner, but they have
       longer compound sentences and more complex vocabulary, using words that
       are more difficult than those at Level 3. For example, the word “immediately”
       may be used at this level, whereas at Level 3 the phrase “right away” would be
      Determine the meaning of words that are not clearly defined in the reading
      Read material and recognize the application of more complex instructions
       including the need to understand cause-effect and if-then relationships.

The SMEs indicated that 15 tasks on the Final Task List required the Reading for
Information Skill, or 35% of the total tasks. Materials include patient charts, care plans,
and chart summaries. This includes information on why the patient is there, what their
current condition or status is, what orders might be, along with patient history. In
addition, “how to do” orders are read, training materials, written lectures from JCAHO
and HIPAA, “Code” descriptions (Red, Blue, Black, etc), and they also read books,
newspapers, and other literature to patients. Material on abuse and neglect rules and
regulations must be read, and in some cases union bylaws which include technical
reading material are read. In all of the examples, terminology and details may be
difficult and not clearly stated. If-then situations exist, meaning that the CNA must
determine what to do and what the result might be for their action.

The SMEs indicated that Level 5 Reading for Information Skill was not required due to
the fact that CNAs are not required to read materials containing technical terms,
acronyms, or jargon having several meanings or are not defined for them, they do not
apply complex instructions that include conditionals without direction or supervision
from the nursing staff, and they do not need to figure out the meaning of difficult words,
but rather they figure out meanings from the context of the reading material.

Locating Information
The WorkKeys Locating Information skill is an employee’s skill in using information
taken from workplace graphics such as diagrams, maps, floor plans, tables, forms,
graphs (including bar charts, pie charts, and line graphs), flowcharts, and instrument
gauges. Employees use this skill when they find information in a graphic or insert
information into a graphic. They also use it when they compare, summarize, and
analyze information found in related graphics. There are five levels of difficulty ranging
from Level 3 the least complex, to Level 6 the most complex. The Locating Information
skill level required for a job is determined by the complexity of the graphic(s) used to
accomplish a task, and the complexity of the task(s) performed. After presenting the
SMEs with definitions of Level 4 and Level 5, the SMEs indicated that the Locating
Information skill is for the job of CNA is Level 4.

These are the characteristics of Level 4 Locating Information:

      Find several pieces of information in one or more graphics
      Understand how the graphics are related to each other (use equipment that
       displays information and then insert or record the information into a patient’s
       chart or into the computer)
      Summarize information form one or more graphic, the CNA needs to review
       several pieces of information in a patient’s chart to determine changes and what
       needs to be done based on those changes
      Identify trends shown in one or more graphic, the CNA needs to recognize
       changes or trends in a patient’s status
      Compare information and trends shown in one or more graphics, the CNA needs
       to be able to use several pieces of information to monitor a patient’s progress

The SMEs indicated that 23 tasks on the Final Task List required the Locating
Information Skill, or 53% of the total tasks. The Locating Information skill is used when
the CNAs chart and/or fill in the blank forms (i.e. intake and output, vital signs, blood
sugars, collections, etc.). It is also used when they review and record chart summaries
which includes information on why the patient is there, what their current condition or
status is, what orders might be, along with patient history. The skill is used when a
patient is transported or transferred (forms). Other forms include ADL sheets (daily
living or activity sheets), patient care forms, release forms used for medication, and
incident reports (reporting accidents). The CNAs indicated they use charts for
determining critical values and enters all values into a computer screen recognizing
when values turn “red” indicating a problem (Vital Signs screen or PCA screen). They
use forms to document appearances on a patient’s body (bruises, rashes, redness,
sores, etc.). They also compare and summarize data on forms and records notifying
appropriate medical staff when necessary.

SMEs indicated that Level 5 Locating Information Skill was not required, that CNAs are
not required to work with complicated graphics presented in less common formats,
they do not need to sort through distracting information (when they are reading vitals
and other measurements-the display is for that vital alone), and the graphics are simple
and without distracting information. When setting the appropriate skill level, the SMEs
considered the number of graphics they use simultaneously, the function of the graphic,
and the format of the graphic.

Applied Mathematics
WorkKeys Applied Mathematics is the skill people use when they use mathematical
reasoning and problem-solving techniques to solve work-related problems. Employees
may use calculators and conversion tables to help with the problems, but they still need
to use math skills to think them through. There are five levels of difficulty, Level 3 is the
least complex, level 7 the most complex. The Applied Mathematics skill level required
for a job is determined by how the information is presented to the employee, if it’s in
order or if they need to reorder it, whether all the information is provided, if the types of
mathematical operations are single-step or multiple-step, and if conversions are used
within systems of measurement or between systems of measurement. After presenting
the SMEs with definitions of Level 3 and Level 4, the SMEs indicated that the Applied
Mathematics skill for the job of CNA is Level 3.

These are the characteristics of Level 3 Applied Mathematics:
      Solve problems that require a single type of mathematical operation. They add
       subtract, multiply or divide. Repetition of the same mathematical function does
       not increase the level
      Change numbers from one form to another such as when dividing fractions into
      No use of formulas
      No use of algebra

The SMEs indicated that 13 tasks on the Final Task List required the Applied
Mathematics Skill or 30% of the total tasks. The Applied Mathematics skills is used
when the CNAs determine pulse and respiratory rates (using division or multiplication),
when they measure total intake, fluids, volumes and when measuring pressure when
moving and turning patients. They also measure tubing, foot size (support hose) using
a single system of measurement. The CNAs count and document time intervals
between turning a patient, they add and document percentages of inputs and outputs
(patient ate 50% of their lunch plus they ate 100% of their dinner), and they calculate
percentage of patient improvement. CNAs use math skills (addition) when mixing
cleaning solutions with water, (MSD Sheets), when calculating the differences in range
and test results. Math skills are used when estimating the distance a patient has
walked (counting floor tiles), and when determining the weight of a patient by adding or
subtracting the weight of a wheelchair, bed, or chair.

The SMEs indicated that Level 4 Applied Mathematics Skill was not required due to the
fact that CNAs are not required to work using formulas, conversions, decimals,
percentages, or multiplying mixed numbers by a whole number or decimal. They do
not need to figure out how to set up and solve a multiple step problem, all the
information they need to solve the problem is provided to them. CNAs do not perform
several steps of calculation to determine what information they need in order to solve
the problem and arrive at the answer. The mathematical problems CNAs work with do
not require transposing formulas or using algebra to arrive at the answer.

     Subject Matter Expert Demographics

SME demographic information is provided below.
                Job Title                         ALL
         Certified Nurse Assistant                 4
          Patient Care Assistant                   1
            Resident Assistant                     1
               Nurse Tech                          1

          Years in Profiled Job                   ALL
                 Average                     11 years
                 Highest                     22 years
                 Lowest                          4 years

                   Age                            ALL
                 Average                     44 years
                 Highest                           65
                 Lowest                            28

                 Gender                           ALL
                 Female                            6
                   Male                            1

           Racial/Ethnic Group                    ALL
         African American, Black
               Non-Hispanic                        3
              Non-Hispanic                         3
                Chicano                            1
                        WorkKeys Terminology
This information is presented in the order that it typically appears during a job profile.

Job Profiling        A procedure to determine the most critical tasks for a job and to
                     determine the WorkKeys skills and skill levels required to perform
                     these tasks.

Job Profile          The result of conducting one or more job profiling sessions which
                     shows the most critical tasks for a job and the WorkKeys skills and skill
                     levels required to perform a job.

Profiler             An individual who has completed ACT's WorkKeys Job Profiling
                     training program successfully. An ACT authorized profiler has been
                     trained to:
                     (1) Facilitate the job profiling process while using the SkillPro®
                     (2) Generate a report of the profile results

SME                  Subject matter experts are employees currently performing the job or
                     people knowledgeable about the job being profiled (e.g., supervisors or
                     people who have been recently promoted from the job).

Profiling Session A focus group meeting facilitated by an ACT authorized job profiler.
                  The job profiler meets with SMEs to perform a task analysis and skill

Initial Task List    Prior to the profiling session, the profiler develops an Initial Task List
                     using information compiled from databases (e.g., Dictionary of
                     Occupational Titles and O*NET), job-related documentation (e.g., job
                     descriptions, resources from similar job profiles, training materials),
                     and information gathered from the tour of the facility.

Task Analysis     A task analysis consists of three parts:
                  (1) The job profiler meets with the SME group to tailor (i.e., add,
                      edit, and delete tasks) the Initial Task List, making sure that the
                      Final Task List accurately and completely describes the job.
                  (2) The SMEs independently rate each task for Importance and
                      Relative Time Spent. (Definitions are shown below.)
                  (3) The profiler calculates the criticality of each task using the SME
                      ratings, and sorts the task statements by placing the most
                      critical tasks at the beginning of the list. The SMEs review and
                      confirm the order of the tasks. The product of the task analysis
                      is the Final Task List.

Importance        The importance of the task to the job.

Relative Time     The amount of time spent performing a task relative to the amount of
Spent             time spent on all the other tasks.

Criticality       The extent to which a task is critical to the job. To calculate Criticality,
                  the profiler multiplies the mean Relative Time Spent rating by the mean
                  Importance rating for each task. These ratings represent aggregate
                  information rather than information reached by the consensus of the

Final Task List   A list specifying the critical tasks for a job in statements that have been
                  reviewed and edited by SMEs and then placed in criticality order using
                  SME ratings.

Skill Analysis    A skill analysis occurs after a task analysis is completed and consists
                  of two parts:
                  (1) The SME group identifies the on-the-job behaviors (i.e., tasks
                      from the Final Task List) that are associated with the WorkKeys
                      skills under consideration.
                  (2) The SME group compares detailed descriptions of the WorkKeys
                      skill levels to the tasks that require the specified skill. The job
                      profiler seeks to bring the group to a consensus regarding the skill
                      levels required at job entry and for effective performance.

Entry-Level      Following the Uniform Guidelines on Employee Selection Procedures
                 (1978), WorkKeys defines entry as an employee’s first day performing
                 the job. The entry-level skill requirements are recommended for use
                 as cutoff scores on the related WorkKeys assessments.

Effective        Effective performance is the point at which an employee performs
Performance      competently without continuous supervision. Effective performance
Level            levels are provided for use as training goals.

Replication      Replication sessions are additional profiling sessions with different
                 groups of SMEs. Replication sessions are used to make sure that the
                 results are consistent from one group to another, especially when there
                 are a large number of incumbents on the job.

Reconciliation   When SME groups do not agree on skill requirements (generally for job
                 entry) the profiler meets with representative SMEs from each group to
                 resolve the differences in a reconciliation session.

Profile Report   A summary generated by the profiler that includes the Final Task List,
                 detailed descriptions of the session discussions of each skill, and
                 recommendations for using the results.

Assessment       A test used to evaluate individuals' performance in a skill area. Scores
                 on the WorkKeys assessments can be compared to the WorkKeys skill
                 levels identified in a profile. The difference between the profiled level
                 and a score indicates the need for training.

Skill Gap        When the profiled skill level is higher than the assessment score, the
                 difference is referred to as a “skill gap.”

Value-Added      A WorkKeys Value-Added Reseller provides a variety of workforce
Reseller (VAR)   development services including profiling, assessment, and training to
                 support employers in the local area.

ACT Center™      ACT Centers support workforce development through a
Network          comprehensive set of distance-delivered and site-based services such
                 as guidance, testing, and training. An ACT Center may also be a
                 WorkKeys Value-Added Reseller.

          WorkKeys and Content Validation

    Recommendations Regarding Validation
In developing the WorkKeys system, including WorkKeys assessments and job
profiling, ACT has and will continue to be guided by professional documents such as
the Standards for Educational and Psychological Testing (1999; developed by the
American Education Research Association, American Psychological Association and
National Council for Measurement in Education), the Principles for the Validation and
Use of Personnel Selection Procedures (2002, Society for Industrial and Organizational
Psychology); and the Uniform Guidelines on Employee Selection Procedures (1978),
which has been adopted by the Equal Employment Opportunity Commission (EEOC)
and various other federal agencies (Ref: 29 C.F.R. Part 1607). ACT believes that use
of the WorkKeys assessments that is consistent with ACT’s guidelines will meet the
standards set forth in the professional standards.

An effective selection system and its components can predict an applicant’s likelihood
of success for a particular job. To be legally defensible against charges of disparate
treatment or adverse impact, an employer must be able to establish a credible link
between the job and the measures used to select employees. In other words, the
validity of a test for selection purposes is generally based upon (1) the development of
the test, and (2) the relationship between the test and the job.

Validation is a process of determining whether the procedure or instrument is effective
in distinguishing individuals who will perform well on the job from those who will not
perform well on the job. The Guidelines support using one of the following strategies:

   Content validation: requires evidence that the selection procedure is
    representative of the content of the job

   Criterion-related validation: requires statistical evidence of the relationship
    between performance on a selection procedure and job performance

   Construct validation: requires evidence of the relationship between a construct
    measured by the selection procedure and the related work behavior(s) for the

Validation is not determined by only testing job incumbents and comparing their scores
to the profile. Though this may be useful for determining individual employee training
and development needs, it is not thorough enough to qualify as validity evidence
according to the professional standards contained in the Guidelines, Standards, and
Principles. While there are many reasons for this, it is primarily because there needs to
be a way to compare test scores for a group of incumbents to their job performance
before drawing a conclusion.

WorkKeys Test Development
The development of the WorkKeys assessments was guided by professional standards
for testing contained in the Guidelines, Standards, and Principles. For example, the
items on the WorkKeys assessments go through a series of screens before the test
development process is completed. Below is a description of characteristics of the
WorkKeys tests to ensure that they are job related and fair to test takers:

    A. The assessments are criterion-referenced (they use job requirements as the
    B. The test specifications are well defined.
    C. People who are familiar with various work situations and have insight into the
       use of a particular skill in different employment settings write the items.
    D. Items measure a particular workplace skill.
    E. Prior to construction of the released assessment, content and fairness
       experts review the items to determine possible differences in responses
       among racial groups and between men and women.
    F. Statistical analyses at the item and test level are conducted to monitor the
       performance of various subgroups. For example, a DIF (Differential Item
       Functioning) analysis, a statistical procedure for identifying bias for specified
       groups (e.g., race, gender) is run for each item.

Paper-and-pencil tests that are intended to replicate a work behavior are most likely to

be appropriate when work behaviors are performed in paper-and-pencil form (e.g.,
editing and bookkeeping). Paper-and-pencil tests of effectiveness in interpersonal
relations (e.g., sales or supervision), or physical activities (e.g., automobile repair), or
ability to function properly under danger (e.g., firefighters) generally are not close
enough approximations of work behaviors to show content validity. For these
behaviors, a different validity strategy would be more appropriate.

In order to demonstrate the content validity of a test of job knowledge, the following
requirements must be met. There must be a defined, well-recognized body of
information and knowledge, and the information must be prerequisite to performance of
the required work behaviors. The work behavior(s) to which each knowledge is related
should be identified on an item-by-item basis. The test should fairly sample the
information that is actually used by the employee on the job, so the level of difficulty of
the test items should correspond to the level of difficulty of the knowledge as used in
the work behavior. Additional technical information regarding the development of the
WorkKeys assessments is available from ACT.

Relationship Between WorkKeys and the Job
The WorkKeys system employs the content validation strategy to show the WorkKeys
assessments reflect the content of the job. This allows employers, regardless of their
size, to establish legally defensible validity evidence. First, WorkKeys test items are
developed from samples of a variety of work situations so items on the assessment
reflect situations that might be found on the job. Next, the job profiling process
establishes a link between tasks performed on the job, the WorkKeys skills needed to
perform the tasks, and the skills and skill levels measured with the WorkKeys

During job profiling, groups of subject matter experts are convened to describe the
tasks performed on the job and the skills required for completing the tasks. SMEs are
knowledgeable about the job and are representative of the job incumbents in terms of

such characteristics as age, race, gender, national origin, and religion. Using
definitions and sample items that match the specifications of the assessment, they then
match the levels of skill needed for the job with the levels of skill measured by the
WorkKeys assessments.


Shared By: