Online Innovations to the Behavior Description Interview

Document Sample
Online Innovations to the Behavior Description Interview Powered By Docstoc
					Online Innovations to the
 Assessment Interview
     Impacts on Accuracy, Scoring,
       and Participant Reactions

                Tom Janz, Ph D
                Chief Science Officer
                Behavior Description Technologies
Why the Assessment Interview went
     .00 - .05 Graphology, Numerology, Astrology
     .05 - .15 Resume Ratings
     .15 - .20 Unstructured 1-on-1 interviews
     .20 - .24 Traditional reference checks
     .30 - .35 Panel Interviews
     .35 - .40 Validated Personality Tests
     .50 - .55 Cognitive Ability Tests
     .50 - .55 Structured Behavior-based Interviews
     .60 - .65 Ability Test plus Structured Interview
     .65 - .70 Ability Test plus Work Simulation
Why the Behavioral Interview is
going Online
   Increase in Candidate Comfort:
       Online Interviews rated as less stressful
       Online Interviews leave a better first impression
       Interviewers report feeling better prepare

   Increase in Decision Power:
       Study finds Online Interview .69 vs .48 Campion & Mueller

   Reduction in Staffing Spend:
       Fewer finalists needed for on-site interviews
       Interviews started online take 30-50% less time
The Road to Online Interviewing

   No one thought candidates would type out
    answers to behavioral questions– Not even
    Lowell Hellervik, Bill Byham, or Paul Green.

   Following the lead of computerized testing,
    early versions put the questions on screen.

   Candidates typed whole answers into a text
    box, guided by answer hints or probes.
Innovations in the Assessment Interview
supported Online:

   Answer Coaching

   Performance Auto-Confirm

   Best-Answer Anchoring

   Behavioral Judgment Testing
Innovation 1: Answer Coaching
Candidate Reactions to the Online Interview:
Here are the questions we asked--
We graph the answer ratings from over 10,000
Restaurant Manager, 900 Pharmaceutical Sales, and
70 Mine Manager candidates who completed a profile.

  On the degree to which E.ssessor is difficult and stressful:

Mine Managers          9              27                           34

 Pharm a Sales     9                       58                             33

Restaurant Mgr 2           34                                 64

             0%            20%             40%         60%              80%        100%

                           MORE       About the SAME   LESS

           On the degree to which E.ssessor is complete and thorough:

Mine Managers                              74                                 24       1

 Pharm a Sales                   52                                44                  4

Restaurant Mgr             37                                55                    8

             0%            20%             40%         60%              80%        100%

                           MORE       About the SAME   LESS
    On the degree to which E.ssessor is a fair assessment:

Mine Managers                     60                               34              6

 Pharm a Sales         32                                    61                7

Restaurant Mgr        27                                62                    11

             0%            20%            40%           60%        80%         100%

                           MORE        About the SAME   LESS

   E.ssessor left me feeling positive about working for you:

Mine Managers                           71                               29            0

 Pharm a Sales                    57                               42                  1

Restaurant Mgr                     61                               38                 1

             0%            20%            40%           60%        80%         100%

                           MORE        About the SAME   LESS

            Compared to past interviews, I like E.ssessor:

Mine Managers                      64                               33             3

 Pharm a Sales             37                                52               11           N=924

Restaurant Mgr             39                                 50              11           N=10151

             0%            20%            40%           60%        80%         100%

                           MORE        About the SAME   LESS
Innovation 2: Performance Auto-Confirm.

   Merely collecting a contact name and a
    means to connect compels more accurate
    self evaluations (bogus pipeline research).
   Users who provide email addresses for their
    contacts enable a labor-less confirmation.
   Users who give permission to confirm their
    performance at any time make auto-
    confirmation possible.
   Here is how it works.
Innovation 3: Best Answer Anchoring.

   Most quality approaches to rating behavioral
    interview answers feature rating anchors.
   Research finds behavioral anchors add about
    10 validity points over Lickert answers.
   However, behavioral anchors often take on
    the flavor of apple pie and motherhood.
   Performance component levels that are
    differentiated primarily by frequency
    adjectives (still better than no guideline).
   How about providing BOTH performance
    components and REAL “Best Answers”?
Innovation 4: Behavioral Judgment
   Situational Judgment Tests probe how people
    WOULD ACT in challenging, job-related, situations.

   How people WOULD ACT cannot be confirmed.

   How people WOULD ACT overlaps with their mental
    ability for picking the most effective answers.

   Behavioral Judgment Tests focus on what
    candidates DID DO, in challenging, job-related,
    situations, seeking confirmation from third parties.
Introductory Screen and Sample Item.
Candidates begin by rating HOW OFTEN they
faced this scenario.
Since this scenario was faced more than 2 times, the
next rating anchors WHEN the Best Example took
place. Then we show the first Action Option, and collect
its similarity to what the person DID.
Now we independently collect the candidate’s similarity
rating to the second action option. (Note: The candidate
does not know how many action options there are for
each scenario.)
The Ratings Review Page displays ALL the action
options (up to 5) and asks candidates whether they
would like to adjust their ratings.
Finally, candidate are asked to provide the name,
position, and contact info for someone who can confirm
their actions in the situation they selected for this

Shared By: