Wild guess_ lucky guess_ good guess –

							     Guessing
at a multiple-choice
  test of listening
  comprehension
     EALTA, Krakow
       20.5 2006




              Joanna Anckar
     CALS, Jyväskylä University, Finland
                         Data
 Short written introspection:
 226 test-takers justify their selection of a
  particular option at 17 items
 973 responses (25% of all responses):
 ”a guess”, ”to guess” or similar
 Why & how?
  247 of these include further comments
 10   types of guessing

                        Joanna Anckar
               CALS, Jyväskylä University, Finland
Examples
’They talked too quickly, the answer is a
   pure guess’
‘I did understand what was said on the
   tape, but I don’t understand options B
   and C, B is a stronger guess’
’I made a guess, as I wasn’t sure’



                      Joanna Anckar
             CALS, Jyväskylä University, Finland
   MC-tests and guessing
 Seldom completely random
 Informed guessing:
   – language use situations
   – problem solving in general
 Use affected by:
  – personality characteristics,
  – ability level
  – nature of the task
(Haladyna, 1994; Bachman & Palmer, 1996; Linn & Miller, 2005)

                                Joanna Anckar
                       CALS, Jyväskylä University, Finland
                                                             Number of cases
Pa
  rti
     al
               co
                 m
                      pr




                                          0
                                                  10
                                                            20
                                                                      30
                                                                                40
                                                                                      50
                                                                                           60
                                                                                                     70
                        eh
     Si                    e   ns
       ng                         ion
         le




                                                             19
                wo
                  rd
                         gr
                           as
         M               pe
                            d
          isu

                                                                           32
             nd
                er
                   st
                      an
                         di
         No                ng




                                                       10
            un
               de
                  rs
                     ta
                       nd
                          ing
                                                                                 39




                      Un
O                       ce
    pt                    rta
       ion
           s                     int
                                     y
               no
                                                                 20




                  t   un
                        de
                      rs
                                                                                                          10 types of guessing




        Ru               to
                            od
          l in
                                                                           31




              g-
                ou
                  ts
                      tra
                          te
         Se                 gy
                                                                                                64




              co
                nd
                   lis
                       te
                          nin
                              g
                                                            18




                       No
                              re
                                as
                                     on
               No
                                                   9




                      go
                         od
                              op
                                ti o
                                    n
                                              3
                                                   % of correct responses
Pa
  rti
     al
               co
                 m
                      pr




                                          0
                                              10
                                                   20
                                                         30
                                                                   40
                                                                        50
                                                                                   60
                                                                                        70
                                                                                             80
                        eh
     Si                    e   ns
       ng                         ion
         le




                                                                        47
                wo
                  rd
                         gr
                           as
         M               pe
                            d
          isu




                                                    19
             nd
                er
                   st
                      an
                         di
         No                ng
                                                                             50




            un
               de
                  rs
                     ta
                       nd
                          ing
                                                                        46




                      Un
O                       ce
    pt                    rta
       ion
           s                     int
                                     y
               no
                                                                             50




                  t   un
                        de
                     rs
        Ru              to
                           od
          lin
                                                                         48




             g-
               ou
                 ts
                     tra
                         te
         Se                gy
                                                              34




             co
               nd
                  lis
                      te
                         nin
                             g
                                                                                  55




                       No
                                                                                                  Type of guessing and success




                              re
                                as
                                     on
               No
                                                                                        67




                      go
                         od
                              op
                                tio
                                   n
                                                                                        67
Examples
’I heard the word restaurant and I made a
   guess’
’A half-guess, she said something about
   losing weight’
‘At least it can’t be A so either B or C. C is
   a guess…’
’Not a complete guess, nor a sure answer
   either’
                       Joanna Anckar
              CALS, Jyväskylä University, Finland
       Comments on guessing
           at a MC-test
 Guessing is indeed seldom random guessing,
  but is often based on available knowledge
 Guessing is not automatically a bad strategy
  associated with the MC-test format.
 Rather, bad items require bad guessing; a
  good item may lead to the use of the
  strategies of inferring, reasoning and
  elimination - ingredients of many
  communicative language use situations!

                       Joanna Anckar
              CALS, Jyväskylä University, Finland
         Joanna Anckar
CALS, Jyväskylä University, Finland