Human visual object categorization is best described by

Document Sample
Human visual object categorization is best described by Powered By Docstoc
					                                                  Human visual object categorization is best described by a model with few stored exemplars
                                                  Robert J. Peters (1), Alex Backer (2), Fabrizio Gabbiani (3), and Christof Koch (1)
                                                  (1) Computation and Neural Systems, (2) Biology, Caltech, Pasadena, CA 91125
                                                  (3) Baylor College of Medicine, Houston, TX 77030

 1         Introduction                                                                                                                                                            5       Categorization models                                                                                                                                                                                                      5a       Detail: categorization mechanisms
                                                                                                                                                                                                                                                                                                                                                                                                                                   input image
•Models of visual object categorization:                                                                                                                                                                                             Boundary                         Exemplar models                                                                                                                                                                                          input units                       x1...xD        (eye height, eye separation, ...)
                                                                                                                                                                                   The RXM[1] has the                                                                 An exemplar model computes the sum of the similarities between a test
                                                                                                                                                                                                                                                                                                                                                                                                                             (vector in feature space)
           all-exemplar models (e.g., GCM; Nosofsky, 1991)                                                                                                                                                                           models                           exemplar and each of the stored exemplars for a given category.                                                                                                                                          weights
                                                                                                                                                                                   set of properties that                            Categories are represented as
                                                                                                                                                                                                                                                                                                                                                                                                                               1     2            ...      D                                                     w11...wDN
           prototype models (e.g., Reed, 1972)                                                                                                                                                                                       Gaussian distributions, so       prototype                          roaming-exemplar                                                           all-exemplar                                                                               In exemplar models, these are attentional weights, which

           decision boundary models (e.g., Maddox & Ashby, 1993)
                                                                                                                                                                                   allow it to best fit                              decision boundaries are linear   The one stored exemplar per        The stored exemplars are fitted to best match human behavior.              All of the training exemplars,                                                             typically vary only with the input unit, not with the hidden unit.
                                                                                                                                                                                                                                     or quadratic surfaces.           category is the mathematical       RXM[1]                                  RXM[N]                             regardless of their number, are
                                                                                                                                                                                   human behavior                                    linear boundary
                                                                                                                                                                                                                                                                      average of the training
                                                                                                                                                                                                                                                                      exemplars of that category.        One stored exemplar per category
                                                                                                                                                                                                                                                                                                                                                 The same number of stored          taken to be stored exemplars
                                                                                                                                                                                                                                                                                                                                                                                    in the model.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               In boundary models, these weights form boundaries in feature
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               space through their dot-products with the input vectors.
                                                                                                                                                                                                                                                                                                                                                 exemplars as training exemplars
•Very often the all-exemplar models win out—why?
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               hidden units                      z1...zN        (stored exemplars)

           memory capacity?                                                                                                                                                        categorization         category 1:                                                                                                                     S1-2                                                                                                                                 In exemplar models, the hidden units compute the weighted
                                                                                                                                                                                                      training exemplar                                                         P1                                                 S1-1                                                                                                                                        distance between a stored exemplar µ and the input pattern x:
           orientation of decision boundary?                                                                                                                                       mechanism stored exemplar/prototype                                 d
                                                                                                                                                                                                                                                                                                                                                                                                                        1      2     ...          ...      ...        N                                          zj = sigmoid{[            D   wij (xi - µ i)2]1/2}
                                                                                                                                                                                                                   category 2:
           shape of decision boundary?                                                                                                                                                                          training exemplar
                                                                                                                                                                                                                                                                                                                                                   S2-2                                                                                                                        In boundary models, the hidden units compute the distance of
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               input pattern x from the boundary normal to the weight vector wj:
                                                                                                                                                                                                                                                                                         P2                                                       S2-1
                                                                                                                                                                                                       stored exemplar/prototype                                                                                                                                                                                                                                                                                 zj = sigmoid(             wij xi)
                                                                                                                                                                                                                test exemplar
•Past comparisons of models have not resolved these factors
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               weights                           v0...vN
                                                                                                                                                                                   stored exemplars                                             none                            1, fixed                         1, roaming                               N, roaming                          N, fixed                                                                         These weights, through their sign and magnitude, reflect the
•We introduced the roaming exemplar model to help sort them out                                                                                                                                                                                                                                                                                                                                                                                                                strength with which each hidden unit is associated with one
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               category or the other. An extra weight v0 serves as a bias term.
                                                                                                                                                                                   main decision                   (orientation)               arbitrary                    constrained                            arbitrary                               arbitrary                       constrained
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               output unit                       y = sigmoid(          N   vj zj)
                                                                                                                                                                                   boundary                              (shape)                plane                            plane                               plane                                  curved                            curved
                                                                                                                                                                                                                                                                                                                                                                                                                              predicted probability

 2         Brunswik faces                                                                                                                                                          iso-probability lines                 (shape)                plane                           curved                              curved                                  curved                            curved

Eye separation (ES)                                                                                                                                                                goodness of fit                    rank (AIC)           2nd (177.3)                      4th (191.0)                         1st (176.6)                               5th (279.8)                     3rd (180.4)                 5b       Detail: model fits
                                                                                                                                                                                                                                                                                                                                                                                                                      Without correcting for free parameters,
Eye height (EH)                                                                                                                                                                    decision surfaces of                                                                                                                                                                                                               model fits improves with more memory.
                                                                                                                                                                                   fitted models
                                                                                                                                                                                                                                                                                                                                                                                                                                                     uncorrected                                  after correction
                                                                                                                                                                                   (four examples out of                    hf/45º                                                                                                                                                                                                             for # of free parameters                      for # of free parameters
Nose length (NL)                                                                                                                                                                   108 such data sets)                                                                                                                                                                                                                                              (minus loglikelihood)                      (Akaike information criterion)
                                                                                                                                                                                                                                                                                                                                                                                                                                           0      20     40      60       80    100      0     50    100   150    200   250     300

                                                                                                                                                                                                                                     ES                               ES                                ES                                       ES                                ES
 Mouth height (MH)                                                                                                                                                                                                                                                                                                                                                                                                            RXM[1]                                                                                                  These differences are
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      statistically significant,
                                                                                                                                                                                                                                          EH                               EH                                 EH                                      EH                                 EH                                   RXM[2]                                                                                                  though small in
                                                                                                                                                                                                                                                                                                                                                                                                                              RXM[3]                                                                                                  magnitude.
                                                                                                                                                                                             category 1:                                                                                                                                                                                                                      RXM[6]

                                                                                                                                                                                       training exemplar
           Categorization task                                                                                                                                                          stored exemplar
                                                                                                                                                                                                                                                                                                                                                                                                                             RXM[10 ]
                                                                                                                                                                                                                                                                                                                                                                                                                      linear boundary
1 Training phase                                       2 Testing phase                                                                                                                       category 2:                                                                                                                                                                                                                    prototype
   • uses category 1 & 2 training exemplars              like testing phase, except:                                                                                                   training exemplar                             ES                               ES                                ES                                       ES                                ES
   • presented one at a time in random order             • also uses additional test exemplars                                                                                                                                                                                                                                                                                                                                                                      After correcting for free parameters,
                                                                                                                                                                                        stored exemplar
   • subject guesses category 1 or 2                     • feedback is not given
   • auditory feedback is given                          • repeat until each exemplar is seen 7 times
                                                                                                                                                                                                                                          EH                               EH                                 EH                                      EH                                 EH                                                                the RXM[1] fits better than any other model.
                                                                                                                                                                                    p(category 1) = 90%
   • repeat until subject reaches 85% correct            • subjects' responses are used to fit the models                                                                           p(category 2) = 10%
                                                                                                                                                                                    p(category 1) = 50%
9 subjects did training and testing for each of 12 sets of categories                                                                                                               p(category 2) = 50%
                                                                                                                                                                                    p(category 1) = 10%
                                                                                                                                                                                    p(category 2) = 90%
                                                                                                                                                                                                                                     ES                               ES                                ES                                       ES                                ES
                                                                                                                                                                                                                                                                                                                                                                                                                      When memory capacity is accounted for with free parameters:
 4         Twelve sets of categories                                                                                                                                                                                                      EH                               EH                                 EH                                      EH                                 EH
                                                                                                                                                                                                                                                                                                                                                                                                                      • a model with low memory capacity accounts best for human
• 10 training      0º     15º       30º          45º                        0.75
                                                                                           boundary at 0º
                                                                                                                                                      boundary at 0º
                                                                                                                                                                                                                                                                                                                                                                                                                      performance in a subordinate-level categorization task
exemplars in
                                                           eye separation

                                                                                                                        mouth height


category 1                                                                                                                             -0.6                                                                                                                                                                                                                                                                           • a successful model with low memory capacity must have

                                                                                                                                                                                                                          jg/120º                                                                                                                                                                                     sufficient flexibility in its decision boundary
• 10 training                                                               0.45

exemplars in      60º     75º       90º         105º
category 2
                                                                                                                                                                                                                                                                                                                                                                                                                      • the success of all-exemplar models (such as the GCM) is due to
                                                                                   0.25     0.35   0.45   0.55   0.65                         0.25     0.35   0.45   0.55   0.65
                                                                                                                                                                                                                                     ES                               ES                                ES                                       ES                                ES
• 60 test
                                                                                              eye height

                                                                                          boundary at 135º
                                                                                                                                                        nose length
                                                                                                                                                                                                                                                                                                                                                                                                                      their relatively more flexible decision boundary, not to their high
                                                                                                                                                     boundary at 135º
                                                                                                                                       -0.5                                                                                               EH                               EH                                 EH                                      EH                                 EH                           memory capacity
                                                           eye separation

(not shown)
                                                                                                                        mouth height

                 120º    135º      150º         165º
                                                                                                                                                                                                                                                                                                                                                                                                                      • categorization may rely on a sparse representation that is
• 12 sets of

                                                                                                                                                                                                                                                           This work was supported by a Predoctoral Fellowship from the Howard Hughes Medical Institute to R.J. Peters, by the Engineering                            different from prototype abstraction

with different                                                              0.35
                                                                                                                                       -0.8                                                                                                                Research Centers Program of the National Science Foundation under Award Number EEC-9402726, by the National Institute of
boundaries                                                                         0.25     0.35   0.45
                                                                                              eye height
                                                                                                          0.55   0.65                         0.25     0.35   0.45
                                                                                                                                                        nose length
                                                                                                                                                                     0.55   0.65                                                                           Mental Health and by the W.M. Keck Foundation Fund for Discovery in Basic Medical Research at Caltech.                                                                                                                                      if you can read this you're standing too close!
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     presented 2001.Nov.12 at SFN, San Diego, CA