Attention by yurtgc548



Faces, Gaze, and higher-order
1. Biological functions of faces

- Expression / perception / memory.
- Categorical perception / identification / emotional cognition.
- Social interaction, non-verbal and verbal communication,

- Biological significance of facial attractiveness?
 a) Darwinian
 b) Incidental effects of sensory bias from adaptation to object
c) Adaptationist view (Thornhill & Gangestad, '01);
 Facial features for attractiveness judgments as special-purpose
 adaptations to discriminate the mate value (i.e. fitness, health).

 Three lines of evidence
  1) Symmetry.
   Early supporting evidence, though contaminated.
   (Rhodes, et al. '98, Mearly, et al. '99 )
  2) Averageness (related to Leopold's prototype theory;
    Rhodes, et al. '99)
  3) Secondary sex traits as hormone markers.
2. Face perception as a specific "module"

-Face has traditionally been regarded as a specific biological module,
 just as biological motion and speech perception. Though not as
 popular recently, some latest brain imaging evidence are not
 necessarily inconsistent with this view.

- Evidence for specialty of face processing
 a) Holistic (neuropsychology, Lincoln's face, visual search, upside-
   down effect, etc.)
 b) Unusual memory (recognition) (Bahrick, et al. '75)
 c) Prosopagnosia (face selective cognitive deficit; Tipett, et al.'00)
 d) Development (infant studies; Slater & Quinn, '01, Mondloch,
   et al., '99, Lundy, '00)
 e) Specific brain locus (Electrophysiology; Rolls, Perret, etc.; fMRI
   & PET; Kanwisher, et al., ‘99), fusiform gyrus.
3. Face as a non-specific complex stimulus?

-fMRI; counter-evidence against the face-specificity. The "face area"
 (the fusiform gyrus) may not be as specific as believed, it may be
 specialized for fine object identification (Gauthier, et al, '99, and
 many more).

-Psychophysics; prototyping and averaging, "anti-Adam effect"
 (Leopold, '01). Faces behave just as other complex yet familiar
 visual stimuli.

-”Subliminal” influence of facial expressions. A solid chunk of
 evidence, both from patients and normals.
  “Explicit processing evoked significantly more activity in temporal lobe cortex than
   implicit processing, whereas implicit processing evoked significantly greater activity in
   amygdala region.” (Critchely, et al., ‘00; Whalen, et al., ‘98)
 Also see Hacjin Kim’s unpublished data later.
- A third view: distributed encoding/decoding. (????)

4. Gaze and emotional judgment

-Are there somatic precursors preceding and preparing for
 conscious, cognitive emotional judgment, such as preference?

- The answer is YES. Eg. Orienting behavior (gaze shift)
  (Shimojo, et al., '03)
(a) Locations of face-selective neurons
   in the macaque (summary of 8

(b) Locations of neurons selective for
 facial expressions (open circles),
 identitiy (close circles), both (half-
 filled), or interaction (squares).

                     (Haxby, et al. ‘00)
Locations of face-responsive
regions in the fusiform gyrus
(PET and fMRI studies)

Haxby et al. (PET), Clark et al.
(fMRI): face vs. non-sense pictures.

Sergent et al. (PET): Facial identity

Kanwisher et al., MacCarthy et al.
(fMRI) : face vs. non-face.

             (Haxby, et al. ‘00)
Regions activated by faceless animals
and faces overlap. The former weaker,
yet broader.

               (Haxby, et al. ‘00)
Regions showing enhanced
responses to birds or cars in

Right occipital & fusiform face-
responsive regions (slices 2-4).

Right & left parahippocampal
place areas (slices 4,5).

Specialized for faces, or
specialized for fine pattern

              (Haxby, et al. ‘00)
There are face- and house-responsive
regions identified in inferior occipital
& ventral temporal areas.

Inversion effect on the responses to
houses in the house-responsive regions
(b-left) shows the same pattern as the
inversion effect on the responses to
faces in the face-responsive regions.

The only effect specific to the face
inversion was an increased response
to inverted faces in the house-
responsive regions (a-right).

                (Haxby, et al. ‘00)
The core system (circuit)
for visual analysis of faces.

           (Haxby, et al. ‘00)
(a) The intraparietal sulcus activated by averted gaze.
(b) The amygdala activated by fear in the face.
(c) The auditory superior temporal regions by hearing speech. (Haxby, et al. ‘00)
(Haxby, et al. ‘00)
Negative Face Aftereffect (Leopold et al., ‘01)
Adaptation makes the neutral face appear to be an anti-face
(negative aftereffect).
Adapting to anti-face makes identification more sensitive. Inversion makes
identification harder, but it still improves with adaptation (left).
Translation invariance of face adaptation effect (right).
                  Face Adaptation / Aftereffects

         1. Identification (Leopolod, et al., ‘01)

         2. Emotional expression (Halberstadt, et al., ‘01)

         3. Attractiveness (Rhodes, et al., ‘02, ‘03)

Face perception is just like other perception, such as luminance,
color, orientation, shape, motion, depth, etc.
5. Object perception and recognition

-How do humans and animals perceive object 3-D structure and
 identify it as a familiar object (from nearly 2-D retinal images)?

- Prototype and familiy resemblance (Rosh)

- Classical theory (Marr, '82):
 * 2 1/2 D-sketch matched with 3-D models of object in LTM.
 * Significance of canonical axis.
 * Cylinder model of object representation (Marr & Nishihara, '78)
6. Current issues in object perception

(1) The issue of constancy:
 How could object recognition be accomplished over a wide range
 of view-dependent changes?

(2) View-variant(i.e. image-based) vs. view-invariant processes.
    --> Geon theory (Biederman)
    -->Aspect graph (Koenderink)

(3) Spatio-temporal liminations in detection of visual events.
    --> RB (Repetition Blindness) (Kanwisher)
    --> CB (Change Blindness) (Rensink)
(1)   Behavioral/physiological state
(2)   Experience (social psychology)
(3)   Expression (facial, bodily)
(4)   Perception / cognition
        (3), (4) ---> Social, non-verbal communication

(1) Hierarchy of neural visual pathways
(2) Feature vs. holistic
(3) Specialized vs. generic
(4) Generation vs. perception (analysis-by-synthesis,
     mirror neuron)?
(5) Explicit vs. implicit
              Emotion is contagious.

(1) Mimicking facial expressions (yawning)
      (Yoshikawa, et al.)

(2) Implicit priming by seeing faces (Shimojo, et al.,
       Seeing Is Liking
: Gaze Cascade Effect towards

              Shinsuke Shimojo1,2 ,
Claudiu Simion1, Eiko Shimojo3, Christian Scheier1
              1- California Institute of Technology
   2 - NTT Comm. Sci. Laboratories, Atsugi, Kanagawa, Japan
         3 - Bunkyo Gakuin University, Saitama, Japan

   Supported by Genesis Research Institute, Inc.
• Emotions operate along the dimension of approach or aversion.

• Overt orienting may be intrinsically related to mental representation
  of the emotional state, i.e. feeling.

• How they are causally related?

   • Classical psychology: does one cry because one feels sad or
     one feels sad because one cries? (cf. James-Lange Theory )

   • Psychophysics: do we see something more because we like
     it, or we like it more because we see it?
                 Broader Background

1. Mere exposure effect (Zajonc, 1980).

2. Two-stage theory of emotion (Schacter, 1964).

3. Preferential looking in infants (Fantz, 1959).

4. Perceptual memory and eye movements (Althoff, 1999).

5. Role playing paradigm in social psychology.
Task: Compare freely and decide which face is more attractive?

 Question: Can we find any eye movement pattern prior to the
                decision which predicts it?
  Experimental paradigm:
                            Stimulus; free inspection, no time limit

  Fixation point (1 s)

                                                         Key press upon
                                                      decision (dep. on task)
Faces in a pair were matched for attractiveness
(pre-experiment rating), gender, race and age.

Task: Which face is more attractive?
  Facegen / EyeLink 2 : Example of Raw Data

Easy calibration, high spatial-temporal resolution.
                                                                    Faces: Liking, Matched
                                                            Average for N=5 subjects
Likelihood that the chosen face is inspected

                                                            Interpolated curve - 4 parameter sigmoidal                            R2=0.91
                                                            Chance level (50%)                                                    max - 0.83

                                                                                     “cascade effect”



                                               0.4                                                                     Decision

                                                     1.80    1.60    1.40     1.20     1.00     0.80     0.40   0.20       0.00

                                                                            Time before decision (s)
                                                           Face Pairs: Liking/Roundness/Disliking                                  Effect is
                                               1.0                                                                                 not solely
Likelihood that the chosen face is inspected

                                                                                                         “Like”                    due to
                                                                                                                  Max - 0.83
                                                                                        “Round”                                     explained
                                                                                                                                   by selection
                                                                    “Dislike”                                                          bias
                                                                                                                                   max - 0.62
                                                                                                                                   max - 0.57

                                               0.4                                                                      Decision

                                                     1.6      1.4       1.2       1.0     0.8    0.6       0.4    0.2        0.0
                                                                              Time before decision (s)
   The “cascade effect” - possible interpretations

1) The effect could be artifactual:
    – Gaze bias = Attractiveness bias + Selection bias
      (not necessary to “create” preference)

2) Our hypothesis (VSS 01): Gaze bias - bodily basis for
   subjective decisions

                  Exposure, perceptual facilitation

    Gaze                                              Attractiveness
                       “cascade effect”                 template
                        Preferential looking
Do we see the effect with stimuli other than faces?
• Fourier Descriptor-generated shapes (abstract, symmetric,
  complex) (Sakai & Miyashita, 1991).
• Unfamiliar stimuli (no direct memory)- decision should be
  more difficult.

• Same experimental paradigm and task, but comparison is
  between abstract shapes.
                                                             Fourier Descriptors: Liking
                                                           Fourier descriptor pairs, attractiveness task
                                               1.0                                                                            R2=0.98
Likelihood that the chosen face is inspected

                                                                                                                              max - 0.95

                                                                         Fourier Descriptors
                                                                        Larger cascade effect


                                               0.4                                                               Decision

                                                     1.6   1.4    1.2      1.0     0.8     0.6     0.4     0.2       0.0
                                                                        Time before decision (s)
Facilitation in the memory trace

   looking,               Gaze bias,
   cognitive              exposure

            Eye movements
(orient to the stimulus, inspect more)
(1) Pre-rating of attractiveness --> (2) pairing.
(3) Gaze Manipulation.

       Short (300 ms)               2 loop
                                                    (4) Press a button
                                   6 loops             for preference.
                                   12 loops

                                  - Longer gaze
                                     ---> more preference?
                                  - Mere exposure, or gaze itself?
       Long (900 ms)
In how many cases (40 total) was the longer shown face preferred?

                2 alternations            6 alternations         12 alternations
    N                 15                       15                       12
 Mean           20.5 ( 51.2%)              23.6 (59%)            23.7 ( 59.2%)

 S.e.m.          0.95 (2.4%)              0.86 (2.2%)              1.18 (3.0%)
    p                0.3                     0.0005                    0.005
parametric           9/15                 12/15 (P<0.02)           9/12 (P<0.02)
  Note       Subjects don’t realize one face is on the screen   Subjects realize the
                                  longer                            difference

 T test                          p<0.02                                 ----

被験者の視線の動きに伴って、窓が動く (delay: 2ms)。
          “Peep-hole” exp. Results


Consistent only with the
Gaze intrinsic prediction


                            Note the time scale!
Perception, gaze, and emotion are intermingled from the outset!

  S             P                       C                    E         R
(local)      (holistic)         (matching w. M)

      C                                                          D

                          cf. Very quick feedback from the amygdala
      R                       to the visual cortex (teens of ms in EEG).
         Time-Frequency Diagram (Like-face task)
          Morlet wavelet analysis on EEG signals


                                                 LEFT           RIGHT


Large Gamma Band Responses in Frontal (Left-lateralized) Regions
Time -frequency diagram
based on Morlet wavelet             Beta, Gamma
analysis for coherence              (-2s~)

                     Delta, Theta

   (Bhattacharya, Shams,&
    Shimojo, ‘02)
                           Mid-Frontal Region (Fz)
                                  Like (abstract)

                                                        Like (face)
ERP Response (microV)


                                       Clear separation!

                                        No separation

                        Passive                 Round

                                         Time (sec)                   Button Press
   +    Preference

            +    Preference                           Variable interval (~3 sec)


                                               +   Preference

                                                                  +   Preference
  Brief Fixed duration (~100 msec)
* Presentation order random but balanced              DECISION!
  in terms of number of repetitions
 C3               U3   C2               U2                               U1               C1

         T3                    T2                                                T1            C#: Chosen stimuli
      (C3 - U3)             (C2 - U2)                                         (C1 - U1)        U#: Unchosen stimuli

Module 1
                                                                                                    Module 2

                                             Signal Difference (C - U)
(Attractiveness encoding area)
 Early discrimination (Implicit?)

Module 2                                                                                            Module 1

(Preference decision making area)
 Late discrimination (Explicit?)                                         T3              T2            T1
# of trials

                     QuickTime™ and a
                 TIFF (LZW) decomp resso r
              are need ed to see this picture.

                       # of cycles
  C2 – U2 (preference)
       p < 0.005

Nucleus Accumbens (9, 6, -9):
Z = 3.50, p < 0.001 (uncorrected)
                            C1 – U1 (preference)
                                 p < 0.005

Medial OFC (-15, 27, -15):                    Motor Cortex (-39, 0, 15):
Z = 3.78, p < 0.001 (uncorrected)             Z = 3.94, p < 0.001 (uncorrected)
   C2 – U2 (preference)                              C1 – U1 (preference)

Nucleus Accumbens (9, 6, -9):                 Medial OFC (-15, 27, -15):
Z = 3.50, p < 0.001 (uncorrected)             Z = 3.78, p < 0.001 (uncorrected)



                        T2          T1   T2           T1
                        Preference        Roundness               F(1, 13) = 5.53, p < 0.05
        F = 24.12, p < 0.001

2.0                                       2.0

1.5                                       1.5

1.0                                       1.0

0.5                                       0.5

0.0                                       0.0
                                   NAC                                NAC
-0.5                               mOFC   -0.5                        mOFC
                                   MC                                 MC
-1.0                                      -1.0
       T2         T1           R                 T2     T1        R
               Preference                             Roundness
         A Related fMRI Study (Kampe, et al., (2001) Nature, 413, 589.)

                               No single brain region whose activity is correlated
                               with attractiveness in general (although the ventral
                               striatum showed a positive, when gaze in contact,
More attractive when gaze      and a negative, when gaze away, correlation with
    “in contact” (?).          attractiveness).
(1)Gaze cascade in all cases.                   Both-forward
(2)Pref. for away / forward.
(3) Mixed-away is special.

                     Scenario of Co-evolution

             Actor                         Perceiver

      Bodily Orienting                   Perception of Gaze
    (eg. Eye Movements)

Shimojo,                                                Kampe,
et al.                                                  et al.

                                         Social Behavior
        Preference/                    (Approach to others)

           Perceiver                            Actor

(1) Development.

(2) Clinical Psychology.

(3) Neurology.

(4) Robotics.

(5) Media (CM, internet, A-V, etc.).
• The “cascade effect” seems to be embedded in the decision
  making mechanism.
• It is modulated by task difficulty.
• It is enhanced if the task involves deciding preference.

• Manipulating gaze can bias preference.

• Brain relies on own bodily reactions; orienting,
  in particular.
Biederman, I. (1987) Recoginition-by-components: A theory of human image
 understanding. Psychological Review, 94, 115-147.
Bruce, V. and Young, A. (2000) In the eye of the beholder: The science of face perception.
 Part 1. British journal of psychology, 91, 141-142.
Kanwisher, N. (2000) Domain specificity in face perception. Nature neuroscience, 3, 759-
Mondloch, C. J. , Lewis, T. L., Budreau, D. R., Maurer, D., Dannemiller, J. L., Stephens,
 B. R., Kleiner-Gathercoal, K. A. (1999) Face perception during early infancy.
 Psychological science, 10, 419-422.
Palmer, S. E. (1999) Vision Science: Photon to Phenomenology, Cambridge, MA, MIT
 Press, Chap. 9, Perceiving function and category, 408-461.
Shimojo, S., Simion, C., Shimojo, E., and Scheier, C. Seeing and Liking: Gaze Cascade
 towards Preference. Nature Neuroscience, 6, 1317-1322, 2003.
Thornhill, R. & Steven W. Gangestad, S. W. (2001) Facial Attractiveness. Trends in
 Cognitive Science, 3, 452-460.
Tippett, L. J., Miller, L. A., Farah, M. J. (2000) Prosopamnesia: A selective impairment in
 face learning. Cognitive neuropsychology, 17, 241-255.
Whalen, P.J., Rauch, S. L., Etcoff, N. L., et al. (1998) Masked presentations of emotional
 facial expressions modulate amygdala activity without explicit knowledge. Journal of
 Neuroscience, 18, 411-418.

To top