CNS186-FaceGaze&Object-030906 Faces, Gaze, and higher-order 1. Biological functions of faces - Expression / perception / memory. - Categorical perception / identification / emotional cognition. - Social interaction, non-verbal and verbal communication, attractiveness. - Biological significance of facial attractiveness? a) Darwinian b) Incidental effects of sensory bias from adaptation to object perception. c) Adaptationist view (Thornhill & Gangestad, '01); Facial features for attractiveness judgments as special-purpose adaptations to discriminate the mate value (i.e. fitness, health). Three lines of evidence 1) Symmetry. Early supporting evidence, though contaminated. (Rhodes, et al. '98, Mearly, et al. '99 ) 2) Averageness (related to Leopold's prototype theory; Rhodes, et al. '99) 3) Secondary sex traits as hormone markers. 2. Face perception as a specific "module" -Face has traditionally been regarded as a specific biological module, just as biological motion and speech perception. Though not as popular recently, some latest brain imaging evidence are not necessarily inconsistent with this view. - Evidence for specialty of face processing a) Holistic (neuropsychology, Lincoln's face, visual search, upside- down effect, etc.) b) Unusual memory (recognition) (Bahrick, et al. '75) c) Prosopagnosia (face selective cognitive deficit; Tipett, et al.'00) d) Development (infant studies; Slater & Quinn, '01, Mondloch, et al., '99, Lundy, '00) e) Specific brain locus (Electrophysiology; Rolls, Perret, etc.; fMRI & PET; Kanwisher, et al., ‘99), fusiform gyrus. 3. Face as a non-specific complex stimulus? -fMRI; counter-evidence against the face-specificity. The "face area" (the fusiform gyrus) may not be as specific as believed, it may be specialized for fine object identification (Gauthier, et al, '99, and many more). -Psychophysics; prototyping and averaging, "anti-Adam effect" (Leopold, '01). Faces behave just as other complex yet familiar visual stimuli. -”Subliminal” influence of facial expressions. A solid chunk of evidence, both from patients and normals. “Explicit processing evoked significantly more activity in temporal lobe cortex than implicit processing, whereas implicit processing evoked significantly greater activity in amygdala region.” (Critchely, et al., ‘00; Whalen, et al., ‘98) Also see Hacjin Kim’s unpublished data later. - A third view: distributed encoding/decoding. (????) 4. Gaze and emotional judgment -Are there somatic precursors preceding and preparing for conscious, cognitive emotional judgment, such as preference? - The answer is YES. Eg. Orienting behavior (gaze shift) (Shimojo, et al., '03) (a) Locations of face-selective neurons in the macaque (summary of 8 studies) (b) Locations of neurons selective for facial expressions (open circles), identitiy (close circles), both (half- filled), or interaction (squares). (Haxby, et al. ‘00) Locations of face-responsive regions in the fusiform gyrus (PET and fMRI studies) Haxby et al. (PET), Clark et al. (fMRI): face vs. non-sense pictures. Sergent et al. (PET): Facial identity task. Kanwisher et al., MacCarthy et al. (fMRI) : face vs. non-face. (Haxby, et al. ‘00) Regions activated by faceless animals and faces overlap. The former weaker, yet broader. (Haxby, et al. ‘00) Regions showing enhanced responses to birds or cars in experts. Right occipital & fusiform face- responsive regions (slices 2-4). Right & left parahippocampal place areas (slices 4,5). Specialized for faces, or specialized for fine pattern discrimination? (Haxby, et al. ‘00) There are face- and house-responsive regions identified in inferior occipital & ventral temporal areas. Inversion effect on the responses to houses in the house-responsive regions (b-left) shows the same pattern as the inversion effect on the responses to faces in the face-responsive regions. The only effect specific to the face inversion was an increased response to inverted faces in the house- responsive regions (a-right). (Haxby, et al. ‘00) The core system (circuit) for visual analysis of faces. (Haxby, et al. ‘00) (a) The intraparietal sulcus activated by averted gaze. (b) The amygdala activated by fear in the face. (c) The auditory superior temporal regions by hearing speech. (Haxby, et al. ‘00) (Haxby, et al. ‘00) Negative Face Aftereffect (Leopold et al., ‘01) Adaptation makes the neutral face appear to be an anti-face (negative aftereffect). Adapting to anti-face makes identification more sensitive. Inversion makes identification harder, but it still improves with adaptation (left). Translation invariance of face adaptation effect (right). Face Adaptation / Aftereffects 1. Identification (Leopolod, et al., ‘01) 2. Emotional expression (Halberstadt, et al., ‘01) 3. Attractiveness (Rhodes, et al., ‘02, ‘03) Face perception is just like other perception, such as luminance, color, orientation, shape, motion, depth, etc. 5. Object perception and recognition -How do humans and animals perceive object 3-D structure and identify it as a familiar object (from nearly 2-D retinal images)? - Prototype and familiy resemblance (Rosh) - Classical theory (Marr, '82): * 2 1/2 D-sketch matched with 3-D models of object in LTM. * Significance of canonical axis. * Cylinder model of object representation (Marr & Nishihara, '78) 6. Current issues in object perception (1) The issue of constancy: How could object recognition be accomplished over a wide range of view-dependent changes? (2) View-variant(i.e. image-based) vs. view-invariant processes. --> Geon theory (Biederman) -->Aspect graph (Koenderink) (3) Spatio-temporal liminations in detection of visual events. --> RB (Repetition Blindness) (Kanwisher) --> CB (Change Blindness) (Rensink) Emotion (1) Behavioral/physiological state (2) Experience (social psychology) (3) Expression (facial, bodily) (4) Perception / cognition (3), (4) ---> Social, non-verbal communication Perception (1) Hierarchy of neural visual pathways (2) Feature vs. holistic (3) Specialized vs. generic (4) Generation vs. perception (analysis-by-synthesis, mirror neuron)? (5) Explicit vs. implicit Emotion is contagious. (1) Mimicking facial expressions (yawning) (Yoshikawa, et al.) (2) Implicit priming by seeing faces (Shimojo, et al., ‘02) Seeing Is Liking : Gaze Cascade Effect towards Preference Shinsuke Shimojo1,2 , Claudiu Simion1, Eiko Shimojo3, Christian Scheier1 1- California Institute of Technology 2 - NTT Comm. Sci. Laboratories, Atsugi, Kanagawa, Japan 3 - Bunkyo Gakuin University, Saitama, Japan Supported by Genesis Research Institute, Inc. Introduction • Emotions operate along the dimension of approach or aversion. • Overt orienting may be intrinsically related to mental representation of the emotional state, i.e. feeling. • How they are causally related? • Classical psychology: does one cry because one feels sad or one feels sad because one cries? (cf. James-Lange Theory ) • Psychophysics: do we see something more because we like it, or we like it more because we see it? Broader Background 1. Mere exposure effect (Zajonc, 1980). 2. Two-stage theory of emotion (Schacter, 1964). 3. Preferential looking in infants (Fantz, 1959). 4. Perceptual memory and eye movements (Althoff, 1999). 5. Role playing paradigm in social psychology. Task: Compare freely and decide which face is more attractive? Question: Can we find any eye movement pattern prior to the decision which predicts it? Experimental paradigm: Stimulus; free inspection, no time limit Fixation point (1 s) Key press upon decision (dep. on task) Faces in a pair were matched for attractiveness (pre-experiment rating), gender, race and age. Task: Which face is more attractive? Facegen / EyeLink 2 : Example of Raw Data Easy calibration, high spatial-temporal resolution. Faces: Liking, Matched 0.9 Average for N=5 subjects Likelihood that the chosen face is inspected Interpolated curve - 4 parameter sigmoidal R2=0.91 Chance level (50%) max - 0.83 0.8 “cascade effect” 0.7 0.6 0.5 Decision 0.4 Decision 0.3 1.80 1.60 1.40 1.20 1.00 0.80 0.40 0.20 0.00 Time before decision (s) Face Pairs: Liking/Roundness/Disliking Effect is 1.0 not solely Likelihood that the chosen face is inspected “Like” due to selection 0.9 Max - 0.83 Not 0.8 “Round” explained by selection “Dislike” bias 0.7 R2=0.91 max - 0.62 0.6 R2=0.80 max - 0.57 0.5 0.4 Decision 0.3 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Time before decision (s) The “cascade effect” - possible interpretations 1) The effect could be artifactual: – Gaze bias = Attractiveness bias + Selection bias (not necessary to “create” preference) 2) Our hypothesis (VSS 01): Gaze bias - bodily basis for subjective decisions Exposure, perceptual facilitation Gaze Attractiveness “cascade effect” template bias Preferential looking Do we see the effect with stimuli other than faces? • Fourier Descriptor-generated shapes (abstract, symmetric, complex) (Sakai & Miyashita, 1991). • Unfamiliar stimuli (no direct memory)- decision should be more difficult. • Same experimental paradigm and task, but comparison is between abstract shapes. Fourier Descriptors: Liking Fourier descriptor pairs, attractiveness task 1.0 R2=0.98 Likelihood that the chosen face is inspected max - 0.95 0.9 0.8 Fourier Descriptors Larger cascade effect 0.7 0.6 Faces 0.5 0.4 Decision Decision 0.3 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Time before decision (s) Facilitation in the memory trace Preferential looking, Gaze bias, cognitive exposure assessment Eye movements (orient to the stimulus, inspect more) (1) Pre-rating of attractiveness --> (2) pairing. (3) Gaze Manipulation. Short (300 ms) 2 loop (4) Press a button 6 loops for preference. 12 loops - Longer gaze ---> more preference? - Mere exposure, or gaze itself? Long (900 ms) Results: In how many cases (40 total) was the longer shown face preferred? 2 alternations 6 alternations 12 alternations N 15 15 12 Mean 20.5 ( 51.2%) 23.6 (59%) 23.7 ( 59.2%) S.e.m. 0.95 (2.4%) 0.86 (2.2%) 1.18 (3.0%) p 0.3 0.0005 0.005 Non- parametric 9/15 12/15 (P<0.02) 9/12 (P<0.02) test Note Subjects don’t realize one face is on the screen Subjects realize the longer difference T test p<0.02 ---- 顔ディスプレイの例（スナップショット） 被験者の視線の動きに伴って、窓が動く (delay: 2ms)。 “Peep-hole” exp. Results Preference Consistent only with the Gaze intrinsic prediction Roundness Note the time scale！ Perception, gaze, and emotion are intermingled from the outset! S P C E R (local) (holistic) (matching w. M) S P C D E cf. Very quick feedback from the amygdala R to the visual cortex (teens of ms in EEG). Time-Frequency Diagram (Like-face task) Morlet wavelet analysis on EEG signals FRONT LEFT RIGHT BACK Large Gamma Band Responses in Frontal (Left-lateralized) Regions Time -frequency diagram based on Morlet wavelet Beta, Gamma analysis for coherence (-2s~) (EEG) Delta, Theta （-1s~） (Bhattacharya, Shams,& Shimojo, ‘02) Mid-Frontal Region (Fz) Like (abstract) Like (face) ERP Response (microV) Passive Clear separation! No separation Dislike Passive Round Time (sec) Button Press + Preference + Preference Variable interval (~3 sec) + Preference + Preference + Preference Brief Fixed duration (~100 msec) Preference * Presentation order random but balanced DECISION! in terms of number of repetitions Decision C3 U3 C2 U2 U1 C1 T3 T2 T1 C#: Chosen stimuli (C3 - U3) (C2 - U2) (C1 - U1) U#: Unchosen stimuli Predictions: Module 1 Module 2 Signal Difference (C - U) (Attractiveness encoding area) Early discrimination (Implicit?) Module 2 Module 1 (Preference decision making area) Late discrimination (Explicit?) T3 T2 T1 # of trials QuickTime™ and a TIFF (LZW) decomp resso r are need ed to see this picture. # of cycles C2 – U2 (preference) p < 0.005 Nucleus Accumbens (9, 6, -9): Z = 3.50, p < 0.001 (uncorrected) C1 – U1 (preference) p < 0.005 Medial OFC (-15, 27, -15): Motor Cortex (-39, 0, 15): Z = 3.78, p < 0.001 (uncorrected) Z = 3.94, p < 0.001 (uncorrected) C2 – U2 (preference) C1 – U1 (preference) Nucleus Accumbens (9, 6, -9): Medial OFC (-15, 27, -15): Z = 3.50, p < 0.001 (uncorrected) Z = 3.78, p < 0.001 (uncorrected) 1.5 NAC mOFC 1.0 0.5 0.0 -0.5 T2 T1 T2 T1 Preference Roundness F(1, 13) = 5.53, p < 0.05 F = 24.12, p < 0.001 2.0 2.0 1.5 1.5 1.0 1.0 0.5 0.5 0.0 0.0 NAC NAC -0.5 mOFC -0.5 mOFC MC MC -1.0 -1.0 T2 T1 R T2 T1 R Preference Roundness A Related fMRI Study (Kampe, et al., (2001) Nature, 413, 589.) No single brain region whose activity is correlated with attractiveness in general (although the ventral striatum showed a positive, when gaze in contact, More attractive when gaze and a negative, when gaze away, correlation with “in contact” (?). attractiveness). Both-Away Mixed-away (1)Gaze cascade in all cases. Both-forward (2)Pref. for away / forward. (3) Mixed-away is special. Mixed-forward Scenario of Co-evolution Actor Perceiver Bodily Orienting Perception of Gaze (eg. Eye Movements) Shimojo, Kampe, et al. et al. Social Behavior Preference/ (Approach to others) Attractiveness Perceiver Actor Implications (1) Development. (2) Clinical Psychology. (3) Neurology. (4) Robotics. (5) Media (CM, internet, A-V, etc.). Conclusions • The “cascade effect” seems to be embedded in the decision making mechanism. • It is modulated by task difficulty. • It is enhanced if the task involves deciding preference. • Manipulating gaze can bias preference. • Brain relies on own bodily reactions; orienting, in particular. References Biederman, I. (1987) Recoginition-by-components: A theory of human image understanding. Psychological Review, 94, 115-147. Bruce, V. and Young, A. (2000) In the eye of the beholder: The science of face perception. Part 1. British journal of psychology, 91, 141-142. Kanwisher, N. (2000) Domain specificity in face perception. Nature neuroscience, 3, 759- 763. Mondloch, C. J. , Lewis, T. L., Budreau, D. R., Maurer, D., Dannemiller, J. L., Stephens, B. R., Kleiner-Gathercoal, K. A. (1999) Face perception during early infancy. Psychological science, 10, 419-422. Palmer, S. E. (1999) Vision Science: Photon to Phenomenology, Cambridge, MA, MIT Press, Chap. 9, Perceiving function and category, 408-461. Shimojo, S., Simion, C., Shimojo, E., and Scheier, C. Seeing and Liking: Gaze Cascade towards Preference. Nature Neuroscience, 6, 1317-1322, 2003. Thornhill, R. & Steven W. Gangestad, S. W. (2001) Facial Attractiveness. Trends in Cognitive Science, 3, 452-460. Tippett, L. J., Miller, L. A., Farah, M. J. (2000) Prosopamnesia: A selective impairment in face learning. Cognitive neuropsychology, 17, 241-255. Whalen, P.J., Rauch, S. L., Etcoff, N. L., et al. (1998) Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411-418.
Pages to are hidden for
"Attention"Please download to view full document