Neural Plasticity Mechanisms for Learning of Biological Motion
Jan Jastorff , Zoe Kourtzi & Martin Giese
1. ARL, Hertie Institute for Clinical Brain Research, Tübingen 2. MPI for Biological Cybernetics, Tübingen, Germany
1 2 1
MPI for Biological Cybernetics
Introduction
Psychphysical and neurophysiological studies suggest that human body motion presented as point-light displays can be readily recognized. These strongly impoverished stimuli are even sufficient to allow a discrimination of the underlying type of action, the gender and other details of the person. Earlier psychophysical work indicates that biological motion recognition is based on learning. The aim of this study was to determine the neural correlates of this learning process. Previous KO imaging studies using a classical block design MT+ have proposed a number of regions to be involved in biological motion processing. FFA Amongst them are early visual areas like VP and higher visual areas like hMT+/V5, , KO/V3B, FFA and STSp (e.g.[1,2]). In our experiments, we focused on the neuronal activity in the above areas applying an fMRI adaptation paradigm [3]. This paradigm has the advantage that it allows to distinguish multiple functionally distinct neural subpopulations within the same voxel, exploiting the fact that the BOLD signal decays if the same stimulus is presented repeatedly.
Functional Imaging
Trial Structure
Blank Center or Off-Center
Stimulus 2
Theoretical Modeling
Adaptation Paradigm
Stimulus 1 Stimulus 2
Classical Paradigm
Theoretical Model [5]
Motion Pathway
V1/2, MT
local motion detectors
Form Pathway
RF-Size
V1/2
local orientation detectors
Tim
e
Stimulus types:
“Center stimuli”: All 3 movements contribute with equal weights (1/3) to the morph “Off-center stimuli”: One movement contributes more to the morph than the two others.
fMRI Data
hMT +/V 5
R ebound Index
Off-Center Center
R ebound Index
Natural movements included different forms of locomotion, dancing, martial arts techniques, and aerobics To guarantee a high degree of naturalness, only triples from the same movement category were used for morphing (e.g. marching, running and limping)
K O/V 3B
1.4 1.2 1.0 PR E P OS T
1.0 PR E P OS T P OS T
W = connectivity matrix u = activity vector s = input signal f = step activatoin function m = [1,1,...1] T M = mmT
e ights ma x im > 0 u c ons ta m we ight pe r ne u r nt inhi on bition
Neuron #
*
*
g mu urin sti d
olsented trpre ng onot raini c li n t
1.4 1.2
*
*
l ented trores ing onot ptrain c li n g u im urin st d
S OM
) [6 ] ma p ing a niz org s e lf (
F eature
Motion morphing between triples of natural as well as synthetic patterns using spatiotemporal morphable models [4] Prototype
“Prototype“: One of the three move-ments contributing to the morph
Sp ST
Blank Center
Blank Stimulus 1 Time Blank Blank
Stimulus 1
Time
M(S)T, KO
complex OF detectors
V2, V4
invariant bar detectors
Experimental Design
Postscan
Behavioral Data
natural
100 75 50 25
identical different
synthetic
100 75 50 25
identical
STS, FFA, F5, ?
recognition layer
STS, FFA, F5, ?
recognition layer
*
*
Stimulus Generation
“natural“ movements
2D tracking of the trajectories of the joints of 21 movement pat-terns from video
Training on 3 consecutive days
competitive neural networks
competitive neural networks
Prescan
0
PR E
P OS T
0
different
PR E
P OS T
Learning of the feed-forward and feed-back connectivity
1
Learned feed-forward connections
1 0.8 0.6
...
...
10
20
0.4 0.2 0
30 1 20 40 60
Neuron #
H e bb
C o ns t ra ints : w
ia n L ea rni ng
1
Learned recurrent connections
1.2
[7 ]
20
0.8
0.4 40 0 60
-0.4 20 40 60
natural s ynthetic
P OS T
R ebound Index = different condition identical condition
1
Neuron #
References:
[1] Vaina, L.M. et al (2001) Functional neuroanatomy of biological motion perception in humans. PNAS. 25: 11656-11661 [2] Grossman, E.D. and Blake, R. (2002) Brain areas active during visual perception of biological motion. Neuron, 35: 1167-1175 [3] Grill-Spector, K. and Malach, R. (2001) fMRI-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychologica, 107: 293-321 [4] Giese, M.A. and Poggio, T. (2000) Morphable models for the analysis and synthesis of complex motion pattern. International Journal of Computer Vision, 38: 59-73 [5] Giese M.A. and Poggio T. (2003) Neural mechanisms for the recognition of biological movements and action. Nature Reviews Neuroscience 4, 179-192. [6] Kohonen, T. (1982) Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43: 59-69 [7] Jastorff J. and Giese M.A. (2004): Time-dependent hebbian rules for the learning of templates for visual motion recognition. In Ilg U, Bülthoff HH, Mallot H (eds): Dynamic Perception; Infix, Berlin, 151-156
“synthetic“ movements
R ebound Index
The synthetic skeleton model was dissimilar to any naturally occurring body structures Joint angles were animated with sinusoidal motion, amplitude and frequency approximately matched with human movements
FFA
1.4 1.2 1.0 PR E
*
R ebound Index
*
g mu urin sti d
l nted o trprese ng onot raini c li n t
1.4 1.2 1.0
S TS
*
Simulation results: Sequence selectivity
*
PR E
*
u ing stim dur
l nted o trprese ng onot raini c li n t
*
P OS T
P OS T
Tim
Results:
P OS T
P OS T
right temporal order
revers e temporal order
e
Support: DFG, Volkswagenstiftung, Max Planck Society
Discrimination learning results in consistent fMRI signal changes in several brain areas Emerging sensitivty for the differences in motion related areas KO/V3B and hMT+/V5 for natural and synthetic stimuli Enhanced sensitivity for the differences in biological motion realated areas STSp and (FFA) for natural stimuli Emerging sensitivity for the differences in areas FFA and STSp for the synthetic stimuli
Outlook:
The extended model will be used to simulate the measured BOLD activity changes. In this way different hypotheses about the underlying plasticity mechanisms can be studied theoretically.