Get documents about "Cognitive and Semantic Forms (CF SF)
Shared by: wantyou
-
Stats
- views:
- 10
- posted:
- 2/26/2009
- language:
- English
- pages:
- 18
Document Sample
Cognitive and Semantic Forms (CF & SF)
Grammaticization: From bag of tricks to systematic syntax
Karine Megerdoomian: Unlocking the CF of verbs
Unpacking syntactic categories: e.g., mass nouns versus
count nouns.
From scene components to constituents
Heine: in front of/behind
SF CF
PF
Language-Specific “Almost” Language-Independent
I will use the term SF for Semantic Form (not San Francisco!)
The idea is that this occupies the same place as LF in
the approach of many linguists, but emphasizes that Logic is more likely
to be a useful descriptive tool rather than a strict match for neural representations.
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 1
Performance Revisited Cognitive Structures
Observations: (Schema Assemblages) CF
Each form is distributed across
multiple brain regions.
Binding of subrepresentations is
required both within and across Semantic Structures
Forms. (Hierarchical Constituents
expressing objects, SF
The Problem of Serial Order:
Linking hierarchical constituents to
actions and relationships)
ordered expressive gestures.
Hypothesis:
Linkages of the Basal Ganglia to
multiple levels play a crucial role.
“Phonological” Structures
(Ordered Expressive PF
Gestures)
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 2
Extending the Mirror System
A subtle issue: We start with a clear distinction between the representation of the grasp
(the action/proto-verb) and the raisin (the thing/proto-noun) in the brain, but in the
spoken language both noun and verb are uttered by actions. Given this, how are we to
maintain the distinction between verb and noun?
Review the imaging data in this light.
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 3
A Key to Distinguishing CF from Linguistic Forms
Distinguishing the sign from the affordance or the schema
Recall: Two Roles for Imitation in the Evolution of Manual-Based Communication
1. Extending imitation from imitation of hand movements by hand movements to
pantomime which uses the degrees of freedom of the hand (and arm and body) to
imitate degrees of freedom of objects and actions other than hand movements.
Distinguishing the neural representation of the action or object per se (CF) from
the gesture which represents it (PF)
2. Extending these pantomime movements to to provide ad hoc gestures that may
convey to the observer information which is hared to pantomime in an “obvious”
manner. This requires extending the mirror system from the grasping repertoire to
mediate imitation of gestures to support the transition from ad hoc gestures to
conventional signs which can reduce ambiguity and extend semantic range.
The beginning of morphology - modifying a gesture to provide shadings of
meaning.
Such modifications may be ad hoc, yet become more systematic as historical
evolution regularizes certain of these constructions.
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 4
Activity of F5 canonical neurons is part of the code for
Command: Grasp-A(Object)
Object features
cIPS Object The Mirror Neuron
affordance System (MNS) Model
extraction F5canonical
7b: PF/PG Object affordance AIP Motor
-hand state program
Integrate
association temporal (Grasp)
association
Hand
shape
recognition Mirror Action Motor
Feedback recognition Motor execution
Hand (Mirror
motion program M1
Hand-Object Neurons)
detection (Reach)
spatial relation F5mirror
analysis F4
STS
7a
Object
location
The full neural representation of the “Cognitive Form” (CF): Grasp-A(Object)
requires not only the regions AIP and F5canonical shown in the MNS diagram, but also
inferotemporal cortex (IT) which holds the identity of the object.
How are these representations bound together?
NOTE: This is only the Cognitive Form. There are no “Linguistic Forms” in the monkey.
How are these in humans linked to the CF (assumed homologous to monkey’s)?
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 5
Activity of F5 mirror neurons is part of the code for
Declarative: Grasp-A(Agent, Object)
Object features
cIPS Object The Mirror Neuron
affordance System (MNS) Model
extraction F5canonical
7b: PF/PG Object affordance AIP Motor
-hand state program
Integrate
association temporal (Grasp)
association
Hand
shape Action
recognition Mirror Motor
Feedback recognition execution
Motor
Hand program M1
motion Hand-Object (Reach)
detection spatial relation F5mirror
analysis F4
STS
7a
Object
location
The full neural representation of the “Cognitive Form” (CF): Grasp-A(Agent, Object)
requires not only the regions AIP, STS, 7a, 7b and F5miirror shown in the MNS diagram, but
also inferotemporal cortex (IT) which holds the identity of the object and regions of STS (?)
not included in MNS which hold the identity of the agent.
How are these representations bound together?
NOTE: This is only the Cognitive Form. There are no “Linguistic Forms” in the monkey.
How are these in humans linked to the CF (assumed homologous to monkey’s)?
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 6
Beyond the Mirror to Neurolinguistics
Object features
cIPS Object
The Mirror Neuron
affordance System (MNS) Model
extraction F5canonical
7b: PF/PG Object affordance AIP Motor
-hand state Integrate program
association temporal (Grasp)
association
Hand
shape
recognition Mirror Action Motor
Feedback recognition Motor execution
Hand (Mirror
motion program M1
Hand-Object Neurons)
detection (Reach)
spatial relation F5mirror
analysis F4
STS
7a
Object
location
If the monkey needs so many brain regions for the mirror system for
grasping, how many more brain regions will we need for
an account of the language-ready brain that goes beyond the mirror
that goes far beyond the F5 Broca’s area homology to develop a full
neurolinguistic model linking CF, SF and PF ??
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 7
Towards a Computational Neurolinguistics
Cooperative computation in the brain: to make sense of data relating
different brain regions to different aspects of language.
Do these data reflect the brain's genetic prespecification and/or the
results of the self-organization of the infant brain when the infant
develops within a particular language community?
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 8
Cooperative Computation
The HEARSAY Paradigm for Speech Understanding
Woodja … ?
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 9
HEARSAY II (1976)
A Serial Implementation of
a Distributed Architecture:
Consider how it might
relate to the interaction of
multiple brain regions
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 10
A Simplistic View of Perceptual Schemas:
Constraint Satisfaction
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 11
Cooperative Computation
The VISIONS Paradigm for Visual Scene Analysis
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 12
A 20-year-old Overview of
Neurolinguistics:Luria (Arbib & Caplan)
B A
Selective Naming Visual Percept ion
____ ____
A summary of diagrams Tert iary L. Parieto-
Occipit al Zone
L. Temporo-Occipit al
Zones
developed by Arbib and C D
Caplan (1979) based on Switching Control
____
Articulat ory Syst em
____
Visual Input
Luria's (1973) analyses of Inferior Zone of
L. Premotor Cortex
Inferior Zone of
L. Post cent ral Cortex
Audit ory Input
Naming of objects; F" E H
Verbal expression of Updating the Plan
of the Expression
...
Phonemic Analysis
____
Lexical Analysis
____
____
Post erior Zone of
motives; Front al Lobes
Secondary Zone of
L. Temporal Cortex
L. Temporo-Occipit al
Region
Speech understanding; F G
I
Speech repetition Form at ion of the Speech Memory
Plan Format ion Linear Scheme ____
____ ____ Middle Zones of
Inferior Zone of L. Temporal Region
Front al Lobes L. Fronto-T emporal Deep Zones of
Cort ex L. Temporal Lobe
F' J
Active Analysis of Logical Scheme
Most Significant ____
Elements
____ L. Pariet o-Temporo-
Front al Lobes Occipit al Zones
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 13
Naming of objects
B A
Selective Naming Visual Perception
____ ____
Tertiary L. Parieto- L. Temporo-Occipital
Occipital Zone Zones
C D
Switching Control Articulatory System
____ ____
Visual Input
Inferior Zone of Inferior Zone of
L. Premotor Cortex L. Postcentral Cortex
Arbib and Caplan (1979)
based on Luria's (1973) analysis
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 14
Speech repetition
E
F"
Updating the Plan Phonemic Analysis
of the Expression ____
____ ...
Secondary Zone of
Frontal Lobes L. Temporal Cortex
Arbib and Caplan (1979)
based on Luria's (1973) analysis
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 15
Speech understanding
H
Lexical Analysis
____
Posterior Zone of
L. Temporo-Occipital
Region
I
Speech Memory
____
Middle Zones of
L. Temporal Region
Deep Zones of
L. Temporal Lobe
F' J
Active Analysis of Logical Scheme
Most Significant ____
Elements
____ L. Parieto-Temporo-
Frontal Lobes Occipital Zones
Arbib and Caplan (1979)
based on Luria's (1973) analysis
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 16
Verbal expression of motives
F G
Formation of the
Plan Formation Linear Scheme
____ ____
Inferior Zone of
Frontal Lobes L. Fronto-Temporal
Cortex
Arbib and Caplan (1979)
based on Luria's (1973) analysis
Arbib and Itti: CS 664 (University of Southern California, Spring 2002) Integrating Vision, Action and Language 17
Arbib and Caplan (1979) B Selective
A Visual
based on Luria's (1973) analysis Naming Perception
C D
Switching Articulatory
Control System Visual Input
Auditory Input
E
F" Updating Phonemic
H Lexical
the Plan of Analysis Analysis
...
the Expr’n
G
Formation
I Speech
F Plan of the Memory
Formation Linear
Scheme
Analysis of
J Logical
F' Scheme
Significant
Elements
Related docs
