Anaphoric dependencies:
A window into the architecture of
the language system
Sergey Avrutin
Eric Reuland
Frank Wijnen
Olga Khomitsevitch
Arnout Koornneef
Natalia Slioussar
Nada Vasic
Goals of the course
• Explain how language structure and
neurocognitive organization can meet
• Provide some background on current
approaches
• Present a number of issues on which current
discussions focus
Overview
• Fundamentals of Linguistics against a
neurocognitive background
• The grammatical encoding of anaphoric
dependencies
• Linguistic architecture and cognitive
architecture:
– Language processing
– Language impairment
• Acquired
• Congenital
What is Language?
Language: Systematic relation between
• Forms: events in an (external) medium
(sound, gesture, ink on paper)
&
• Interpretations: change in information
state of the mind
How is language represented in
the brain?
• Reflects general issue of the division of
labour between brain areas
– Modularity
– Functionality
– Plasticity
Theoretical approach: Minimalist Program
The minimal language system
PF interface C-I interface
Sensori- CHL Interpretation
Motor system system (system
of thought)
Lexicon
- dedicated +dedicated(?) -dedicated
Levelt : Speaking (1989)
Task: find map between
linguistic operations and
neurocognitive processes
PF-interface
|
Computational system of
Human Language
(CHL) (+Lexicon) ?
|
Conceptual-Intentional
Interface (C-I
interface)
The triangle of cognitive
neuroscience (Hagoort 2003)
Computational
model
Neuro-
physiology
Cognitive Neural
Archtecture Architecture
Neuro
anatomy
Behaviour
A note on method
• The brain is doing a lot at the same time
• In whatever you try to measure, you will
find a lot of noise
• no escape from forming precise (and
falsifiable) hypotheses: a theory is your
eyes - without it you are blind
On the relation between
linguistics and psycholinguistics
"The split between linguistics and psycholinguistics in the
1970‟s has been interpreted as being a retreat by linguists
from the notion that every operation of the grammar is a
mental operation that a speaker must perform in speaking
and understanding language.
But, putting history aside for the moment, we as linguists
cannot take the position that there is another way to
construct mental representations of sentences other than
the machinery of grammar.
....There is no retreat from the strictest possible interpretation
of grammatical operations as the only way to construct
linguistic representations" (Alec Marantz, lecture notes
2000)
Correspondence Thesis
• Differences between operations within
(major) modules of the grammatical
system correspond with differences in
processes at the neural level and vice
versa. (Reuland 2003)
In plain language
• We have to figure out what the brain does in
order to be able to figure out how the brain
does it
• We have to figure out how the brain does
things in order to figure out how it can do
what it does
• the main danger is not being precise
enough on either side
Tensions between requirements
on Linguistic descriptions
• What do we need for easy description?
• What do we need for explanation?
• Compare
- Quantum physics
- Newtonian mechanics
For understanding planetary motion
For understanding why there are no white holes
An example: how local are the
dependencies we (can) compute?
• What did John see?
• What did John see –
• What did John [ [see - ] ]
Issues of this type may occasionally seem
abstract but are crucial for our
understanding
Current views on modularity
Is there a division of labour between brain areas?
Answer: There is specialisation
• Lateralisation: left-right asymmetry
• Specialized areas of the cortex:
- Motor-cortex
- Visual cortex
- Auditory cortex
Etc.
Specialisation within areas
Example: (Kandel et al. 2000, Principles of Neural
Science; Ch 28):
Visual system: specific neurons for:
• Black/white detection
• Colour detection
• Form detection
• Depth detection
• Movement detection
• Facial recognition
FUNCTIONS & functions
Kosslyn & König (1995) The Wet Mind:
• FUNCTIONS (Vision, Hearing, etc. ) v.s.
functions (movement detection, depth
detection, etc.)
• Binding problem: How do the different
functions lead to one unified perception?
Language: FUNCTION vs functions
Is there one language system? Or:
• Are there different subsystems that contribute to
the FUNCTION Language?
• If so, what are the „functions‟ subserving
language?
• How elementary are these functions?
• Are there any „functions‟ dedicated to language?
Summary of the task
• Precise analysis of the operations needed to
capture the structure of language
• Match these operations with real time
processes in the brain
• Identify brain areas involved in these
processes
Methods
1. Grammar: Precise modeling of structure and
interpretation
2. Studying “Experiments of nature”: language
impairment (genetic, acquired)
3. Behavioral studies: complexity, processing
resources
4. Eye-tracking
5. ERP
6. fMRI
7. PET
Required for explanation
• What do we minimally need to account for
language structure?
• What do we minimally need to assume is
dedicated to language?
• Behind these questions:
– What kind of elements and what type of
properties can be plausibly represented in the
brain?
The minimalist program
Start out assuming:
• What has to be the case by conceptual
necessity (and no more)
• Add no enrichment to the system unless
empirically unavoidable
• But be as precise as possible
Lexicon
Lexicon: Atomic form-meaning combinations (morphemes)
Each morpheme contains
- phonological information how to pronounce
(phonology: Sound system will not be discussed here)
- grammatical information category (Noun, Verb, Adj.,
Preposition, etc.), features (person, gender, number, Case,
etc.)
- semantic information concept, instructions for
computation (quantifiers: every, a, some, etc.)
The basic combinatory process
BINARY OPERATION Merge: a , b a b
But:
Not all combinations of morphemes are possible
words
• *real, real-hood, real-able, real-ish, ...
• *work, work-ish, work-hood,...
• *hold, holded, up-holded | hold, held, up-held
• *boy, boy-en | boy, boys
Morphemes select what they combine with
Hierarchical relations:
Tree structures
A
N A
boy -ish
N
A
A A N
un- happy ness
[N [A unA- [A happy]] –nessN]
Syntax: the computational system
Basic operation: Merge a,b (= combine a,b)
Each combination has a head represented in structure
• [A grey] + [N mice] [NP [A grey] + [N mice] ]
the N mice is the head of the Noun Phrase
grey mice
• [V feed] + [NP [A grey] [N mice] ]
[VP [V feed] [NP [A grey] [N mice] ]]
the V feed is the head of the Verb Phrase
An elementary Tree structure
V(P)
N(P)
V A N
feed grey mice
Basic clause structure 1
Three types of information
1. What happened to whom? Core Predicate
- (I saw) Mary feed the cat
2. When did it happen? Tense/Mood
- Mary will feed the cat
3. Force: Assertion, Question, Command
- (I saw) that Mary fed the cat
- (I wondered) if Mary fed the cat
- (I wondered) who Mary fed
Basic clause structure 2
Force in root clauses
• Ø Mary will feed the cat
• Will Mary __ feed the cat
• Who will Mary __ feed __
Forming questions requires dislocation
Some more examples
loveV, BonzoN VP
loveV BonzoN
John, [VP loveV BonzoN] VP
V‟
JohnN loveV BonzoN
V‟ indicates that the construction of the verb phrase continues
Terms: Head, complement, specifier
Selection
Selection restricts possible combinations
Syntactic selection:
• D selects NP, T selects VP, C selects TP
Semantic selection:
• John loves Mary
• ??The brick loves Mary
• John opened the lock/the key opened the lock
• ??Serenity opened the lock
Dependencies
• A fundamental property of all human languages:
Dependency Relations.
• Local: Semantic roles, Case, agreement, category
selection
(functional-lexical: D-NP, T-VP)
• Non-local: dislocation, anaphors, pronominals
• Dependencies are always constrained must be
obeyed in putting expressions together
Putting expressions together
(I saw) John feed Fluffy (bare VP)
(I expect) John to feed Fluffy (to + VP but!! mismatch)
John will feed Fluffy (T+VP, T takes over, but!! mismatch)
John feeds Fluffy (T+VP, but !! mismatch)
TP
T'
T VP
will/to V'
John V
feed Fluffy
Rearranging elements
(I saw) [John [feed Fluffy]] (bare VP)
------ [John [to [(John) feed Fluffy]]] (to+VP+rearrangement)
[John [will [(John) feed Fluffy]]] ([Twill]+VP+rearrangement)
[John [(-s) [ (John) feeds Fluffy]]] ([T –s] +VP+ rearr.)
TP
John T'
T VP
to/will/-s V'
(John) V
feed Fluffy
Dislocation 1
Dislocation: Mismatch between positions of
interpretation and position of realization
• Metaphorical term: Movement
• Dislocation/Movement expresses Double Duty:
Essence: One and the same element is active in
two (or more) positions and realized in only
one position.
Dislocation 2
The specifier of T must be filled:
• it will rain
• there arrived a man
Dual use: re-use an element from the structure
TP
He T'
T VP
will V'
(he) V
love Mary
Adding Force: CP 1
• (I thought) [that [TP John would love *(her)]]
(........) CP declarative marker: that
---- C'
C TP
that T'
John T VP
would V'
(John) loveV her
Expressing questions: CP 2
• (Mary wondered) [CP ifC [TP John would love her]]
(........) CP Question marker added
---- C'
C TP
if T'
John T VP
would V'
(John) loveV her
Expressing Questions: CP 3
• (Mary wondered) [whom [TP John would love]]
(........) CP
whom C'
C TP
- T'
John T VP
would V'
(John) loveV (whom)
How to express dislocation?
• (Mary wondered) [whomi [TP John would love - ]]
(........) CP
whomi C'
C TP
- T'
John T VP
would V'
(John) loveV -
The canonical trace notation
• (Mary wondered) [whomi [TP John would love ti]]
(........) CP
whomi C'
C TP
- T'
John T VP
would V'
(John) loveV ti
The status of traces
• What do traces represent?
• What kind of elements are they?
• Are they needed? If so, why?
Answer in Minimalist Program:
• Double duty can be expressed without an
additional element in the theory
• Copies can do the same job
Merge: Internal/External traces only for
convenience
Questions in root clauses
• Whom did [John love t]
CP
whomh C'
C TP
didj Johni T'
T VP
tj V'
ti V th
love
Clausal layers
Predicational core: verb + arguments
Tense/mood layer: coordinates for evaluation
Force layer (C): assertion, question, command
Movement enables one and the same element to be used in more
than one layer
• Whomi did [John love ti]
Whom: argument of love in predicational core; signals question in
Force domain
Did: carrier Tense in Tense/mood layer; identifies C in Force
domain
Wh-movement as an interpretive
dependency
The interpreter must crucially know:
i) a wh-element up front of the clause is part of the
Force layer, and must therefore be interpreted as
signalling a question;
ii) a wh-element up front must be related to a gap
(a trace, silent copy, etc.) and his computational
system must be able to figure out where that gap is.
Some questions and relatives
Wh-movement: Movement to a Force position
(non-argument: no semantic role, no Case)
• Question formation and relativization
I wonder [CPwhich mani [ ti read the book]]
I wonder [CPwhich booki [the man read ti]]
Subject versus object relatives:
I admired the man [CPwhoi [ti wrote the book]]
I admired the book [CPthati [the man wrote ti]]
Structure and processing
• What would you predict about the
representation of
– functional structure versus
– core predication
by subjects with reduced processing capacity?
• They will be selective: functional structure
affected
Common denominator of
processing models
• Modularity
– language results from a number of specialized
components responsible for different aspects of
language representation/processing
• Major divisions
– form (syntax) vs. meaning (semantics) vs. use
(discourse)
• Hypothesis
– Syntax, meaning, and use are subserved by different
types of processes
• Investigation tool
– Dissociability of processing mechanisms
Evidence: Neurolinguistics
• The study of brain – language relationships through
neurological deficits
• Prime example: aphasia
– A deficit in producing and understanding spoken and written
language due to focal brain damage in persons who have
gone through normal language development.
(Prins & Bastiaanse 1997)
– Incidence
• approx. 6000 new cases per year in NL
• approx. 20.000 patients in NL
• Founding fathers of aphasiology:
– Paul Broca
– Carl Wernicke
Paul Broca (1824-1880)
• 1861: Broca discovers in a post mortem study
(Monsieur Tan) that speech/language
(production) is associated with the foot of the 3rd
convolution of the frontal lobe
– Brodmann‟s areas 44 & 45
• today, we call this Broca’s area
Mr. Tan‟s brain
Carl Wernicke (1848-1904)
• 1874: Wernicke discovers a second cortical area
connected to language: the posterior part of the
uppermost temporal gyrus (STG), right behind the
primary auditory cortex
– Brodmann‟s area 22
• today: Wernicke’s area
Brodmann’s areas
Korbinian Brodmann
(1868-1918)
Aphasia: syndromes
concepts
motor images word images
Wernicke-Lichtheim-Geschwindt
Agrammatism and the CP-layer
Question production in agrammatism: The Tree pruning
hypothesis, Naama Friedman, Tel Aviv University, Brain
and Language 80, 160-187
Patients: Hebrew versus English speakers with similar
brain injuries
Variable: Wh-questions versus Yes-no Questions
English: both involve C
Hebrew: only Wh-questions involve C
Observations and results
Wh-questions (similar for Hebrew and Arabic)
(1) H: Miri mecaryeret portret
E: Miri draws a portrait
(2) H: Mai Miri mecayeret ti : Wh moves to C
E: What (does) Miri paint: Wh moves to C
Yes/no questions
• Usually differ from declarative sentences in intonation only no
involvement of C
(3) H: Miri mecaryeret portret
E: Does Miri draw a portrait: T moves to C
Result: In Hebrew only Wh-questions were affected; in English both were
structure is reflected in pathology
Explanation
• Tree pruning hypothesis: The highest nodes of the
syntactic tree are inaccessible in agrammatism:
CP
whh C'
C TP
Tj Mirii T'
T VP
tj V'
ti V th
draw
Dependencies: Passive
Movement into the subject position
T'
T VP
was V'
V' DP
chased the mouse
Passive morphology:
- no semantic role to the subject
- no case for the object
- double use of the object requirement to move into the
Tense system
Dependencies: Passive 2
• Movement into the T-system:
TP
the mousei T'
T VP
was V'
V' DP
chased ti
Passive morphology to interpreter:
- do not assign standard semantic role to the subject
- look for a gap
- assign to subject the role that otherwise would go to the gap
Reversible and non-reversible
passives
Non-reversible
• The apple was eaten by John
Reversible
• The dog was chased by the cat
The ability to process passive morphology is a
prerequisite for the correct interpretation of
reversible passives
Passives and the language system
• The prerequisite of being able to accurately
process morphology is naturally satisfied in the
mature and intact language system, but it need not
be met in an immature or impaired system.
• Both for young children and for patients with
certain types of language impairment this
condition may not be met, and hence such
speakers may have considerable problems with
reversible passives.
Passives and agrammatism
• A Restrictive Theory of Agrammatic
Comprehension, Yosef Grodzinsky, Tel Aviv
University and Aphasia Research Center, Boston
University School of Medicine, Brain and
Language 50, 27-51
Observations
(1) Above chance performance
The girl pushed the boy
(2) Chance performance
The boy was pushed by the girl
Hypothesis: syntactic movement yields problems
More precisely:
- traces are invisible to semantic role assignment
What do subjects do?
Strategy
The agent role is the most prominent role in a
hierarchy of semantic roles
Subjects use this for an auxiliary strategy:
Assign the agent role to the leftmost NP of the
clause as a default role
Result
TP
T'
NPi T VP
the boy was
V' PP
V
pushed ti by the girl
• The default strategy + correctly interpreting the by-
phrase ill-formed interpretation guessing
A below-chance performance
Agrammatic role assignment Normal assignm.
a. The mani is pushing the woman
| |
agent theme
b. The womani is pushed ti by the man
| |
*agent agent
c. The mani is hated ti by the woman
| |
*agent experiencer
c) yields below chance performance, since agent wins over
experiencer
Reversibility in wh-movement
• The ball that [the boy is kicking t] is red
• The cat that [the dog is chasing t] is black
The latter also presents problems for Broca‟s aphasics
Again the trace deletion hypothesis (Grodzinsky et
al.) can be adduced: Broca’s aphasics have problems
processing traces they use a default strategy to
interpret sentences with traces
Some Caveats
• Lesion in Broca‟s area neither sufficient nor
necessary to induce syntactic deficits:
– Broca‟s area is not always lesioned in a clinically
significant Broca‟s aphasia;
– Broca‟s area can be affected in patients who do not
display a Broca syndrome; most of these patients are
mildly anomic.
• Severity of morphosyntactic problems in aphasia
is correlated with the extent of damage in BA 22.
• Also semantic deficits in Broca‟s aphasia.
Disadvantages of patient studies
• damage to neural tissue may not be well
delineated (in functional terms)
– rather, depends on histological properties or on
structure of vascular system
• possibility of compensation/adaptation
• aphasic symptoms evolve over time (post onset)
unclear which symptoms (and hence: processes)
are linked to which neural networks
Healthy subjects: Imaging studies
• PET
• fMRI
Is Broca‟s area the „syntax
center‟?
• Kaan & Swaab 2002, review (Trends in
Cognitive Sciences)
• perception/comprehension studies; subtraction
method
• paradigms:
1. complex sentences vs. simple sentences
2. sentences vs. word lists
3. jabberwocky/syntactic prose vs. word lists/normal sts
4. syntactic violations vs. correct sentences
Where‟s syntax in the brain?
Overview: Kaan & Swaab 2002, review (TICS)
• Broca‟s area
– NOT necessarily involved in syntactic processing
– more activation with more working memory demands
• other area‟s associated with syntax
– anterior temporal lobe
– anterior parts of BA 21, 22
– superior & middle temporal gyri
– not only Left Hemisphere!
Broca‟s area …
• … is an excitable piece of tissue!
(David Poeppel, p.c.)
• activated by (i.a.)
– word/syllable lists (memory)
– semantic tasks
– phonological tasks
– music perception
Suggestions (K&S 2002)
• Middle/superior temporal lobe
– lexical processing (activating semantic/syntactic,
phonological features of words)
• Anterior temporal lobe
– combining activated information
• Broca’s area
– storing non-integrated materials
• Right hemisphere
– prosody
– ambiguity
– discourse
– error detection
dislocation/movement
• In structural terms:
– an element is doing „double duty‟ by having
two copies in the structure – only one of these
is spelled out phonetically (ie., has an audible
form)
• in processing terms:
– the processor has to recognize an „empty spot‟
at the location of the object NP and infer which
noun phrase can be connected to it.
Electro-magnetic signals
• EEG/ERP
• (MEG/ERF)
1. Different signatures for syntactic and
semantic processing?
2. Autonomy of syntactic processing?
EEG
ERP (Bressler 2002)
• The physiological basis of the cortical ERP: Fields of potential
generated by interacting neurons. Field potentials result from the
summed extracellular currents generated by electromotive forces
(EMFs) in the dendrites of synchronously active cortical neurons. The
EMFs, arising from synaptic activation of postsynaptic ion channels,
circulate current in closed loops across the cell membrane and through
the intracellular and extracellular spaces. Summed closed-loop currents
generated by an ensemble of neighboring neurons flow across the
external resistance to form the local ensemble mean field potential.
• The event-related potential (ERP): Neural signal that reflects
coordinated neural network activity. The cortical ERP provides a
window onto the dynamics of network activity in relation to a variety of
different cognitive processes at both mesoscopic and macroscopic levels
on a time scale comparable to that of single-neuron activity.
Good: Temporal resolution
Bad: Spatial resolution
Event-Related Potentials
ERP
ERP & language: N400
Kutas & Hillyard 1980
N400
• negative(-going) component
• peak latency around 400ms
• bi-lateral slightly posterior distribution
• N400 effect (amplitude modulation):
– (mis)match of word meaning with preceding
context
– semantic priming
P600/SPS
Osterhout & Holcomb 1993; Hagoort,
Brown & Groothusen 1993
P600
• positive(-going) deflection
• peak latency around 600ms
• bilateral, centro-parietal distribution
• grammatical anomalies
• ambiguities that are resolved in a dispreferred way
• long-distance dependencies
ERP: Early Left-Anterior Negativity
add picture
ELAN
• negative(-going) deflection
• peak latency around 200ms
• left anterior distribution
• grammatical violations, e.g.
– phrase structure
– inflection, function words
What do ERPs signify?
• physiologically: synchronous post-synaptic
activation of several hundreds of thousands
of radially oriented pyramidal cells
• functionally: No idea!
– the brain (groups of neurons) responds in a
particular, consistent way to particular stimuli
The big picture? (Friederici et al)
Linguistic Theory and Neural
Activity
What can we expect?
Very abstractly:
•Match between properties of derivations and processes
in production or comprehension?
•Match at the architectural level
- differences between modules involved in a mental
computation reflect differences in neural activity