Get documents about "Diapositive
Document Sample
GEPPETO1:
A modeling approach to study the
production of speech gestures
Pascal Perrier (ICP – Grenoble)
with
Stéphanie Buchaillard (PhD)
Matthieu Chabanas (ICP)
Ma Liang (PhD),
Yohan Payan (TIMC – Grenoble)
1 GEstures shaped by the Physics and by a
PErceptually oriented Targets Optimization
Outline
• Introduction
• Current hypotheses implemented in
GEPPETO
• Some results obtained with a 2D
biomechanical tongue model
• New issues raised by the use of 3D
biomechanical tongue model
Basic issues
in Speech Production Research
• Phonology/Phonetics Interface
– Link between discrete representations and
continuous physical signals
– Nature of physical correlates of speech
units
Basic issues
in Speech Production Research
• Control and Production of Speech
Gestures
– Control variables
– Central representations of physical
characteristics of the speech production
apparatus
– Interaction Perception-Action
Basic issues
in Speech Production Research
• From Gestures to Speech Sounds
– Nature of acoustic sources
– Relations between motor commands and
acoustics
– Interaction between airflow and
articulatory gestures.
What is GEPPETO?
• An evolutive modeling framework to
quantitatively test hypotheses about the
control and the production of speech
gestures.
• It includes
– Hypotheses about the physical correlates of
phonological units.
– Models of motor control
– Physical models of the speech production
apparatus
Current Hypotheses
• Phonology/Phonetic Interface
– The smallest phonological unit is the
phoneme
– Phonemes are associated with target
regions in the auditory domain
– Larger phonological units are associated
with speech sequences for which specific
constraints exist for target optimization
or for motor commands sequencing
Current Hypotheses
• Control of speech gestures
– Control variables: l commands (EP Hypothesis,
Feldman, 1966)
– No on line use of feedback going through the
cortex.
– Short-delay orosensory and proprioceptive
feedbacks are taken into account.
– Existence in the brain of internal
representations of the speech apparatus
(internal models).
Current Hypotheses
• Control of speech gestures
– Internal representations do not account for the
whole physical complexity of the speech
production apparatus
– Kinematic characteristics are not directly
controlled. They are the results of the
interaction between motor control setups and
physical phenomena of speech production
• Which characteristics of speech signals are
specifically controlled?
Application to the generation
of speech gestures with a 2 D
biomechanical tongue model
• Implementation of the model of
control
• Inversion from desired perceptual
objectives to motor commands
• Generation of gestures
2D Biomechanical Model
• Finite element structure
• Linear elasticity (small deformations)
• No account of the gravity
2D Biomechanical Model
Posterior genioglossus Anterior Genioglossus Hyoglossus
2D Biomechanical Model
Styloglossus Verticalis Inferior Longitudinalis
Learning a static internal model
From l commands to formants
Step 1:
- Uniform sampling of
the l commands space
-Generation of the
corresponding tongue
shapes. 9000 simulations
Learning a static internal model
From l commands to formants
Step 2: Computation of the area function.
Learning a static internal model
From l commands to formants
Step 3: Formants computation for 2 lip apertures
(red dots: spread lips; blue dots: rounded lips)
Learning a static internal model
From l commands to formants
Step 4: Learning and generalizing
with radial basis functions
1ere couche 2nd couche
1st layer nd
2 layer
1
W 11 2
X1 + F W 11
... + Y1
...
+ F ... ...
Xn 1W ... + Yp
1m
1
W nm
+ F 2
W mp
m neurones p neurones
Inversion
From target regions to l commands
Target regions
•Dispersion ellipses
in the (F1, F2, F3)
space
• Currently defined
by Fc1, Fc2, Fc3 and
sF1, sF2, sF3
Target regions for some non rounded
French phonemes
Inversion
From target regions to l commands
Target regions
•Dispersion ellipses
in the (F1, F2, F3)
space
• Currently defined
by Fc1, Fc2, Fc3 and
sF1, sF2, sF3
Target regions for some non rounded
French phonemes
Inversion
From target regions to l commands
Optimization
Cost minimization (Gradient descent technique)
Cost for a sequence made of N phonemes
+
Speaker oriented Listener oriented
with
Inversion
From target regions to l commands
Example 1
Sequence [œ-e-k-i]
Inversion
From target regions to l commands
Example 2
Sequence [œ-e-k-a]
Production of tongue movements
from inferred l commands
Serial command patterns
No difference between vowels and consonants
[oe] [e] [k] [a]
Execution of tongue movements
from inferred l commands
Öhman’s model: Vowel-to-Vowel basis
Consonants are seen as perturbation of V-V
[oe] [e] [k] [a]
Execution of tongue movements
from inferred l commands
Observed flesh point
Production of tongue movements
from inferred l commands
Serial command patterns
[a]
[i]
Production of tongue movements
from inferred l commands
Öhman’s command patterns
[a]
[i]
Interaction control / physics.
Influence on the shapes
of the articulatory paths
Example: the Articulatory loops
[aka] [ika]
R. Houde (1969)
Fluid-Wall Interaction
Imposed
pressure
difference Forces
Flow model Mechanics of the tissues.
Finite element model)
Deformation
Interaction control / physics.
Influence on the shapes
of the articulatory paths
Example: the Articulatory loops
Deplacement X - Y
120
115
110
Y - mm
105
100
+++ PS = 3000 Pa
...... PS = 800 Pa
95
-------No aerodynamics
90
40 50 60 70 80 90 100 110 120
X - mm
[aka]
Interaction control / physics.
Influence on the shapes
of the articulatory paths
Example: the Articulatory loops
No aerodynamics With aerodynamics
[aka]
Interaction control / physics.
Influence on the shapes
of the articulatory paths
Example: the Articulatory loops
Deplacement X - Y
113
... PS = 1600 Pa 112
---- No aerodynamics
111
Y - mm
110
109
108
107
61 62 63 64 65 66 67
X - mm
[ika]
Interaction control / physics.
Influence on the shapes
of the articulatory paths
Example: the Articulatory loops
No aerodynamics With aerodynamics
[ika]
A 3D biomechanical tongue model:
For a better account of physics
• Visible Human Project ® data
(Wilhelms-Tricarico, 2003)
• Finite Element Mesh made of Hexahedres
• Adaptation of the mesh to a specific speaker (PB)
Wilhelms-Tricarico R.,1995 Gerard et al., ICP Grenoble
Inner muscle structure
of the tongue
Genioglossus (posterior)
Genioglossus (medium)
Superiorlongitudinalis
Inferior longitudinalis
Styloglossus
Transversus
Hyoglossus
Geniohyoid
Mylohyoid
Verticalis
(anterior)
Vocal tract structure
TONGUE’S BODY
HYOID BONE
MANDIBLE
PALATE
OTHER MUSCLES
Elastical properties of tongue muscles
Displacement
0 Force
Linear Non Linear
Tongue Indentator
• Hyperelastic material (2nd order Yeoh
model) with large deformation
hypothesis
Effect of gravity
[1s]
Dealing with gravity with the
EP hypothesis
[300ms]
Dealing with gravity with the
EP hypothesis
• Activation of GGp
and MH
Increase of reflex
activity
[300ms]
Dealing with gravity with the
EP hypothesis
GGP activation
Dealing with gravity with the
EP hypothesis
Example of a good choice of control parameters
[300ms]
Conclusions
• A model of control based on perceptual
objectives specified in terms of formants
target regions associated with l motor
commands and on an optimization process
using a static model of the motor-
perception relations can generate realistic
speech movements if it is applying to a
realistic physical model of speech
production.
Conclusions
• It supports our hypothesis that there
is not need to assume the existence
of a central optimization process that
would apply to the articulatory
trajectories in their whole (i.e.
minimum of jerk, minimum of torque…)
Conclusions
• It gives an interesting account of
coarticulation phenomena by
separating the effects of planning
and those of physics.
• It permits to test hypotheses about
the phonological units (see serial
model versus Öhman’s model).
Conclusions
However
• a systematic comparison with data is
required (currently in progress for
French, German, Chinese, Japanese)
• No account for time control, or for
hypo/hyperspeech
• No account for gravity
Conclusions
• Necessity to work on a more complex
internal representations that would
integrate some aspects of
articulatory dynamics.
Influence of elasticity modeling
Hyperelastic
Large defo.
Linear
Small defo.
Linear
Activation of the Hyoglossus (2N)
EP Hypothesis
(Feldman, 1966)
Perrier, Ostry, Laboissière, 1996
EP Hypothesis
(Feldman, 1966)
Perrier, Ostry, Laboissière, 1996
Static Internal Models
Central Nervous System 1ere couche 2nd couche
Desired
1
W 11 2
X1 + F W 11
... + Y1
ld Peripheral motor
...
+
formants
F ... ...
Xn 1W ... + Yp
system
1m
1
W nm
+ F 2
W mp
m neurones p neurones
Inverse Model
l
yi(t)
1ere couche 2nd couche
1
W 11 2
X1 + F W 11
... + Y1
...
+ F ... ...
Formants
Xn 1W ... + Yp
1m
1
W nm
+ F 2
W mp
m neurones p neurones
Direct Model
follow-
up
Maltreatment groups did not
differ on MAOA activity
MAOA activity moderates the Maltreatment experienced < 11 years
impact of early
maltreatment on the severe maltreatment
developement of violence probable maltreatment
no maltreatment
Arguments to support this model
Plan of the talk
• Theoritical arguments related to neural network
modelisation
• The lack of direct genetic determinism contributing to
externalizing disorders
• The contribution of environmental variables to
externalizing disorders
• The contribution of animal models to understand
consequences of stress on development
• Towards a probabilistic epigenetic model
Cohen, Neurosciences BioBehavioral Review, 2010
TOXIC AND PER NATAL FACTORS that impact brain
development either during pregnancy and/or infancy)
MICRO-ENVIRONMENTAL VARIABLES that impact the child
and his/her family in a proximal way
Low socioeconomic status, Early separation, Single/disrupted parent (father
absence), Large number of siblings, Individual handicap or poor social skills,
Sexual and/or physical abuse, Family violence and/or alcoholism, Mentally ill
parent, Parental use of punishment as opposed to reward
MACRO-ENVIRONMENTAL VARIABLES that impact at a
more general societal level
Urban residency, TV exposure, Minority, Rejection from school, Inclusion in
at-risk pro-social alternatives (peer grouping: e.g., ganging; drug abuse),
Competitive and violent culture
environt
epigenesis Stress & non genomic
Disorder
transmission of behaviors
Cohen, Neuroscience BioBehav Rev 2010 Mealey L, BBS. 1995
Maltreatment and antisocial behaviors: e.g. MICRO
5 years 7 years
Jaffee et al, J Abnorm Psychol. 2004
Twin study – E-risk study - N=1116 same sex twin pairs
Physical maltreatment plays a causal role in the
development of children’s antisocial behavior
Maternal depression and antisocial behaviors: e.g.
MICRO
Kim-Cohen et al.
Arch Gen Psy 2005
Twin study – E-risk
study
N=1116 same sex twin
pairs
Post natal maternal
depression predicted
antisocial behaviors at
age 7:
-with a ”dose effect”
-after controlling for
mother’s and father’s
ASPD
-with additive effects
-with high heritability
Intergenerational transmission of childhood
conduct problems: a twin study
D’Onofrio et al. 2007 Arch Gen Psy
High-risk sample – 889 twin families – N=2554
Australia
Using the number of conduct disorder symtoms, there was
a significant intergenerational transmission for all
offsprings (male>female)
In male, largely mediated by environmental
variables
In female, not the same. A common genetic liability
accounted for the intergenerational relations
Affiliation to an antisocial group during
adolescence: e.g. MACRO
1 Getting inside the
group of ASB
Leaving the group
of ASB
Two trajectories regarding admission to a group:
Early-adolescence then mid adolescence Lacourse et al.
Dev Psych 2006
Arguments to support this model
Plan of the talk
• Theoritical arguments related to neural network
modelisation
• The lack of direct genetic determinism contributing to
externalizing disorders
• The contribution of environmental variables to
externalizing disorders
• The contribution of animal models to understand
consequences of stress on development
• Towards a probabilistic epigenetic model
Cohen, Neurosciences BioBehavioral Review, 2010
Ealy stress, inproper maternal care, stress during gestation
influence development through non genomic transmissions
of behaviors in animal models
Victor Denenberg (1970) – Michael Meaney (2000)
environt
epigenesis Stress & non genomic
Disorder
transmission of behaviors
Early experiences and epigenetic programming
Early experiences have long-term effects on behavioral and
biological systems: e.g. handling (pup/mother separation)
Denenberg et al, Science, 1967; Liu et al., Science, 1997
Early experiences affect future generation providing a non-
genomic mechanism for the transmission of behavioral traits
Denenberg & Whimbey, Science, 1963; Francis et al., Science, 1999
The uterine environment affects development through
environmental factors rather than genetic ones
Denenberg et al, Neuroreport, 1998; Francis et al., Nat Neurosci, 2003
Maternal care impacts development through behavioral
programming and responses to stress in adulthood…
Maternal care ( pup licking/grooming and arched back nursing)
influences:
- Hypothalamic-Pituitary-Adrenal responses to stress in offsprings
Liu et al. Science, 1997
- Hippocampal synaptogenesis, plasticity and spatial learning and
memory
Liu et al. Nat Neurosci, 2000; Mirescu et al. Nat Neurosci, 2004
- The offspring epigenome in the hippocampus; and reversal
occurs when (1) cross-fostering is proposed; (2) histone
deacetylase is infused in early post natal
Weaver et al. Nat Neurosci, 2004
Rats models
implicate
epigenetic
regulation of
hippocampal
glucocorticoid
R expression
in mediationg
the effects of
early life
experience on
adult behavior
How adversity
gets under the
skin?
Hyman et al. Nat Neurosc 2009
Family context, early interactions and antisocial
behaviors: e.g. MICRO
Caspi et al. Dev Psychopathol,
2004
Twin study – E-risk study
N=565 MZ twin pairs
The twin receiving more
maternal negative emotional
expression and less warmth at
age 5 had more antisocial
behaviors at age 7
Jaffee et al. Child Dev, 2003
Twin study – E-risk study – N=1116 twin pairs
The less time father with low antisocial behaviors lived with their
children, the more conduct problems their children had receiving
When father engaged in high levels of antisocial behavior, the more
time they lived with their children, the more conduct problems
their children had
Reversibility of adverse rearing condition
consequences by restoring normal family rearing
Adoption study
N=144 Romanian (≤42 months) adoptees
reared in very depriving institutions
vs. 52 UK born (≤6 months) adoptees
At entry, cognitive impairment was
associated with institutional deprivation
and with its duration
At follow-up (6 years of age):
- There was a remarkable degree of
recovery after restoration of normal
family rearing
-Major deficits persisted in a
substantial minority: General Cognitive
Index was 25 points lower in those
who entered the UK after 2 years vs.
those who entered before 6 months
whatever the degree of malnutrition
Rutter et al, Dev Psychol. 2004
Institutional Rearing and Psychiatric Disorders
in Romanian Preschool Children
Zeanah et al, Am J Psy. 2009
Institutional Rearing and Psychiatric Disorders
in Romanian Preschool Children
Children with any history of institutional rearing had
more psychiatric disorders than children without
such a history (53.2% vs. 22.0%)
Children removed from institutions and placed in foster
families were less likely to have internalizing
disorders than children who continued with care as
usual (22.0% vs. 44.2%)
Boys were more symptomatic than girls regardless of
their caregiving environment
Unlike girls, boys had no reduction in total psychiatric
symptoms following foster placement
Zeanah et al, Am J Psy. 2009
Timing of intervention affects brain electrical activity in
children exposed to severe psychosocial neglect
Scalp topography of alpha power Mean alpha power across sites
Children exposed to PS neglect (institution) and cared as usual, N=48
Children exposed to PS neglect (institution) and placed in foster care
after 24 months, N=28
Children exposed to PS neglect (institution) and placed in foster care
before 24 months, N=25
Children never institutionalized, N=42
Vanderwert et al, PlosOne. 2010
Association of exposure to peer verbal abuse with
elevated psychiatric symptom scores
Teicher et al, Am J Psy. 2010
Association of exposure to peer verbal abuse with
corpus callosum abnormalities
N=63
No history of sexual abuse
No history of physical abuse
Peer verbal abuse score > 30
%
Teicher et al, Am J Psy. 2010
Arguments to support this model
Plan of the talk
• Theoritical arguments related to neural network
modelisation
• The lack of direct genetic determinism contributing to
externalizing disorders
• The contribution of environmental variables to
externalizing disorders
• The contribution of animal models to understand
consequences of stress on development
• Towards a probabilistic epigenetic model
Cohen, Neurosciences BioBehavioral Review, 2010
Developmental view of externalizing disorders
Family factors
Protective factors Psychosocial factors
Peer grouping
Drug abuse
School exclusion
ODD CD ASP
Cultural factors
Psychosocial factors
Hostile parenting Impulsivity
Early life adversities Aggressiveness
Toxic factors Hyperactivity
Genetic factors Empathy/Callous-unemotional traits
Biological factors Narcissism/Self-esteem
ADHD Insecure attachment
Manic defense
Child Adolescent Young Adult
Modified from Loeber et al, JAACAP. 2000
Metatheoritical model of probabilistic epigenesis
[≠ predetermined epigenesis]
ENVIRONMENT Individual development
Physical – Social – Cultural
BI-DIRECTIONAL
BEHAVIOR and CONDUCT
NEURAL LEVEL
GENETIC LEVEL
Early Adolescence
interaction
In this view, neural (and other) structures begin to function before they are
fully mature, and this activity, whether intrinsically derived or extrinsically
stimulated plays a significant role in the developmental process.
Since the coordination of formative functional and structural influences within
and between all levels of analysis is not perfect, a probabilistic element is
introduced in all developing systems and their outcome.
Some tension may occur (e.g. early life, culture, CU traits)
Modified from Goetlieb, Dev Science, 2007
Rearing condition, 5HTT polymorphism and (i)
response to stress or (ii) alcohol preference
from Caspi and Moffitt, Nat Rev Neurosci. 2006
Mother- Peer-
reared reared
Carence Modulation Axe Consommation
précoce génétique corticotrope d’alcool
Barr et al, Biol Psy. 2004 Barr et al, Arch Gen Psy. 2004
Epigenetic regulation of the glucocorticoid R
in human brain associates
with childhood abuse and suicide
McGowan et al. Nat Neurosci 2009
Hippocampal glucocorticoid R expression
(a) total GR; (b) GR 1F
Increased cytosine
methylation of the NR3C1
promoter in the
hippocampus of abused
individuals who suicide
McGowan et al. Nat Neurosci 2009
As clinician, let’s work within a developmental
framework, a dimensional assessment and an
integrative approach
Psycho
Organic Social
Familial
Psychodynamic
SUBJET Cognitive
Clinical
