Semantics Without Categorization by MikeJenny

VIEWS: 3 PAGES: 44

									   Representation, Development and
Disintegration of Conceptual Knowledge:
    A Parallel-Distributed Processing
                Approach


             James L. McClelland
           Department of Psychology and
       Center for Mind, Brain, and Computation
                  Stanford University
Parallel Distributed Processing
Approach to Semantic Cognition
• Representation is a pattern
  of activation distributed
  over neurons within and
  across brain areas.             language



• Bidirectional propagation
  of activation underlies the
  ability to bring these
  representations to mind
  from given inputs.

• The knowledge underlying
  propagation of activation is
  in the connections.
      A Principle of Learning and
            Representation

• Learning and representation are sensitive to
  coherent covariation of properties across
  experiences.
    What is Coherent Covariation?

• The tendency of properties of objects to co-
  occur in clusters.
     • e.g.
        – Has wings
        – Can fly
        – Is light
     • Or
        – Has roots
        – Has rigid cell walls
        – Can grow tall
   Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development
  of conceptual knowledge.

• Gradual degradation of the representations
  constructed through this developmental
  process underlies the pattern of semantic
  disintegration seen in semantic dementia.
 Some Phenomena in Development

• Progressive differentiation of concepts
• Overgeneralization
• Illusory correlations
The Rumelhart Model
The Training Data:
All propositions true of
items at the bottom level
of the tree, e.g.:

Robin can {grow, move, fly}
Target output for ‘robin can’ input
Forward Propagation of Activation

aj
     wij

       neti=Sajwij   ai


                              wki
     Back Propagation of Error (d)


aj
        wij

                          ai
          di ~   Sdkwki

                                 wki

                                       dk ~ (tk-ak)
Error-correcting learning:

        At the output layer:    Dwki = edkai
        At the prior layer:     Dwij = edjaj
        …
Early
        E
        x
        p
        e
        r
Later
        i
        e
        n
        c
        e
Later
Still
       What Drives
        Progressive
      Differentiation?
•   Waves of differentiation reflect
    coherent covariation of properties
    across items.

•   Patterns of coherent covariation are
    reflected in the principal
    components of the property
    covariance matrix.

•   Figure shows attribute loadings on
    the first three principal components:

     – 1. Plants vs. animals
     – 2. Birds vs. fish
     – 3. Trees vs. flowers

•   Same color = features covary in
                           component
•   Diff color = anti-covarying
                 features
Coherence                                   Properties
Training                           Coherent          Incoherent
Patterns                is can has is can has …

                    1
                    2
                    3
                    4
                    5
  Items             6
                    7
                    8
                    9
                   10
                   11
                   12
                   13
                   14
                   15
                   16


No labels are provided
Each item and each property occurs with equal frequency
Effect of Coherence on Representation
Overgeneralization of Frequent
  Names to Similar Objects


    “tree”                    “goat”

                      “dog”
           Illusory Correlations

• Rochel Gelman found that children think that
  all animals have feet.
   – Even animals that look like small furry balls
     and don’t seem to have any feet at all.
• A tendency to over-generalize properties
  typical of a superordinate category at an
  intermediate point in development is
  characteristic of the PDP network.
A typical property that
a particular object lacks
e.g., pine has leaves




     An infrequent,
     atypical property
Sensitivity to Coherence
 Requires Convergence




                           A




            A
                           A
 Another key property of the model

• Sensitivity to coherent covariation can be
  domain- and property-type specific, and such
  sensitivity is acquired as differentiation occurs.
• Obviates the need for initial domain-specific
  biases to account for domain-specific patterns
  of generalization and inference.
              Differential Importance
                  (Marcario, 1991)
• 3-4 yr old children see a puppet
  and are told he likes to eat, or
  play with, a certain object (e.g.,
  top object at right)
   – Children then must choose
     another one that will “be the
     same kind of thing to eat” or
     that will be “the same kind of
     thing to play with”.
   – In the first case they tend to
     choose the object with the
     same color.
   – In the second case they will
     tend to choose the object
     with the same shape.
    Adjustments to
       Training
     Environment
•   Among the plants:
     – All trees are large
     – All flowers are small
     – Either can be bright or
        dull
•   Among the animals:
     – All birds are bright
     – All fish are dull
     – Either can be small or
        large
•   In other words:
     – Size covaries with
        properties that
        differentiate different
        types of plants
     – Brightness covaries
        with properties that
        differentiate different
        types of animals
       Testing Feature Importance

• After partial learning, model is shown eight test objects:
   – Four “Animals”:
       • All have skin
       • All combinations of bright/dull and large/small
   – Four “Plants”:
       • All have roots
       • All combinations of bright/dull and large/small
• Representations are generated by using
  back-propagation to representation.
• Representations are then compared to see which
  animals are treated as most similar, and which plants
  are treated as most similar.
    Similarities of Obtained
       Representations




Size is relevant   Brightness is relevant
for Plants         for Animals
   Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development
  of conceptual knowledge.

• Gradual degradation of the representations
  constructed through this developmental
  process underlies the pattern of semantic
  disintegration seen in semantic dementia.
    Disintegration of Conceptual
  Knowledge in Semantic Dementia

• Progressive loss of specific knowledge of
  concepts, including their names, with
  preservation of general information
• Overgeneralization of frequent names
• Illusory correlations
Picture naming
and drawing in
Sem. Demantia
  Grounding the Model in What we Know
   About The Organization of Semantic
         Knowledge in The Brain
• There is now evidence for
  specialized areas subserving
  many different kinds of
  semantic information.
• Semantic dementia results      language
  from progressive bilateral
  disintegration of the
  anterior temporal cortex.
• Rapid acquisition of new
  knowledge depends on
  medial temporal lobes,
  leaving long-term semantic
  knowledge intact.
  Proposed Architecture for the
Organization of Semantic Memory


  action                name

            Temporal               motion
              pole
                                            color
 valance                          form



           Medial Temporal Lobe
                                    • Gradually learns through
Rogers et al (2005)                   exposure to input patterns
model of semantic                     derived from norming
dementia                              studies.
                                    • Representations in the
                                      temporal pole are
                                      acquired through the
                                      course of learning.
                                    • After learning, the
             temporal
               pole
                                      network can activate each
                                      other type of information
                                      from name or visual input.
                                    • Representations undergo
                                      progressive differentiation
                                      as learning progresses.
 name   function   assoc   vision   • Damage to units within
                                      the temporal pole leads to
                                      the pattern of deficits seen
                                      in semantic dementia.
Errors in Naming for As a Function of Severity

Patient Data                   Simulation Results

                               omissions




                          within categ.

                             superord.




   Severity of Dementia      Fraction of Neurons Destroyed
       Simulation of Delayed Copying

                                   • Visual input is
                                     presented, then
                                     removed.
            temporal               • After several time
              pole
                                     steps, pattern is
                                     compared to the
                                     pattern that was
                                     presented initially.
name   function   assoc   vision
                                   • Omissions and
                                     intrusions are
                                     scored for typicality
 IF‟s „camel‟
                                       DC‟s „swan‟




                 Simulation results
Omissions by feature type   Intrusions by feature type
   Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development
  of conceptual knowledge.

• Gradual degradation of the representations
  constructed through this developmental
  process underlies the pattern of semantic
  disintegration seen in semantic dementia.
Sensitivity to Coherence
 Requires Convergence




                           A




            A
                           A

								
To top