The development of perceptual integration by yurtgc548


									The development of
perceptual integration

Topics in Cognitive Development, V.
3 December 2008
Zoltán Jakab
Department of Cognitive Science, Budapest University of
Technology and Economics
  Remember receptive fields
  How were they discovered?
Stimulating the retina with a small light spot
Recording from the optic nerve
Stephen W. Kuffler (1913-1980)
  Receptive fields in the retina
         Stimulation        Response
         of the             of the
         retinal field      ganglion cell

         All illuminated
                                                                ganglion cell

         Center only                                                  excitatory
         illuminated                                                  connection from
         Periphery only                                               ganglion cell
                                                                      connection from
         Periphery                                                    photoreceptor
                                            Ganglion cell
         partially illum.                                             ganglion cell
                                            spikes along
                                            the time axis
         All dark

So in total darkness the response is stronger than when periphery only is illuminated!
The other way around…
  Stimulation        Response
  of the             of the
  retinal field      ganglion cell

  All illuminated
                                     ganglion cell

  Center only                        excitatory
  Illuminated                        connection from
  Periphery only                     ganglion cell
                                     connection from
  Periphery                          photoreceptor
  partially illum.                   ganglion cell

  All dark
 Receptove fields and the retina

Photoreceptors in the retina connect to ganglion cells in
  ”bundles” with a special structure.
cones             rods

                         bipolar cells (B)
                         horizontal cells (H)
                         amacrine cells (A)

                         ganglion cells
                         fibers of the optic

  A beeső fény iránya
 An application of the concept: the
 Hermann grid

We see the gray blobs at
 every junction except
 the one we fixate on
  An explanation
Note: this story presupposes that RFs come in different sizes

Bigger ones are found on the periphery; smaller ones at the fovea
(BTW, that explains what?)
  It this the whole story?
Do we see the blobs in the sinusoud version of the Hermann grid?

What is the problem?
The scintillating grid
  Receptive fields at higher levels
Direction-selective cells in the primary visual cortex

David Hubel & Torsten Wiesel (1960s & 70s)

                                          Stimulus OFF ON OFF

                             Electric signal
            Microelectrode                                      Tuning curve

                                                                Direction of stimulus
 Orientation and direction
 selectivity in the primary visual
 Hubel and Wiesel (1968):
  presented drifting oriented
  luminance bars and drifting
  luminance spots to
  monkeys while recording
  form their V1.

 The orientation selectivities
  of neighboring cells are
  similar – there is a
  systematic variation.
Retinal/thalamic receptive fields
are organized in more complex
structures to give rise to a direction-
selective V1 cell
  Receptive fields of different sizes
Campbell and Robson (1968)

Tested by sinusoidal gratings

Four parameters:
1. frequency
2. amplitude
3. phase
4. orientation
  Images and sinusoidal gratings
                                     Add two low-freq
Any 2D image can be synthetized      sinusoid gratings
                                     with orientations
  out of a properly chosen set of    45 and 315
  sinusoidal gratings
 We need to add up the

                                    Add this one
Put another way:
Any 2D image can be
  decomposed into sinusoidal
  gratings – the list of such
  grating is the image’s
 Fourier analysis
Making a square-wave
out of sine-waves
  Modulation transfer functions
-- Lenses, mirrors, and other optical devices
-- they change image spectra
-- attenuating especially the higher frequencies.
-- think of a clean vs. a greasy lens
The MTF in
                                 original intensity

                                        anal.) of

                                         lens transfer

                                          modified by
 Intensity distribution of the            the lens
 Image formed by the lens
How can we measure perceptual transfer
-- Can’t see the image directly
-- Discriminative responses can help
Contrast sensitivity functions
  Lightness, color, and CSFs
The ability to resolve high-
  contrast patterns depends
  on illumination

Window of visibility
  Chromatic CSFs
They develop considerably in infancy, as color vision improves
  (significant improvement between 1 to 4 months)
  Lenses versus nervous systems
Humans and other animals: CSFs are often bandpass filters
Optical devices: MTFs are low-pass filters.

Lenses cannot help but transmit low frequencies.
Not so for nervous systems: low frequencies, like high ones,
  are transmitted only if some neurons selectively respond to
  Another evidence for neuronal
  action: increasing visual acuity
Improvement is due to development of the primary visual cortex
Amblyopia: loss of visual acuity due to lack of proper
  stimulation – the eye may be optically perfect – the visual
  system is affected (incurable and irreversible)
Measuring CSFs: animals and
  The stuctural basis of CFSs
The overall curve arises
… selective responses of
   individual neurons to
   different spatial
-- Receptive fields of different
   sizes are likely part of the
Selective adaptation to
   different frequencies
 direction-selectivity
(likely a product of direction-selective
 Division of labor in the visual system
Two visual (sub)systems
- Primate physiology and anatomy
- Human psychophysics and lesion studies

 The two subsystems process the same input information in
  different ways (David Milner and Melvyn Goodale, 1995)
 Separation after the primary visual cortex
Earlier: ”what” and ”where” systems
More recently: perception vs. action

Double dissociation between the subsystems
 System-specific tasks

 Ventral system lesion:
- shape perception is disrupted – shape and size-based
   discrimination is not possible (color perception, and color-
   based object discrimination may stay intact!)
- verbal report on viewed shapes is severely affected
- the mailing task; grasping an object
- walking along a rocky path
 Dorsal system lesion:
The opposite pattern – object representation remains
Source of the pictures: Milner-Goodale: Sight Unseen)
 Optic ataxia (Bálint syndrome)

 Dorsal system lesion
– conscious vision, object representation and report in
   language are all intact
– visuomotor coordination is impaired
The two pathways
An example of dissociation: The
Ebbinghaus illusion

  The central disks look different in size, although they are the same size
  The illusion also obtains for disk configurations of this kind.
  Measuring grip size

The distance between the index finger and thumb
Infrared markers attached to the ends of the fingers
 The grasping hand is NOT fooled by the illusion
More illusions that behave the same way
  A contrasting case:

Rod and frame illusion (top): no visuomotor effect (posting task)
Simultaneous tilt illusion (bottom): visuomotor effect is just as strong as the
   perceptual one.
How do we explain this? The latter illusion is likely the result of local effects in the
   primary visual cortex, and is passed on to both streams.
  Further functional differentiation within the
  ventral stream

FFA: fusiform face area
PPA parahippocampal
place area
LO: lateral occipital area

 Visual agnosia has different forms.
(1) Object agnosia (2) Prosopagnosia (3) Topographic agnosia (activated by
    pictures of buildings and scenes; memory-based spatial orientation is one of its
    functions – e.g., remembering a route and prominent landmarks)
 The action system: encodes object properties that change
  each moment wrt the observer.
 Online processing;
 Egocentric coordinates (the origo is the observer);
 Very short memory span (the past is irrelevant for current
 The perception system: encodes object identity and object
  properties for future cognition
 Object-centered descriptions
 Long-term memories

Only the representatons in the perception system are
The structure of the
primary visual cortex
 The structure of the primary visual
 Organized in layers and
 Several types of cells

 Six Brodmann layers, and
  their later subdivisions

 Layers receiving innervation
  from LGN: 2/3 (supragra-       Source:
  nular) and 4C
 Cortical columns
 Columns are the vertical
  arrangement of cells from the
  surface to the white matter.
 Columns are both anatomical and
  functional units.

 Anatomical: e.g., the apical
  dentrites of pyramidal cells are
  bundled together in a small region

 Functional: columns of cortical cells
  all respond to the same stimulus
  feature (e.g., orientation, direction
  or color).
 Ocular dominance columns

 Visual signals from the
  ganglion cells remain
  segregated in the LGN,
  and up to V1.

 Within layer 4C there are
  abrupt shifts as to which
  eye drives the excitatory
  cells in a small region
  (such shifts occur within
  50 microns).

                              Ocular dominance columns in V1.
 Receptive field properties

 Each cell in the visual
  cortex has a receptive
  field – a retinal region
  and the corresponding
  area in space – where
  the presentation of a
  stimulus causes
  response in the cortical
  cell in question.
                             Mapping of the retina on the unfolded V1.
 Orientation and direction selectivity

 Hubel and Wiesel (1968):
  presented drifting oriented
  luminance bars and drifting
  luminance spots to
  monkeys while recording
  form their V1.

 The orientation selectivities
  of neighboring cells are
  similar – there is a
  systematic variation.
 Orientation-selectivity maps in the
 monkey brain
 A map of V1 orientation-selective cells coded in different
  colors. Smaller sortical columns are location and
 Out of these, larger columnar structures arise which
  contain cells selective to different orientations at the same
  location (i.e., completely overlapping receptive fields with
  different orientation selectivities).

  Source: Csépe-Győri-Ragó: Általános pszichológia, 3.7., p113.
 Hubel and Wiesel also
  found binocular cells in V1 –
  cells that responded
  preferentially to input
  coming simultaneously from
  both eyes.

 Plus, they also found
  neurons: some neurons
  respond proportionally to
  the level of disparity
V. Gabor patches
Gabor patches are tiny pieces
  of sinusoidal gratings (one
  cycle across)
They fit very well to the
  receptive fields of V1
  orientation-selective cells –
  a good way to selectively
  stimulate these cells
VI. Tests of low-level visual integration

The role of integration:

- Representing global patterns within local elements detected
   by the receptive fields
- A useful side-effect: completing fragmentary stimuli
 A classical example by D. Marr
 We see circles and
  other geometrical
  figures overlapping
  and criss-crossing one
 This figure is ambi-
  guous: it permits
  different groupings of
  its primitives
 The ambiguity creates
  the imp-ression of
  continu-ous change,
  or “swarming”, thereby
  demonstrating the
  active structure-
  seeking effort of early
Glass patterns

The local elements are dots – they do not
  have orientation. Pattern is coded in the
  alignment of local elements.

Psychological difference: it is argued that
  extrastriate areas are involved in
  recognizing Glass patterns.
(3) Illusory contours
   (Kanizsa figures).

V1, V2, and other
  extrastriate areas are
  involved in the perception
  of illusory contours
  (Kovács, 2000)
The development of visual integration
 Newborn and infant visual performance:
- Very early preference for moving stimuli
- Good flicker sensitivity from 2 months on
- Chromatic discrimination at 2-3 months
- The onset of stereopsis around 4 months
- Increasing visual acuity
Local vs. integrative?
 Local attributes: contrast; color; motion
 Local processing: does not take between-receptive-fields
 The processing of local features takes place in V1.
                Motion integration: the aperture problem
Retinal cells

 V1 cell
Contour-integration cards: examining a
relatively simple form of visual integration
                       Developed by I. Kovács et
                       Consists of Gabor signals
                       Contains a contour (in this
                        case, a closed one), arising
                        from the collinear alignment
                        of part of the signals
                       The contour is embedded in

                       A key parameter for
                        variation: avg distance of
                        contour elements per avg
                        distance of noise elements
 The lower the delta, the more difficult it is to detect the
 If delta is greater than 1, that is a distinct cue for contour
 Collinear alignment is another critical feature for detection
Orientation is indeed critical...
                                            It is argued that V1 is sufficient for
                                            accomplishing this sort of integra-
                                            A severely agnosic patient with
                                            extensive bilateral damage to a
                                            number of extrastriate areas, but
                                            with spared V1 showed normal
                                            performance on the contour
                                            integration test.
                                            (Riddoch et al, Brain, Vol. 122,
                                            No. 3, 1999)

I. Kovács, Vis. Res. 40 (2000), 1301-1310
 What is the anatomical substrate of
 contour integration?
 Long-range, horizontal connections within the primary
  visual cortex.
 Top-down influences from extrastriate areas may also play
  a role
 Horizontal connections arise from axons of pyramidal cells
  found in the supragranular layers (Br layers 2-3) of V1*
 Conjecture: the network of horizontal connections that is
  thought to underlie contour integration …
…includes connections of cells with similar orientation tuning
  that code for neighboring locations
…the fine-tuning of such connections is due to visual input
  containig long contours
Maturation in humans and animals
 Horizontal connections of a layer 3 pyramidal cell. Actual
  length of such axons is up to 1-2 mm.
 Such connections connect V1columns with similar

  Source: Csépe-Győri-Ragó: Általános pszichológia, 3.7., p113.
 Contour integration in infants:
 continuity and closure
 Two principles of visual organization
– good continuation
– closure
both were pointed out originally by the Gestalt psychologists

Contours on the cards are continuous; they consist of
    collinear elements (Gabor signals) -
- in addition, they may be open or closed.

The perceptual significance of closure: a closed contour likely
  indicates a shape or a surface area.

In experiments with adults, closed contours did have an
   advantage over open ones (Kovács&Julesz, 1993).
 An experiment with infants (3-4 m)
                              Top: a contour that is easy to detect
 Operant conditioning        for a 3-month-old; the bottom one is

 Baseline level of kicking
  assessed; then came
  the training with

 After the training, when
  the infant detected a
  contour on the cards,
  kicking slowed down for
  a while
 Infants this old did detect contours based on orientation
  information alone (delta less than 1)…
 …but their threshold was at or above Δ=0.8 (with a noise
  ratio greater than that, they went on kicking fast).

 In contrast with adults, closure made no difference

 Hatched bars: baseline

 White bars: right after training with

 Dark bars: new stimuli containing

 It looks like the mechanism for detecting continuity is
   present at this age, however, that for detecting closure is
 Increased noise sensitivity might indicate shorter spatial
   range of the horizontal connections that are thought to
   underlie contour integration
 Related findings at this age:
- 4-month-olds lack of interest in static occlusion displays
   (Kellman and Spelke)

           A simple occlusion
           display: an open and
           a closed contour
 A computer model of filtering and
 Visual processing begins with filtering the retinal image.
Filtering is carried out by receptive fields of small sizes –
- filters are orientation-selective (and also spatial frequency
    selective, but their spatial-freq-selectivity is determined by
    their size)
The next step is interactions between the filters

Infants are less sensitive to both color and luminance
   contrast, still, some signs of perceptual integration (filter
   interactions) are detectable in their case
 Output of a computer
   simulation of local filtering
   and filter interactions
 Two differences between
   infants and adults (i.e., at
   this early level of
 more coarse-grained
   resolution in infants due
   to the predominance of
   lower spatial frequencies
   - cf. fig. c) and d)
 More efficient combina-
   tion of local filter outputs
(edge definition and
   ownership, figure-ground
   segmentation, occlusion)
 What is the rate of maturation of
 early integration like?
 Maturation extends into
  childhood (5-14 yrs)

 Children’s lower performance
  is not due to their lower
  contrast sensitivity
 How do we know?

 Adults perform equally well
  on levels of stimulus contrast
  much lower than what
  children experience with the
  original cards
 Spatial range of interaction and the
 horizontal connections of V1
 In addition to Δ, there
  is another parameter
  that may be varied on
  the cards
 avg distance of
  contour elements per
  avg distance of noise
  elements may stay
  constant while the
  absolute values of the
  two distances vary
 Thus we may have
  “sparse” cards and
  “dense” cards with the
  same deltas
 Any psychological significance?
 Yes indeed. A study with three sets of cards (sparse,
  intermediate, dense) – the members in the three series had
  matching delta values
 Adult performance was unaffected by the density
  parameter, whereas 5-6-yr-old children did worse on the
  sparse cards than on the other two sets.
 Why so?

 The spatial range of horizontal connections, therefore long-
  range interactions, may be limited in children.

 In fact, it is one property of neuronal connectivity how far
  the axons reach; another is, how finely their connections
  with other orientation-selective cells are tuned.

 Children’s visual cortex may be less developed on both
 The cue-specificity of integration

 High degree of stimulus-specificity indicates early levels of
  visual cortical processing.
 At these levels stimulus dimensions are still separable.

 Experiment to address this:
- standard (orientation-based) contour-integration cards, and
- color-based ones with identical arrangements of primitives

Is there a transfer of learning from orientation to color and
   vice versa?
Contour integration based on color
vs. orientation
The color-based cards
  had the same
  configuration of
  primitives as the

                 Kovács et al., Proc. Natl. Acad Sci USA, 96(21) 12204-9
 Tests on three consecu-
  tive days
 4 X 2 groups: (orientation;
  color; orientation-to-color;
  color-to-orientation); (5-6
  yrs; 19-35-yrs)

 There was improvement
  over three days in the O
  and C groups (more
  pronounced among
  children than adults);

 No transfer among adults
  (OtoC and CtoO grps)

 A tendency toward transfer
  in children

 Both age groups improve with practice

 Transfer is absent in both groups

 What can we infer from this?
 The same mechanism might be responsible for orientation
  vs. color-based integration, but it operates on different early
  representations in the two cases.
 The processes are separate, but they are of the same kind:
  e.g., long-range interactions between orientation-selective
  cells vs. color-selective ones.
 This explains both lack of transfer and similar performance
  in the two tasks.
 Integration beyond contours – and
 beyond V1
 Pictorial illusions (M-L, Ponzo, Ebbinghaus, perspective
  and relative size, etc.)

 According to preliminary data, they are weaker in children
  than in aduls
 They are absent in the newly sighted
 They arise in the ventral system (Milner and Goodale)
 An experiment             (Kovács et al.)

 How can we measure
  the magnitude of a
  pictorial illusion?

 Take the Ebbinghaus
 2AFC task
 Solid curve: probability of
  ”bigger” and ”smaller”
  responses depending on
  the ratio of sizes of the
  two central disks, when
  the illusion is absent.
 Dashed curve: illusion

 Subjects: 4-yr-old children and adults

 Four conditions; size of one of the central disks is varied

 8 comparison sizes for each condition, 1.33 mm steps

 Each size was presented five times

 Solid: adults,
 Dashed: children

 There was a diffe-
  rence between
  children and adults in
  the upper two
 There was a significant age-related difference in the first
  two conditions
 None in the fourth (i.e., children can reliably estimate the
  size ratios of the disks)
 Contrary to expectations, the third condition did not
  produce a significant difference.
 Children perceive more
  veridically in some settings

 Size constancy: related to
   contextual integration
Adults are more susceptible to size-
   constancy-related illusions than
Information about distance is
   important in size-constant
Which system is involved in size-
   constant perception – ventral or
Dorsal system lesions in monkeys
   do no affect size-constant
Distance-dependent changes in
   neural responses in the ventral
 Finally: the rate of maturation of the
 ventral vs. dorsal system
 New methods are needed to assess dorsal system
 What we know:
The ventral system appears to have an extended course of
Ontogenetically, there is a greater need for early availability of
  spatial orientation and visuomotor cordination (i.e., dorsal
  function) than conscious representation and long-term
  memories of visual impressions
Anatomical and phylogenetic evidence also supports slower
  maturation of the ventral system.

To top