Docstoc

Lecture 07

Document Sample
Lecture 07 Powered By Docstoc
					Biologically Inspired Intelligent
            Systems
             Lecture 07
        Dr. Roger S. Gaborski
               Feature Maps
– Intensity contrast (HW#3)
– Orientation (HW#4)
   • 0, 45, 90 and 135 degrees
– Color Information (HW#5)
   • Red-Green opponent color
   • Blue-Yellow opponent color




                                  2
       Processing of Color Data
• How is color coded in primates?
• Three classes of cone photoreceptors
• Arrangement of cone types seems to be
  random
• Color opponent cells (cone signals brought
  together in opposition)
  – Red-green
  – Blue-yellow

                                               3
     Red-Green RF




White overlap   Red in center   Green in surround

 |   || | |     ||||||||               ||||||||




                                                    4
     Red-Green RF




Green in center     Red in center    Green in center
                   Green in surround Red in surround
| | | | | |       |||||||||||||||||   NO RESPONSE




                                                       5
       Blue-Yellow RF




Same operation as Red- Green



                               6
   Color Channels (approximation)
• Create broadly tuned color channels:
   R = r-(g+b)/2
   G = g- (r+b)/2
   B = b- (r+g)/2
   Y = r+g – 2(|r-g| + b) (negative values set to zero)
• Maximum response to pure hue the channel is tuned to
  (if a pixel contains both r and g it will have a smaller R
  response than if it only had r)
• Zero response to white or black inputs


                                                               7
         Color Difference Map

• Create red-green, blue-yellow color channels
• For each color plane, apply receptive fields at
  different scales, 7x7, 15x15, 31x31, 63x63…
• Combine color receptive fields output to form
  color difference Map




                                                    8
              Potential Red-Green RF Model
         -3
     x 10
10



8
                                                             Create Dog64
                                                             Red_Kernel:
6
                                                             Green_Kernel

4                                                            R_ctr=filter red data with Red_Kernel

2                                                            G_sur=filter green data with Green_Kernel
                           RED
0                                                            Form difference: R_ctr - G_sur
                   GREEN              GREEN
-2
     0        10      20         30     40    50   60   70




                                                                                                     9
                          Note Colorbar
                                                                                           -4
                                        0.01                                        x 10
                                                                                    0
                                        0.009
10
                                                10
                                        0.008

20                                      0.007
                                                20
                                        0.006
30
                                        0.005   30
                                                                                        -6.9154

40                                      0.004
                                                40
                                        0.003
50
                                        0.002   50

                                        0.001
60
                                                60
                                        0
     10   20   30   40   50   60                                                        -13.8308
                                                     10   20   30   40   50   60


     Center                        Surround



                                                                                   10
Color Image
Gray Scale – “Lack of Interest”
                         Still Attention Processing
                                        Color opponent filters




                        Red center      Green center Blue center       Yellow center
                       Green surround   Red surround Yellow surround   Blue surround


                                                          Gabor orientation filters



 Difference of Gaussian filters
(Intensity contrast)
    On Center        Off Center             0 degree       45 degree       90 degree   135 degree




                                                              Image

                                                                                            15
                     Still Attention Processing




Color Salience map   Intensity Contrast Salience map   Orientation Salience map




                                                                                  16
         Still Attention Module
• Combine
  – Normalized Intensity Contrast Map
  – Normalized Orientation Maps
  – Normalized Color difference Map
  to form Still Attention Module




                                        17
Orange trail marker no longer
stands out as it does in color image



                                       18
19
20
21
22
23
24
                               Partial model

                                                                                WHERE n =
                                                                                0, 45, 90 and
                               n degrees     n degrees      n degrees           135 degrees
                               7x7 Gabor     15x15 Gabor    31x31 Gabor


n degrees   n degrees   n degrees                              n degrees     n degrees   n degrees
7x7 Gabor   15x15 Gabor 31x31 Gabor                            7x7 Gabor     15x15 Gabor 15x15 Gabor




                   Contrast Images imCon8   imCon16 imCon32                     R-G B-Y

                                           Retina Model
                                                                                  Color Opponent
                            8x8, 16x16 and 32x32 circular receptive fields

                                               Image
                                                                                                   25
                   Motion Detection
• Types of motion
  – Object
  – Optical flow




                                      26
               Motion Detection
• Types of motion
  – Object: Visual area V5 or MT (middle temporal),
  – Optical flow (MSTd –medial superior
    temporal/dorsal)




                                                      27
              A Sense of time
• We need temporal information to detect
  motion
• Computer vision techniques:
  – Frame differencing
  – Pixel modeling (Gaussian)
• ‘system’ level biological models



                                           28
    Motion- consider a bar moving to the right



y




                   x

At time t0 bar
is moving to the
right


                                                 29
        3D Representation of moving bar



y




            x


    t




                                          30
                 (x,t) Plot – Ignore y axis

                     x


                                          Slope of blue bar represents
                                          the velocity that the bar is moving

                                          Can you relate this plot to a stationary bar
 t                                        with the same slope??




Since object is moving vertical bar, nothing is lost with this representation

                                                                                     31
    Stationary bar detection (x,y)

         x

                     Plot of a stationary bar



y



      MOTION AS ORIENTATION



                                                32
      Stationary bar detection (x,y)

                x

                           Plot of a stationary bar
             - +
              - +
y               - +


    Simple cell response




                                                      33
             Moving bar detection (x,t)

                    x

                                          Plot of a moving bar
                - +
                 - +
t                  - +
TIME

       Spatial temporal receptive field




                                                                 34
           FASTER Moving bar detection (x,t)


                  x

                                      Faster plot of a moving bar (moves
                                      more x distance in same amount of
                                      time

t


     Spatial temporal (space and time) receptive field

We need a separate spatiotemporal for each velocity



                                                                           35
    “Gaussian Derivative Model for spatial-temporal
                 vision” (R.A. Young)
• GENERAL EQUATION
• Gn,o,p(x', y', t') = gn(x')go(y')gp(t') for n = 0,1 2,...
    – G - 3 D GD spatial-temporal filter
      x', y' and t' are the coordinate axes of G
      g are one dimensional GD functions
      n, o, and p are derivative numbers along x', y' and t'
• Additional parameters allow placement of model, spatial
  orientation, optimal speed (see paper for details)




                                                               36
M1n with no theta or phi component should detect
      a stationary edge image at 0 degrees




Red is negative, Blue is positive              37
(x,y) only




             38
M1ntheta should detect a stationary edge at
             theta degrees


                                 Theta = 45 degrees
                                 Red is negative,
                                 blue is positive




                                                  39
(x,y) only




             40
M1ntheta_phi should detect a moving edge at theta
         degrees moving at a rate phi



                                    Theta = 45 degrees
                                    Phi = 45 degrees
                                    Red is negative,
                                    Blue is positive




                                                         41
(x,y) only




             42
M2n with no theta or phi component should detect
       a stationary line image at 0 degrees




                                               43
(x,y) only




             44
M2ntheta should detect a stationary line at
             theta degrees




                                              45
(x,y) only




             46
M2ntheta_phi should detect a moving line at theta
         degrees moving at a rate phi




                                                47
(x,y) only




             48
         Experimental Results
Filter detects stationary edge at 0 degrees
                                    Note magnitude
                                    values for M1n
                                    For plane1,
                                    Max = .1
                                    Min = -.1

                                    For plane 21,
                                    Max = 1
                                    Min = -1

                                    Data block
                                    constant for all
                                    planes




                                                       49
Convolve each plane of data with
   corresponding filter plane

                         Result of filter would
                         be summation of all the
                         data plane responses




                                               50
Same filter, but edge now rotated – not
            optimal for filter




                                          51
Note maximum value less than for ideal edge




                                          52
Moving Edge



              Create a moving edge
              in a 41x41x41 data
              matrix that moves one
              pixel per time step




                                      53
M2n theta = 0 degrees and phi = 45 degrees




                                             54
Data shown at
midpoint of data
block




                   55
Filter Response

              Result of filter would
              be summation of all the
              data plane responses




                                        56
Using same motion filter, reverse the direction of the line
         motion from left to right to right to left




                                                         57
Filter Response

              Result of filter would
              be summation of all the
              data plane responses




                                        58
         Function components
• gn(x') is the nth derivative of the base Gaussian
  function g0 along the axis x'
  go(y') is the oth derivative of the base Gaussian
  function g0 along the axis y'
  gp(t') is the pth derivative of the base Gaussian
  function g0 along the axis t'




                                                  59
                       Monophasic
• When p=0 the function is monophasic
• Monophasic- Derivative along x' axis only, functions along other axes
  are simple Gaussian functions. Only shape number needed is
  derivative number n.
  n+1 gives number of lobes in space-time.
• G00 has one lobe
• G10 has two lobes and is created by taking the product of the first
  derivative function g1(x') and the two Gaussian functions, go(y') and
  go(t')
• G20 has three lobes and is created by taking the product of the second
  derivative function g2(x') and the two Gaussian functions, go(y') and
  go(t')



                                                                           60
           Biphasic fields, Gn1
• In this model differencing-like operations
  occur in both space and time in a single
  receptive field. This is accomplished by
  derivatives along the x' and t' axes.
  G01, G11 and G21 represent models with the
  first subscript denotes a derivative in the x
  direction and the second subscript '1' denotes
  a single derivative in the time direction.


                                               61
                                  Vision Model

                                                                                   WHERE n =
                                                                                   0, 45, 90 and
                                  n degrees     n degrees      n degrees           135 degrees
                                  7x7 Gabor     15x15 Gabor    31x31 Gabor


n degrees    n degrees   n degrees                                n degrees     n degrees   n degrees
7x7 Gabor    15x15 Gabor 31x31 Gabor                              7x7 Gabor     15x15 Gabor 15x15 Gabor


                                                                                        N   S E W
 R-G B-Y              Contrast Images imCon8   imCon16 imCon32


     Color opponent                           Retina Model
                                                                                        Motion Detection
                               8x8, 16x16 and 32x32 circular receptive fields

                                                  Image
                                                                                                     62
• Still Attention Module
  – Combine:
     • Normalized Intensity Contrast Map
     • Normalized Orientation Maps
     • Normalized Color difference Map
  to form Still Attention Module
• Motion Attention Module
  – Combine:
     • Motion Maps

                                           63
         Full Attention Model
• Combine:
  – Still attention map
  – Motion attention maps




                                64
• How can we use this model to explain how we
  interact in with the world?
  – Why do we ‘notice’ certain objects?
  – Why do we ignore others?
  – When viewing a scene, how long does an object
    keep your attention?




                                                    65
          Optical Flow - MSTd
• Navigation as opposed to object motion
• Runing, walking, etc.




                                           66
  16 Different Flow Patterns




Firing Rate:89   46




                               67
     16 Different Flow Patterns




34               33




                                  68
 16 Different Flow Patterns




32             17



                              69
16 Different Flow Patterns




6             6




                             70
16 Different Flow Patterns




6             4



                             71
16 Different Flow Patterns




2             2



                             72
16 Different Flow Patterns




2             2



                             73
16 Different Flow Patterns




1.5           0



                             74
    What if only one of the nine regions is
      excited? (spikes/sec and direction)

Region 6              Region 8
12               28


Region 2          Region 2
5.5               11

 Can we predict flow response from single data?


                                                  75
   What if 2 regions are excited?

Region 6 and Region 8: Firing rate: 28


Region 2 and Region 8: Firing rate: 23


Region 2 and Region 8: Firing rate: 22



                                         76
   What if 2 regions are excited?

Region 6 and Region 8: Firing rate: 28
     12        28

Region 2 and Region 8: Firing rate: 23
     5.5       28

Region 2 and Region 8: Firing rate: 22
      11      28

 Red Numbers are single region excitation only, black numbers 2 regions   77
                       MSTd Model
• Each region composed of 2 Gaussians
  – Parameters:
     •   Orientation
     •   Variance
     •   Gain
     •   Polarity
• 18 Gaussians
• Use Genetic algorithm to find parameter
  values
                                            78
   Computational model of MST neuron receptive field for the
                  perception of self-motion
              Chen-Ping Yu and Roger Gaborski

 Neuron 819R64’s trained receptive field dual-Gaussian model, arrow
  representation. Each arrow represents a Gaussian, the orientation
     corresponds to its tuned motion selectivity angle (μ) in polar
  coordinate; red signifies positive Gaussian which implies excitatory
  response while blue represents negative (inhibitory) Gaussian. The
length of the arrow is proportional to its magnitude (c) and the width
     of the arrowhead denotes its motion selectivity tolerance (σ).
Dual Gaussian Model
     Example 1
 Dual Gaussian Model
      Example 2

Excitatory
Gaussian




 Inhibitory
 Gaussian




                       81

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:5/20/2013
language:Unknown
pages:81
yaofenji yaofenji
About