# Lecture 07

Document Sample

```					Biologically Inspired Intelligent
Systems
Lecture 07
Dr. Roger S. Gaborski
Feature Maps
– Intensity contrast (HW#3)
– Orientation (HW#4)
• 0, 45, 90 and 135 degrees
– Color Information (HW#5)
• Red-Green opponent color
• Blue-Yellow opponent color

2
Processing of Color Data
• How is color coded in primates?
• Three classes of cone photoreceptors
• Arrangement of cone types seems to be
random
• Color opponent cells (cone signals brought
together in opposition)
– Red-green
– Blue-yellow

3
Red-Green RF

White overlap   Red in center   Green in surround

|   || | |     ||||||||               ||||||||

4
Red-Green RF

Green in center     Red in center    Green in center
Green in surround Red in surround
| | | | | |       |||||||||||||||||   NO RESPONSE

5
Blue-Yellow RF

Same operation as Red- Green

6
Color Channels (approximation)
• Create broadly tuned color channels:
R = r-(g+b)/2
G = g- (r+b)/2
B = b- (r+g)/2
Y = r+g – 2(|r-g| + b) (negative values set to zero)
• Maximum response to pure hue the channel is tuned to
(if a pixel contains both r and g it will have a smaller R
response than if it only had r)
• Zero response to white or black inputs

7
Color Difference Map

• Create red-green, blue-yellow color channels
• For each color plane, apply receptive fields at
different scales, 7x7, 15x15, 31x31, 63x63…
• Combine color receptive fields output to form
color difference Map

8
Potential Red-Green RF Model
-3
x 10
10

8
Create Dog64
Red_Kernel:
6
Green_Kernel

4                                                            R_ctr=filter red data with Red_Kernel

2                                                            G_sur=filter green data with Green_Kernel
RED
0                                                            Form difference: R_ctr - G_sur
GREEN              GREEN
-2
0        10      20         30     40    50   60   70

9
Note Colorbar
-4
0.01                                        x 10
0
0.009
10
10
0.008

20                                      0.007
20
0.006
30
0.005   30
-6.9154

40                                      0.004
40
0.003
50
0.002   50

0.001
60
60
0
10   20   30   40   50   60                                                        -13.8308
10   20   30   40   50   60

Center                        Surround

10
Color Image
Gray Scale – “Lack of Interest”
Still Attention Processing
Color opponent filters

Red center      Green center Blue center       Yellow center
Green surround   Red surround Yellow surround   Blue surround

Gabor orientation filters

Difference of Gaussian filters
(Intensity contrast)
On Center        Off Center             0 degree       45 degree       90 degree   135 degree

Image

15
Still Attention Processing

Color Salience map   Intensity Contrast Salience map   Orientation Salience map

16
Still Attention Module
• Combine
– Normalized Intensity Contrast Map
– Normalized Orientation Maps
– Normalized Color difference Map
to form Still Attention Module

17
Orange trail marker no longer
stands out as it does in color image

18
19
20
21
22
23
24
Partial model

WHERE n =
0, 45, 90 and
n degrees     n degrees      n degrees           135 degrees
7x7 Gabor     15x15 Gabor    31x31 Gabor

n degrees   n degrees   n degrees                              n degrees     n degrees   n degrees
7x7 Gabor   15x15 Gabor 31x31 Gabor                            7x7 Gabor     15x15 Gabor 15x15 Gabor

Contrast Images imCon8   imCon16 imCon32                     R-G B-Y

Retina Model
Color Opponent
8x8, 16x16 and 32x32 circular receptive fields

Image
25
Motion Detection
• Types of motion
– Object
– Optical flow

26
Motion Detection
• Types of motion
– Object: Visual area V5 or MT (middle temporal),
– Optical flow (MSTd –medial superior
temporal/dorsal)

27
A Sense of time
• We need temporal information to detect
motion
• Computer vision techniques:
– Frame differencing
– Pixel modeling (Gaussian)
• ‘system’ level biological models

28
Motion- consider a bar moving to the right

y

x

At time t0 bar
is moving to the
right

29
3D Representation of moving bar

y

x

t

30
(x,t) Plot – Ignore y axis

x

Slope of blue bar represents
the velocity that the bar is moving

Can you relate this plot to a stationary bar
t                                        with the same slope??

Since object is moving vertical bar, nothing is lost with this representation

31
Stationary bar detection (x,y)

x

Plot of a stationary bar

y

MOTION AS ORIENTATION

32
Stationary bar detection (x,y)

x

Plot of a stationary bar
- +
- +
y               - +

Simple cell response

33
Moving bar detection (x,t)

x

Plot of a moving bar
- +
- +
t                  - +
TIME

Spatial temporal receptive field

34
FASTER Moving bar detection (x,t)

x

Faster plot of a moving bar (moves
more x distance in same amount of
time

t

Spatial temporal (space and time) receptive field

We need a separate spatiotemporal for each velocity

35
“Gaussian Derivative Model for spatial-temporal
vision” (R.A. Young)
• GENERAL EQUATION
• Gn,o,p(x', y', t') = gn(x')go(y')gp(t') for n = 0,1 2,...
– G - 3 D GD spatial-temporal filter
x', y' and t' are the coordinate axes of G
g are one dimensional GD functions
n, o, and p are derivative numbers along x', y' and t'
• Additional parameters allow placement of model, spatial
orientation, optimal speed (see paper for details)

36
M1n with no theta or phi component should detect
a stationary edge image at 0 degrees

Red is negative, Blue is positive              37
(x,y) only

38
M1ntheta should detect a stationary edge at
theta degrees

Theta = 45 degrees
Red is negative,
blue is positive

39
(x,y) only

40
M1ntheta_phi should detect a moving edge at theta
degrees moving at a rate phi

Theta = 45 degrees
Phi = 45 degrees
Red is negative,
Blue is positive

41
(x,y) only

42
M2n with no theta or phi component should detect
a stationary line image at 0 degrees

43
(x,y) only

44
M2ntheta should detect a stationary line at
theta degrees

45
(x,y) only

46
M2ntheta_phi should detect a moving line at theta
degrees moving at a rate phi

47
(x,y) only

48
Experimental Results
Filter detects stationary edge at 0 degrees
Note magnitude
values for M1n
For plane1,
Max = .1
Min = -.1

For plane 21,
Max = 1
Min = -1

Data block
constant for all
planes

49
Convolve each plane of data with
corresponding filter plane

Result of filter would
be summation of all the
data plane responses

50
Same filter, but edge now rotated – not
optimal for filter

51
Note maximum value less than for ideal edge

52
Moving Edge

Create a moving edge
in a 41x41x41 data
matrix that moves one
pixel per time step

53
M2n theta = 0 degrees and phi = 45 degrees

54
Data shown at
midpoint of data
block

55
Filter Response

Result of filter would
be summation of all the
data plane responses

56
Using same motion filter, reverse the direction of the line
motion from left to right to right to left

57
Filter Response

Result of filter would
be summation of all the
data plane responses

58
Function components
• gn(x') is the nth derivative of the base Gaussian
function g0 along the axis x'
go(y') is the oth derivative of the base Gaussian
function g0 along the axis y'
gp(t') is the pth derivative of the base Gaussian
function g0 along the axis t'

59
Monophasic
• When p=0 the function is monophasic
• Monophasic- Derivative along x' axis only, functions along other axes
are simple Gaussian functions. Only shape number needed is
derivative number n.
n+1 gives number of lobes in space-time.
• G00 has one lobe
• G10 has two lobes and is created by taking the product of the first
derivative function g1(x') and the two Gaussian functions, go(y') and
go(t')
• G20 has three lobes and is created by taking the product of the second
derivative function g2(x') and the two Gaussian functions, go(y') and
go(t')

60
Biphasic fields, Gn1
• In this model differencing-like operations
occur in both space and time in a single
receptive field. This is accomplished by
derivatives along the x' and t' axes.
G01, G11 and G21 represent models with the
first subscript denotes a derivative in the x
direction and the second subscript '1' denotes
a single derivative in the time direction.

61
Vision Model

WHERE n =
0, 45, 90 and
n degrees     n degrees      n degrees           135 degrees
7x7 Gabor     15x15 Gabor    31x31 Gabor

n degrees    n degrees   n degrees                                n degrees     n degrees   n degrees
7x7 Gabor    15x15 Gabor 31x31 Gabor                              7x7 Gabor     15x15 Gabor 15x15 Gabor

N   S E W
R-G B-Y              Contrast Images imCon8   imCon16 imCon32

Color opponent                           Retina Model
Motion Detection
8x8, 16x16 and 32x32 circular receptive fields

Image
62
• Still Attention Module
– Combine:
• Normalized Intensity Contrast Map
• Normalized Orientation Maps
• Normalized Color difference Map
to form Still Attention Module
• Motion Attention Module
– Combine:
• Motion Maps

63
Full Attention Model
• Combine:
– Still attention map
– Motion attention maps

64
• How can we use this model to explain how we
interact in with the world?
– Why do we ‘notice’ certain objects?
– Why do we ignore others?
– When viewing a scene, how long does an object

65
Optical Flow - MSTd
• Navigation as opposed to object motion
• Runing, walking, etc.

66
16 Different Flow Patterns

Firing Rate:89   46

67
16 Different Flow Patterns

34               33

68
16 Different Flow Patterns

32             17

69
16 Different Flow Patterns

6             6

70
16 Different Flow Patterns

6             4

71
16 Different Flow Patterns

2             2

72
16 Different Flow Patterns

2             2

73
16 Different Flow Patterns

1.5           0

74
What if only one of the nine regions is
excited? (spikes/sec and direction)

Region 6              Region 8
12               28

Region 2          Region 2
5.5               11

Can we predict flow response from single data?

75
What if 2 regions are excited?

Region 6 and Region 8: Firing rate: 28

Region 2 and Region 8: Firing rate: 23

Region 2 and Region 8: Firing rate: 22

76
What if 2 regions are excited?

Region 6 and Region 8: Firing rate: 28
12        28

Region 2 and Region 8: Firing rate: 23
5.5       28

Region 2 and Region 8: Firing rate: 22
11      28

Red Numbers are single region excitation only, black numbers 2 regions   77
MSTd Model
• Each region composed of 2 Gaussians
– Parameters:
•   Orientation
•   Variance
•   Gain
•   Polarity
• 18 Gaussians
• Use Genetic algorithm to find parameter
values
78
Computational model of MST neuron receptive field for the
perception of self-motion
Chen-Ping Yu and Roger Gaborski

Neuron 819R64’s trained receptive field dual-Gaussian model, arrow
representation. Each arrow represents a Gaussian, the orientation
corresponds to its tuned motion selectivity angle (μ) in polar
coordinate; red signifies positive Gaussian which implies excitatory
response while blue represents negative (inhibitory) Gaussian. The
length of the arrow is proportional to its magnitude (c) and the width
of the arrowhead denotes its motion selectivity tolerance (σ).
Dual Gaussian Model
Example 1
Dual Gaussian Model
Example 2

Excitatory
Gaussian

Inhibitory
Gaussian

81

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 0 posted: 5/20/2013 language: Unknown pages: 81
yaofenji