VIEWS: 0 PAGES: 81 POSTED ON: 5/20/2013 Public Domain
Biologically Inspired Intelligent Systems Lecture 07 Dr. Roger S. Gaborski Feature Maps – Intensity contrast (HW#3) – Orientation (HW#4) • 0, 45, 90 and 135 degrees – Color Information (HW#5) • Red-Green opponent color • Blue-Yellow opponent color 2 Processing of Color Data • How is color coded in primates? • Three classes of cone photoreceptors • Arrangement of cone types seems to be random • Color opponent cells (cone signals brought together in opposition) – Red-green – Blue-yellow 3 Red-Green RF White overlap Red in center Green in surround | || | | |||||||| |||||||| 4 Red-Green RF Green in center Red in center Green in center Green in surround Red in surround | | | | | | ||||||||||||||||| NO RESPONSE 5 Blue-Yellow RF Same operation as Red- Green 6 Color Channels (approximation) • Create broadly tuned color channels: R = r-(g+b)/2 G = g- (r+b)/2 B = b- (r+g)/2 Y = r+g – 2(|r-g| + b) (negative values set to zero) • Maximum response to pure hue the channel is tuned to (if a pixel contains both r and g it will have a smaller R response than if it only had r) • Zero response to white or black inputs 7 Color Difference Map • Create red-green, blue-yellow color channels • For each color plane, apply receptive fields at different scales, 7x7, 15x15, 31x31, 63x63… • Combine color receptive fields output to form color difference Map 8 Potential Red-Green RF Model -3 x 10 10 8 Create Dog64 Red_Kernel: 6 Green_Kernel 4 R_ctr=filter red data with Red_Kernel 2 G_sur=filter green data with Green_Kernel RED 0 Form difference: R_ctr - G_sur GREEN GREEN -2 0 10 20 30 40 50 60 70 9 Note Colorbar -4 0.01 x 10 0 0.009 10 10 0.008 20 0.007 20 0.006 30 0.005 30 -6.9154 40 0.004 40 0.003 50 0.002 50 0.001 60 60 0 10 20 30 40 50 60 -13.8308 10 20 30 40 50 60 Center Surround 10 Color Image Gray Scale – “Lack of Interest” Still Attention Processing Color opponent filters Red center Green center Blue center Yellow center Green surround Red surround Yellow surround Blue surround Gabor orientation filters Difference of Gaussian filters (Intensity contrast) On Center Off Center 0 degree 45 degree 90 degree 135 degree Image 15 Still Attention Processing Color Salience map Intensity Contrast Salience map Orientation Salience map 16 Still Attention Module • Combine – Normalized Intensity Contrast Map – Normalized Orientation Maps – Normalized Color difference Map to form Still Attention Module 17 Orange trail marker no longer stands out as it does in color image 18 19 20 21 22 23 24 Partial model WHERE n = 0, 45, 90 and n degrees n degrees n degrees 135 degrees 7x7 Gabor 15x15 Gabor 31x31 Gabor n degrees n degrees n degrees n degrees n degrees n degrees 7x7 Gabor 15x15 Gabor 31x31 Gabor 7x7 Gabor 15x15 Gabor 15x15 Gabor Contrast Images imCon8 imCon16 imCon32 R-G B-Y Retina Model Color Opponent 8x8, 16x16 and 32x32 circular receptive fields Image 25 Motion Detection • Types of motion – Object – Optical flow 26 Motion Detection • Types of motion – Object: Visual area V5 or MT (middle temporal), – Optical flow (MSTd –medial superior temporal/dorsal) 27 A Sense of time • We need temporal information to detect motion • Computer vision techniques: – Frame differencing – Pixel modeling (Gaussian) • ‘system’ level biological models 28 Motion- consider a bar moving to the right y x At time t0 bar is moving to the right 29 3D Representation of moving bar y x t 30 (x,t) Plot – Ignore y axis x Slope of blue bar represents the velocity that the bar is moving Can you relate this plot to a stationary bar t with the same slope?? Since object is moving vertical bar, nothing is lost with this representation 31 Stationary bar detection (x,y) x Plot of a stationary bar y MOTION AS ORIENTATION 32 Stationary bar detection (x,y) x Plot of a stationary bar - + - + y - + Simple cell response 33 Moving bar detection (x,t) x Plot of a moving bar - + - + t - + TIME Spatial temporal receptive field 34 FASTER Moving bar detection (x,t) x Faster plot of a moving bar (moves more x distance in same amount of time t Spatial temporal (space and time) receptive field We need a separate spatiotemporal for each velocity 35 “Gaussian Derivative Model for spatial-temporal vision” (R.A. Young) • GENERAL EQUATION • Gn,o,p(x', y', t') = gn(x')go(y')gp(t') for n = 0,1 2,... – G - 3 D GD spatial-temporal filter x', y' and t' are the coordinate axes of G g are one dimensional GD functions n, o, and p are derivative numbers along x', y' and t' • Additional parameters allow placement of model, spatial orientation, optimal speed (see paper for details) 36 M1n with no theta or phi component should detect a stationary edge image at 0 degrees Red is negative, Blue is positive 37 (x,y) only 38 M1ntheta should detect a stationary edge at theta degrees Theta = 45 degrees Red is negative, blue is positive 39 (x,y) only 40 M1ntheta_phi should detect a moving edge at theta degrees moving at a rate phi Theta = 45 degrees Phi = 45 degrees Red is negative, Blue is positive 41 (x,y) only 42 M2n with no theta or phi component should detect a stationary line image at 0 degrees 43 (x,y) only 44 M2ntheta should detect a stationary line at theta degrees 45 (x,y) only 46 M2ntheta_phi should detect a moving line at theta degrees moving at a rate phi 47 (x,y) only 48 Experimental Results Filter detects stationary edge at 0 degrees Note magnitude values for M1n For plane1, Max = .1 Min = -.1 For plane 21, Max = 1 Min = -1 Data block constant for all planes 49 Convolve each plane of data with corresponding filter plane Result of filter would be summation of all the data plane responses 50 Same filter, but edge now rotated – not optimal for filter 51 Note maximum value less than for ideal edge 52 Moving Edge Create a moving edge in a 41x41x41 data matrix that moves one pixel per time step 53 M2n theta = 0 degrees and phi = 45 degrees 54 Data shown at midpoint of data block 55 Filter Response Result of filter would be summation of all the data plane responses 56 Using same motion filter, reverse the direction of the line motion from left to right to right to left 57 Filter Response Result of filter would be summation of all the data plane responses 58 Function components • gn(x') is the nth derivative of the base Gaussian function g0 along the axis x' go(y') is the oth derivative of the base Gaussian function g0 along the axis y' gp(t') is the pth derivative of the base Gaussian function g0 along the axis t' 59 Monophasic • When p=0 the function is monophasic • Monophasic- Derivative along x' axis only, functions along other axes are simple Gaussian functions. Only shape number needed is derivative number n. n+1 gives number of lobes in space-time. • G00 has one lobe • G10 has two lobes and is created by taking the product of the first derivative function g1(x') and the two Gaussian functions, go(y') and go(t') • G20 has three lobes and is created by taking the product of the second derivative function g2(x') and the two Gaussian functions, go(y') and go(t') 60 Biphasic fields, Gn1 • In this model differencing-like operations occur in both space and time in a single receptive field. This is accomplished by derivatives along the x' and t' axes. G01, G11 and G21 represent models with the first subscript denotes a derivative in the x direction and the second subscript '1' denotes a single derivative in the time direction. 61 Vision Model WHERE n = 0, 45, 90 and n degrees n degrees n degrees 135 degrees 7x7 Gabor 15x15 Gabor 31x31 Gabor n degrees n degrees n degrees n degrees n degrees n degrees 7x7 Gabor 15x15 Gabor 31x31 Gabor 7x7 Gabor 15x15 Gabor 15x15 Gabor N S E W R-G B-Y Contrast Images imCon8 imCon16 imCon32 Color opponent Retina Model Motion Detection 8x8, 16x16 and 32x32 circular receptive fields Image 62 • Still Attention Module – Combine: • Normalized Intensity Contrast Map • Normalized Orientation Maps • Normalized Color difference Map to form Still Attention Module • Motion Attention Module – Combine: • Motion Maps 63 Full Attention Model • Combine: – Still attention map – Motion attention maps 64 • How can we use this model to explain how we interact in with the world? – Why do we ‘notice’ certain objects? – Why do we ignore others? – When viewing a scene, how long does an object keep your attention? 65 Optical Flow - MSTd • Navigation as opposed to object motion • Runing, walking, etc. 66 16 Different Flow Patterns Firing Rate:89 46 67 16 Different Flow Patterns 34 33 68 16 Different Flow Patterns 32 17 69 16 Different Flow Patterns 6 6 70 16 Different Flow Patterns 6 4 71 16 Different Flow Patterns 2 2 72 16 Different Flow Patterns 2 2 73 16 Different Flow Patterns 1.5 0 74 What if only one of the nine regions is excited? (spikes/sec and direction) Region 6 Region 8 12 28 Region 2 Region 2 5.5 11 Can we predict flow response from single data? 75 What if 2 regions are excited? Region 6 and Region 8: Firing rate: 28 Region 2 and Region 8: Firing rate: 23 Region 2 and Region 8: Firing rate: 22 76 What if 2 regions are excited? Region 6 and Region 8: Firing rate: 28 12 28 Region 2 and Region 8: Firing rate: 23 5.5 28 Region 2 and Region 8: Firing rate: 22 11 28 Red Numbers are single region excitation only, black numbers 2 regions 77 MSTd Model • Each region composed of 2 Gaussians – Parameters: • Orientation • Variance • Gain • Polarity • 18 Gaussians • Use Genetic algorithm to find parameter values 78 Computational model of MST neuron receptive field for the perception of self-motion Chen-Ping Yu and Roger Gaborski Neuron 819R64’s trained receptive field dual-Gaussian model, arrow representation. Each arrow represents a Gaussian, the orientation corresponds to its tuned motion selectivity angle (μ) in polar coordinate; red signifies positive Gaussian which implies excitatory response while blue represents negative (inhibitory) Gaussian. The length of the arrow is proportional to its magnitude (c) and the width of the arrowhead denotes its motion selectivity tolerance (σ). Dual Gaussian Model Example 1 Dual Gaussian Model Example 2 Excitatory Gaussian Inhibitory Gaussian 81