Document Sample

From Pixels to Features: Review of Part 1 COMP 4900C Winter 2008 Topics in part 1 – from pixels to features • Introduction • what is computer vision? It’s applications. • Linear Algebra • vector, matrix, points, linear transformation, eigenvalue, eigenvector, least square methods, singular value decomposition. • Image Formation • camera lens, pinhole camera, perspective projection. • Camera Model • coordinate transformation, homogeneous coordinate, intrinsic and extrinsic parameters, projection matrix. • Image Processing • noise, convolution, filters (average, Gaussian, median). • Image Features • image derivatives, edge, corner, line (Hough transform), ellipse. General Methods • Mathematical formulation • Camera model, noise model • Treat images as functions I f ( x, y) • Model intensity changes as derivatives f [ I x , I y ]T • Approximate derivative with finite difference. • First-order approximation I (i u, j v) I (i, j ) I x u I y v I (i, j ) u v f • Parameter fitting – solving an optimization problem Vectors and Points We use vectors to represent points in 2 or 3 dimensions y v Q(x2,y2) x2 x1 P(x1,y1) v QP y2 y1 x The distance between the two points: D Q P ( x2 x1 ) ( y2 y1 ) 2 2 Homogeneous Coordinates Go one dimensional higher: wx wx x x y wy y wy wz w z w w is an arbitrary non-zero scalar, usually we choose 1. From homogeneous coordinates to Cartesian coordinates: x1 x1 x x1 / x4 x x1 / x3 2 x2 / x4 2 x2 / x3 x3 x3 x x3 / x4 4 2D Transformation with Homogeneous Coordinates 2D coordinate transformation: cos sin p x Tx p' ' p T sin cos y y 2D coordinate transformation using homogeneous coordinates: p x ' ' cos sin Tx p x p ' ' sin cos Ty p y y 1 0 0 1 1 Eigenvalue and Eigenvector We say that x is an eigenvector of a square matrix A if Ax x is called eigenvalue and x is called eigenvector. The transformation defined by A changes only the magnitude of the vector x Example: 3 2 1 5 1 3 2 2 4 2 1 4 1 5 51 and 1 4 1 2 2 1 1 2 5 and 2 are eigenvalues, and 1 and 1 are eigenvectors. Symmetric Matrix We say matrix A is symmetric if AT A Example: BT B is symmetric for any B, because ( B T B )T B T ( B T )T B T B A symmetric matrix has to be a square matrix Properties of symmetric matrix: •has real eignvalues; •eigenvectors can be chosen to be orthonormal. • BT B has positive eigenvalues. Orthogonal Matrix A matrix A is orthogonal if 1 A A I T or A AT The columns of A are orthogonal to each other. Example: cos sin 1cos sin A A sin cos sin cos Least Square When m>n for an m-by-n matrix A, Ax b has no solution. In this case, we look for an approximate solution. We look for vector x such that Ax b 2 is as small as possible. This is the least square solution. Least Square Least square solution of linear system of equations Ax b Normal equation: A Ax A b T T T A A is square and symmetric 1 The Least square solution x ( A A) A b T T Ax b 2 makes minimal. SVD: Singular Value Decomposition An mn matrix A can be decomposed into: A UDV T U is mm, V is nn, both of them have orthogonal columns: U U I T V V I T D is an mn diagonal matrix. Example: 2 0 1 0 0 2 0 0 3 0 1 0 0 3 1 0 0 1 0 0 0 0 1 0 0 Pinhole Camera Why Lenses? Gather more light from each scene point Four Coordinate Frames Pw pixel frame xim Yw Zw pim yim Yc y Xc world Xw camera frame frame image x Zc plane optical frame center principal point transformation Camera model: pim Pw matrix Perspective Projection P X Y p x f y f y Z Z x principal principal These are nonlinear. optical point axis center image plane Using homogenous coordinate, we have a linear relation: X u f 0 0 0 v 0 f Y 0 0 Z w 0 0 1 0 1 x u/w y v/w World to Camera Coordinate Transformation between the camera and world coordinates: X c RX w T Xc X w Y T Yw c R Z c 0 Z 1 w R,T 1 1 Image Coordinates to Pixel Coordinates x (ox xim ) s x y (o y yim ) s y s x , s y : pixel sizes xim yim y x (ox,oy) xim 1 / s x 0 ox x y 0 1/ s y oy y im 1 0 0 1 1 Put All Together – World to Pixel x1 1 / s x 0 ox u x 0 1/ sy oy v 2 x3 0 0 1 w X 1 / s x 0 ox f 0 0 0 c 0 oy 0 0 0 c Y 1/ sy f Z 0 0 1 0 0 1 0 c 1 X w 1 / s x 0 ox f 0 0 0 0 1/ sy oy 0 R T Yw f 0 0 0 1 0 0 1 0 0 1 Zw 0 1 Xw X w f / s x 0 ox 1 0 0 0 Y 0 f / sy 0 1 0 0 R T Yw K R T w oy 0 1 Z Zw 0 0 1 0 0 1 0 w 1 1 xim x1 / x3 yim x2 / x3 Camera Intrinsic Parameters f / s x 0 ox K 0 f / sy oy 0 0 1 K is a 3x3 upper triangular matrix, called the Camera Calibration Matrix. There are five intrinsic parameters: (a) The pixel sizes in x and y directions s x , s y (b) The focal length f (c) The principal point (ox,oy), which is the point where the optic axis intersects the image plane. Extrinsic Parameters X w X w x1 Y Y pim x2 K R T w M w Zw Zw x3 1 1 [R|T] defines the extrinsic parameters. The 3x4 matrix M = K[R|T] is called the projection matrix. Image Noise Additive and random noise: I x, y I x, y n x, y ˆ I(x,y) : the true pixel values n(x,y) : the (random) noise at pixel (x,y) Gaussian Distribution Single variable 1 ( x ) 2 / 2 2 p( x) e 2 Gaussian Distribution Bivariate with zero-means and variance 2 x 2 y 2 G x, y 1 exp 2 2 2 2 Gaussian Noise Is used to model additive random noise n2 2 2 •The probability of n(x,y) is e •Each has zero mean •The noise at each pixel is independent Impulsive Noise • Alters random pixels • Makes their values very different from the true ones Salt-and-Pepper Noise: • Is used to model impulsive noise I h, k xl I sp h, k imin yimax imin x l x, y are uniformly distributed random variables l , im in,im axare constants Image Filtering Modifying the pixels in an image based on some function of a local neighbourhood of the pixels N(p) p 10 30 10 f p 20 11 20 5.7 11 9 1 Linear Filtering – convolution The output is the linear combination of the neighbourhood pixels m/2 m/2 I A (i, j ) I * A A(h, k ) I (i h, j k ) h m / 2 k m / 2 The coefficients come from a constant matrix A, called kernel. This process, denoted by ‘*’, is called (discrete) convolution. 1 3 0 1 0 -1 2 10 2 1 0.1 -1 = 5 4 1 1 1 0 -1 Image Kernel Filter Output Smoothing by Averaging 1 1 1 1 * 1 1 1 9 1 1 1 Convolution can be understood as weighted averaging. Gaussian Filter x 2 y 2 G x, y 1 exp 2 2 2 2 Discrete Gaussian kernel: h2 k 2 1 G (h, k ) e 2 2 2 2 where Gh, k is an element of an m m array Gaussian Filter * 1 Gaussian Kernel is Separable IG I G m/2 m/2 G(h, k ) I (i h, j k ) h m / 2 k m / 2 m/2 m/2 h2 k 2 e h m / 2 k m / 2 2 2 I (i h, j k ) 2 m/2 h m/2 k2 e h m / 2 2 2 e k m / 2 2 2 I (i h, j k ) h2 k 2 h2 k2 since 2 e 2 2 2 2 2 e e Gaussian Kernel is Separable Convolving rows and then columns with a 1-D Gaussian kernel. 1 I 1 9 18 9 1 = Ir 38 1 9 1 Ir 18 = result 38 9 1 The complexity increases linearly with m instead of with m 2 . Gaussian vs. Average Gaussian Smoothing Smoothing by Averaging Nonlinear Filtering – median filter Replace each pixel value I(i, j) with the median of the values found in a local neighbourhood of (i, j). Median Filter Salt-and-pepper noise After median filtering Edges in Images Definition of edges • Edges are significant local changes of intensity in an image. • Edges typically occur on the boundary between two different regions in an image. Images as Functions 2-D Red channel intensity I f ( x, y) Finite Difference – 2D Continuous function: f x, y f x h, y f x, y lim x h 0 h f x, y f x, y h f x, y lim y h 0 h Discrete approximation: Convolution kernels: f x, y Ix f i 1, j f i , j 1 1 x f x, y 1 Iy f i , j 1 f i , j 1 y Image Derivatives I x I * 1 1 Image I 1 Iy I * 1 Edge Detection using Derivatives 1-D image f (x) 1st derivative f ' ( x) f ' ( x) threshold Pixels that pass the threshold are edge pixels Image Gradient gradient fx f f y magnitude f f 2 x f 2 y direction arctan(fy / fx ) Finite Difference for Gradient Discrete approximation: Convolution kernels: f I x (i, j ) f i 1, j f i , j 1 1 x f 1 I y (i, j ) f i , j 1 f i , j 1 y magnitude G(i, j ) I (i, j ) I (i, j ) 2 x 2 y aprox. magnitude G(i, j ) I x I y direction arctan( I y / I x ) Edge Detection Using the Gradient Properties of the gradient: • The magnitude of gradient provides information about the strength of the edge • The direction of gradient is always perpendicular to the direction of the edge Main idea: • Compute derivatives in x and y directions • Find gradient magnitude • Threshold gradient magnitude Edge Detection Algorithm 1 1 Ix * edges Ix Iy 2 2 Threshold Image I * 1 1 Iy Edge Detection Example Ix I Iy Edge Detection Example G(i, j ) I x (i, j ) I y (i, j ) 2 2 I G(i, j ) Threshold Finite differences responding to noise Increasing noise -> (this is zero mean additive gaussian noise) Solution: smooth first Where is the edge? Look for peaks in Sobel Edge Detector Approximate derivatives with central difference Convolution kernel I x (i, j ) f f i 1, j f i 1, j 1 0 1 x 1 0 1 Smoothing by adding 3 column 2 0 2 neighbouring differences and give more weight to the middle one 1 0 1 1 2 1 Convolution kernel for I y 0 0 0 1 2 1 Sobel Operator Example a1 a2 a3 1 0 1 a4 a5 a6 2 0 2 * a7 a8 a9 1 0 1 a1 a2 a3 1 2 1 a4 a5 a6 0 0 * 0 a7 a8 a9 1 2 1 The approximate gradient at a 5 I x (a1 a3 ) 2(a4 a6 ) (a7 a9 ) I y (a1 a7 ) 2(a2 a8 ) (a3 a9 ) Sobel Edge Detector 1 0 1 2 0 2 Ix * 1 0 1 Ix Iy 2 2 edges Threshold Image I * 1 2 1 0 Iy 0 0 1 2 1 Edge Detection Summary Input: an image I and a threshold . 1. Noise smoothing: I s I h (e.g. h is a Gaussian kernel) 2. Compute two gradient images I x and I y by convolving I s with gradient kernels (e.g. Sobel operator). 3. Estimate the gradient magnitude at each pixel G(i, j ) I x (i, j ) I y (i, j ) 2 2 4. Mark as edges all pixels (i, j ) such that G(i, j ) Corner Feature Corners are image locations that have large intensity changes in more than one directions. Shifting a window in any direction should give a large change in intensity Harris Detector: Basic Idea “flat” region: “edge”: “corner”: no change in no change along significant change all directions the edge direction in all directions C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988 Change of Intensity The intensity change along some direction can be quantified by sum-of-squared-difference (SSD). D(u, v) I (i u, j v) I (i, j ) 2 i, j u v I (i u, j v) I (i, j ) Change Approximation If u and v are small, by Taylor theorem: I (i u, j v) I (i, j ) I x u I y v I I where Ix and Iy x y therefore I (i u, j v) I (i, j ) I (i, j ) I xu I y v I (i, j ) 2 2 I xu I y v 2 I x u 2 2 I x I y uv I y v 2 2 2 Ix 2 I x I y u u v 2 I x I y I y v Gradient Variation Matrix Ix 2 I I u D(u, v) u v x y I x I y I 2 y v This is a function of ellipse. Ix 2 I I C x y I x I y I 2 y Matrix C characterizes how intensity changes in a certain direction. Eigenvalue Analysis Ix 2 I I 0 T 1 C x y Q Q I x I y I 0 2 2 y If either λ is close to 0, then this is not a corner, so look for locations where both are large. I x C Ix Iy I y I x C I x I y AT A I y (max)-1/2 •C is symmetric •C has two positive eigenvalues (min)-1/2 Corner Detection Algorithm Line Detection The problem: •How many lines? •Find the lines. Equations for Lines The slope-intercept equation of line y ax b What happens when the line is vertical? The slope a goes to infinity. A better representation – the polar representation x cos y sin Hough Transform: line-parameter mapping A line in the plane maps to a point in the - space. (,) All lines passing through a point map to a sinusoidal curve in the - (parameter) space. x cos y sin Mapping of points on a line Points on the same line define curves in the parameter space that pass through a single point. Main idea: transform edge points in x-y plane to curves in the parameter space. Then find the points in the parameter space that has many curves passing through. Quantize Parameter Space m m Detecting Lines by finding maxima / clustering in parameter space. Examples input image Hough space lines detected Image credit: NASA Dryden Research Aircraft Photo Archive Algorithm 1. Quantize the parameter space int P[0, max][0, max]; // accumulators 2. For each edge point (x, y) { For ( = 0; <= max; = +) { x cos y sin // round off to integer (P[][])++; } } 3. Find the peaks in P[][]. Equations of Ellipse x2 y 2 2 2 1 r1 r2 ax 2 bxy cy 2 dx ey f 0 Let x [ x 2 , xy, y 2 , x, y,1]T a [a, b, c, d , e, f ]T Then xT a 0 Ellipse Fitting: Problem Statement Given a set of N image points p i [ xi , yi ]T find the parameter vector a 0such that the ellipse f (p, a) xT a 0 fits p i best in the least square sense: N min a D(p i , a)2 i 1 Where D(p i , a) is the distance from p i to the ellipse. Euclidean Distance Fit pi D(pi , a) pi pi ˆ ˆ pi ˆ p i is the point on the ellipse that is nearest to p i f (p i , a) 0 ˆ p i p i is normal to the ellipse at p i ˆ ˆ Compute Distance Function Computing the distance function is a constrained optimization problem: min p i p i 2 ˆ pi ˆ subject to f (p i , a) 0 ˆ Using Lagrange multiplier, define: L( x, y, ) pi pi 2f (pi , a) 2 ˆ ˆ where p i [ x, y ]T ˆ Then the problem becomes: min L( x, y, ) ˆ pi L L p i p i f (p i , a) ˆ ˆ Set 0 we have x y Ellipse Fitting with Euclidean Distance Given a set of N image points p i [ xi , yi ] T find the parameter vector a 0such that N 2 f (p i , a) min f (p i , a) a 2 i 1 This problem can be solved by using a numerical nonlinear optimization system.

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 0 |

posted: | 4/13/2013 |

language: | English |

pages: | 72 |

OTHER DOCS BY dffhrtcv3

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.