Document Sample

Last lecture • Passive Stereo • Spacetime Stereo • Multiple View Stereo Today • Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry & fundamental matrix The epipolar geometry What if only C,C’,x are known? The epipolar geometry epipolar geometry demo C,C’,x,x’ and X are coplanar The epipolar geometry All points on project on l and l’ The epipolar geometry Family of planes and lines l and l’ intersect at e and e’ The epipolar geometry epipolar pole epipolar geometry demo = intersection of baseline with image plane = projection of projection center in other image epipolar plane = plane containing baseline epipolar line = intersection of epipolar plane with image The fundamental matrix F R p p’ C C’ T=C’-C x KX x' K' R(X - T) pK xX-1 p' K' -1 x' R (X - T) The equation of the epipolar plane through X is ( X T) (T p) 0 ( R p ' ) (T p ) 0 The fundamental matrix F ( R p ' ) (T p ) 0 T p Sp 0 Tz Ty S Tz 0 Tx Ty Tx 0 (R p' ) (Sp) 0 (p' R )(Sp) 0 p' Ep 0 essential matrix The fundamental matrix F R p p’ C C’ T=C’-C p' Ep 0 The fundamental matrix F p' Ep 0 Let M and M’ be the intrinsic matrices, then 1 pK x 1 p ' K ' x' (K' 1 x' ) E(K 1x) 0 1 x' K' EK x 0 x' Fx 0 fundamental matrix The fundamental matrix F R p p’ C C’ T=C’-C p' Ep 0 x' Fx 0 The fundamental matrix F • The fundamental matrix is the algebraic representation of epipolar geometry • The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images x' Fx 0 T x' T l' 0 The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies x’TFx=0 for all x↔x’ 1. Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental matrix for (P’,P) 2. Epipolar lines: l’=Fx & l=FTx’ 3. Epipoles: on all epipolar lines, thus e’TFx=0, x e’TF=0, similarly Fe=0 4. F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2) 5. F maps from a point x to a line l’=Fx (not invertible) The fundamental matrix F • It can be used for – Simplifies matching – Allows to detect wrong matches Estimation of F — 8-point algorithm • The fundamental matrix F is defined by x' Fx 0 for any pair of matches x and x’ in two images. f11 f12 f13 • Let x=(u,v,1)T and x’=(u’,v’,1)T, F f 21 f 22 f 23 f 31 f 32 f 33 each match gives a linear equation uu' f11 vu ' f12 u ' f13 uv' f 21 vv' f 22 v' f 23 uf31 vf 32 f 33 0 8-point algorithm f11 f 12 f13 u1u1´ v1u1´ u1´ u1v1´ v1v1´ v1´ u1 v1 1 u u ´ v u ´ u ´ u v ´ v v ´ v ´ u v 1 f 21 2 2 2 2 2 2 2 2 2 2 2 2 f 0 22 f 23 unun ´ vn un ´ un ´ un vn ´ vn vn ´ vn ´ un vn 1 f 31 f 32 f 33 • In reality, instead of solving Af 0, we seek f to minimize Af , least eigenvector of A A . 8-point algorithm • To enforce that F is of rank 2, F is replaced by F’ that minimizes F F' subject to det F' 0 . • It is achieved by SVD. Let F UΣV, where 1 0 0 1 0 0 Σ 0 2 0 , let Σ' 0 2 0 0 0 3 0 0 0 then F' UΣ' V is the solution. 8-point algorithm % Build the constraint matrix A = [x2(1,:)‘.*x1(1,:)' x2(1,:)'.*x1(2,:)' x2(1,:)' ... x2(2,:)'.*x1(1,:)' x2(2,:)'.*x1(2,:)' x2(2,:)' ... x1(1,:)' x1(2,:)' ones(npts,1) ]; [U,D,V] = svd(A); % Extract fundamental matrix from the column of V % corresponding to the smallest singular value. F = reshape(V(:,9),3,3)'; % Enforce rank2 constraint [U,D,V] = svd(F); F = U*diag([D(1,1) D(2,2) 0])*V'; 8-point algorithm • Pros: it is linear, easy to implement and fast • Cons: susceptible to noise Problem with 8-point algorithm f11 f 12 f13 u1u1´ v1u1´ u1´ u1v1´ v1v1´ v1´ u1 v1 1 u u ´ v u ´ u ´ u v ´ v v ´ v ´ u v2 f 21 1 2 2 2 2 2 2 2 2 2 2 2 f 22 0 f 23 unun ´ vn un ´ un ´ un vn ´ vn vn ´ vn ´ un vn 1 f 31 ~10000 ~10000 ~100 ~10000 ~10000 ~100 ~100 ~100 1 f 32 Orders of magnitude difference f between column of data matrix 33 ! least-squares yields poor results Normalized 8-point algorithm normalized least squares yields good results Transform image to ~[-1,1]x[-1,1] (0,500) (700,500) (-1,1) (1,1) 2 700 0 1 2 1 500 1 (0,0) (0,0) (700,0) (-1,-1) (1,-1) Normalized 8-point algorithm 1. Transform input by xi Tx i , x' Tx ' ˆ ˆi i ˆ ˆ ' ˆ 2. Call 8-point on x i , x i to obtain F 3. F T' Τ FTˆ x' Fx 0 1 x' T' FT x 0 ˆ ˆ ˆ F Normalized 8-point algorithm [x1, T1] = normalise2dpts(x1); [x2, T2] = normalise2dpts(x2); A = [x2(1,:)‘.*x1(1,:)' x2(1,:)'.*x1(2,:)' x2(1,:)' ... x2(2,:)'.*x1(1,:)' x2(2,:)'.*x1(2,:)' x2(2,:)' ... x1(1,:)' x1(2,:)' ones(npts,1) ]; [U,D,V] = svd(A); F = reshape(V(:,9),3,3)'; [U,D,V] = svd(F); F = U*diag([D(1,1) D(2,2) 0])*V'; % Denormalise F = T2'*F*T1; Normalization function [newpts, T] = normalise2dpts(pts) c = mean(pts(1:2,:)')'; % Centroid newp(1,:) = pts(1,:)-c(1); % Shift origin to centroid. newp(2,:) = pts(2,:)-c(2); meandist = mean(sqrt(newp(1,:).^2 + newp(2,:).^2)); scale = sqrt(2)/meandist; T = [scale 0 -scale*c(1) 0 scale -scale*c(2) 0 0 1 ]; newpts = T*pts; RANSAC repeat select minimal sample (8 matches) compute solution(s) for F determine inliers until (#inliers,#samples)>95% or too many times compute F based on all inliers Results (ground truth) Results (8-point algorithm) Results (normalized 8-point algorithm) From F to R, T x' Fx 0 1 x' M' EM x 0 E M' FM If we know camera parameters E R[T] Hartley and Zisserman, Multiple View Geometry, 2nd edition, pp 259 Application: View morphing Application: View morphing Main trick • Prewarp with a homography to rectify images • So that the two views are parallel • Because linear interpolation works when views are parallel Problem with morphing • Without rectification morph morph prewarp prewarp output input input homographies Video demo Triangulation • Problem: Given some points in correspondence across two or more images (taken from calibrated cameras), {(uj,vj)}, compute the 3D location X Richard Szeliski CSE 576 (Spring 2005): Computer 40 Vision Triangulation • Method I: intersect viewing rays in 3D, minimize: • X is the unknown 3D point • Cj is the optical center of camera j • Vj is the viewing ray for pixel (uj,vj) X • sj is unknown distance along Vj • Advantage: geometrically intuitive Vj Cj Richard Szeliski CSE 576 (Spring 2005): Computer 41 Vision Triangulation • Method II: solve linear equations in X • advantage: very simple • Method III: non-linear minimization • advantage: most accurate (image plane error) Richard Szeliski CSE 576 (Spring 2005): Computer 42 Vision

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 4 |

posted: | 8/11/2011 |

language: | English |

pages: | 42 |

OTHER DOCS BY hcj

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.