Document Sample

Approximate Nearest Subspace Search with Applications to Pattern Recognition Ronen Basri, Tal Hassner, Lihi Zelnik-Manor presented by Andrew Guillory and Ian Simon The Problem • Given n linear subspaces Si: Z x0 i T The Problem • Given n linear subspaces Si: Z x0 i T • And a query point q: The Problem • Given n linear subspaces Si: Z x0 i T • And a query point q: • Find the subspace Si that minimizes dist(Si,q). Why? • object appearance variation = subspace – fast queries on object database Why? • object appearance variation = subspace – fast queries on object database • Other reasons? Approach • Solve by reduction to nearest neighbor. – point-to-point distances Approach • Solve by reduction to nearest neighbor. – point-to-point distances not actual reduction Approach • Solve by reduction to nearest neighbor. – point-to-point distances not actual reduction • In higher-dimensional space. Point-Subspace Distance • Use squared distance. dist x, S 2 T 2 Z x Z x Z x T T x T ZZ T x xx ZZ T T 2hxx hZZ T T Point-Subspace Distance a11 • Use squared distance. 2 a 12 dist x, S 2 a1d 2 T Z x a11 a12 a1d Z x Z x T T a12 a22 a2 d a22 x T ZZ T x h 2 xx ZZ T T a 1d a2 d add 2hxx hZZ T T a2 d add 2 Point-Subspace Distance a11 • Use squared distance. 2 a 12 dist x, S 2 a1d 2 T Z x a11 a12 a1d Z x Z x T T a12 a22 a2 d a22 x T ZZ T x h 2 xx ZZ T T a 1d a2 d add 2hxx hZZ T T a2 d add • Squared point-subspace distance 2 can be represented as a dot product. The Reduction • Let: Remember: u h ZZ T v h xx T dist 2 x, S 2h xx T h ZZ T The Reduction • Let: Remember: u h ZZ T v h xx T dist 2 x, S 2h xx T h ZZ T • Then: dist 2 u, v u 2u v v 2 2 dist x, S u v 2 2 2 The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T v h xx T The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T constant over query v h xx T The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T ? constant over query v h xx T The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T ? constant over query v h xx T u 2 h ZZ T 2 hZZ hZZ T T TrZZ ZZ 1 T T ZTZ = I 2 TrZZ 1 T 2 TrZ Z 1 T 2 1 d k 2 The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T ? constant over query v h xx T u 2 h ZZ T 2 hZZ hZZ T T TrZZ ZZ 1 T T ZTZ = I 2 TrZZ 1 T 2 TrZ Z 1 T Z is d-by-(d-k), columns orthonormal. 2 1 d k 2 The Reduction dist u, v dist x, S u v 2 2 2 2 u h ZZ T ? constant over query v h xx T u 2 h ZZ T 2 hZZ hZZ T T TrZZ ZZ 1 T T ZTZ = I 2 TrZZ 1 T 2 TrZ Z 1 T Z is d-by-(d-k), columns orthonormal. 2 1 d k 2 The Reduction • For query point q: dist 2 u, v dist 2 q, S 1 2 d k q 4 The Reduction • For query point q: dist 2 u, v dist 2 q, S 1 2 d k q 4 • Can we decrease the additive constant? Observation 1 Tr ZZ T d k • All data points lie on a hyperplane. Observation 1 Tr ZZ T d k • All data points lie on a hyperplane. • Let: d k T q2 u h ZZ T I v h qq I d d • Now the hyperplane contains the origin. Observation 2 • After hyperplane projection: 2 T d k h ZZ 2 u I d k d k d 1 • All data points lie on a hypersphere. Observation 2 • After hyperplane • Let: projection: 1 k d k T q2 I 2 T d k v h qq h ZZ 2 u I q 2 d 1 d d k d k d 1 • Now the query point lies on the hypersphere. • All data points lie on a hypersphere. Observation 2 • After hyperplane • Let: projection: 1 k d k T q2 I 2 T d k v h qq h ZZ 2 u I q 2 d 1 d d k d k d 1 • Now the query point lies on the hypersphere. • All data points lie on a hypersphere. Reduction Geometry • What is happening? Reduction Geometry • What is happening? Finally dist 2 u , v dist 2 q, S 1 k d k q 2 d 1 k k d k 1 k d d 1 • Additive constant depends only on dimension of points and subspaces. • This applies to linear subspaces, all of the same dimension. Extensions • subspaces of different dimension – lines and planes, e.g. – Not all data points have the same norm. • Add extra dimension to fix this. Extensions • subspaces of different dimension – lines and planes, e.g. – Not all data points have the same norm. • Add extra dimension to fix this. • affine subspaces Z iT x b – Again, not all data points have the same norm. Approximate Nearest Neighbor Search • Find point x with distance d(x, q) <= (1 + ε) mini d(xi,q) • Tree based approaches: KD-trees, metric / ball trees, cover trees • Locality sensitive hashing • This paper uses multiple KD-Trees with (different) random projections KD-Trees • Decompose space into axis aligned rectangles Image from Dan Pelleg Random Projections • Multiply data with a random matrix X with X(i,j) drawn from N(0,1) • Several different justifications – Johnson-Lindenstrauss (data set that is small compared to dimensionality) – Compressed Sensing (data set that is sparse in some linear basis) – RP-Trees (data set that has small doubling dimension) Results • Two goals – show their method is fast – show nearest subspace is useful • Four experiments – Synthetic Experiments – Image Approximation – Yale Faces – Yale Patches Image Reconstruction Yale Faces Questions / Issues • Should random projections be applied before or after the reduction? • Why does the effective distance error go down with the ambient dimensionality? • The reduction tends to make query points far away from the points in the database. Are there better approximate nearest neighbor algorithms in this case?

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 3 |

posted: | 8/1/2012 |

language: | |

pages: | 39 |

OTHER DOCS BY huanghengdong

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.