Document Sample

STEIN’s UNBIASED RISK ESTIMATOR (SURE) FOR OPTICAL FLOW Dr. Mingren Shi Mathematics and Computing University of Southern Queensland OUTLINE What is optical ﬂow? Overview: optical ﬂow estimation Lucas-Kanade Method Stein’s Unbiased Risk Estimator (SURE) Conclusion 1 What is optical ﬂow? Optical ﬂow is the apparent motion of the bright- ness/intensity patterns observed when a camera is moving relative to the objects being imaged. Optical ﬂow equation (OFE) • Let I = I(x1 , x2 , t) be image intensity (irradiance) function at time t at the image point (x1 , x2 ). • Let u = (u1 , u2 )T ≡ ( dx1 , dx2 )T be optical ﬂow. dt dt • Assume that the change of intensity of a particular point in a moving pattern with the time is very small, dI ∂I dx1 ∂I dx2 ∂I i.e. 0 ≈ = + + . (0.1) dt ∂x1 dt ∂x2 dt ∂t ∂I ∂I ∂I or u1 + u2 + ≈ 0. (0.1) ∂x1 ∂x2 ∂t • But the OFE (0.1) is ill-posed. Why? There are two variables (u1 , u2 ), but only one equation =⇒ it has inﬁnity many solutions. u2 u u1 Fig. 1: Optical ﬂow vector 2 • A constraint should be imposed on u. How to estimate optical ﬂow? • Many smoothing methods to estimate the optical ﬂow. Lucas-Kanade method Assumption: the optical ﬂow in a neigh- bourhood of the central pixel P is a constant. 3 3 3 P 3 Fig. 2: 3 × 3 neighbourhood size 3 The smoothing parameter is the neighborhood size n (for (2n + 1) × (2n + 1) pixels). Selection of this tuning parameter is diﬃcult in general. But has a crucial rule and a profound eﬀect on the results. • n too small =⇒ the numerical errors may be excessively ampliﬁed. • n too large =⇒ the solution may be not close to satisfying the real solution. • This is a tradeoﬀ situation. • There is an optimal value of the n size. • Example: Divergent tree (The optical ﬂow in each pixel is plotted in vector, see in Fig. 1.) image ﬁle: treed.mat data ﬁles: Treed1.dat - 11 × 11 (optimal) Treed2.dat - 3 × 3 (too small) Treed3.dat - 31 × 31 (too large) Treed4.dat - SURE curve 4 Lucas-Kanade Method Simpliﬁed natation ∂I ∂I ∂I It = , Ix 1 = , Ix2 = , ∂t ∂x1 ∂x2 (1.0) P = (x1 , x2 ) : (central) position or pixel Rewrite OFE (0.1) (wrt P ) as −It,P = u1,P Ix1 ,P + u2,P Ix2 ,P + P . (1.1) Moreover, let gP = (Ix1 ,P Ix2 ,P )T , uP = (u1,P u2,P )T , yP = −It,P , (1.2) and (1.1) becomes T T yP = gP uP + P = sP + P , (sP = gP uP ). (1.3) The local constant LK estimator • Assumptions NP is a pixel neighborhood of P . There are m = (2n + 1) × (2n + 1) pixels in NP . A1 1, . . . , m are independent. 5 A2 = ( 1, . . . , N )T ∼ N (0, σ 2 I) • Deﬁnition of the estimator uP = arg min T (yQ − gQ uP )2 . (1.4) Q∈NP • Euler equations wrt the estimator T uP =0⇒ (yQ − gQ uP )gQ = 0, Q∈NP T (1.5A) or ( gQ gQ )uP = yQ gQ Q∈NP Q∈NP Therefore, we have −1 T uP = MP yQ gQ , MP = gQ gQ . Q∈NP Q∈NP (1.5B) • Remark NP = {P1 , P2 , . . . , Pm } ⊃ {P } gQ = (Ix1 ,Q , Ix2 ,Q )T and set Ix1 ,P1 . . . Ix1 ,Pm AT 2×m = = [gP1 . . . gPm ], Ix2 ,P1 . . . Ix2 ,Pm y = −(It,P1 . . . It,Pm )T = (yP1 . . . yPm )T . MP = AT A is a 2 × 2 matrix, yQ gQ = AT y. Q∈NP T (1.5A) will become AT AuP = A y which is the “Regularized equation” of AuP = y. 6 Stein’s Unbiased Risk Estimator (SURE) • The SURE was originally used to estimate the mean of a multivariate normal distribution. • We use it to estimate the tuning parameters in the context of image restoration. Deﬁnition of SURE • Minimising the expected value of the discrep- ancy (E is the expectation): R = E( |gP (uP − uP )|2 ) T P = E( |sP − sP |2 ), T (sP = gP uP ), P = E( |(yP − sP ) − (yP − sP )|2 ) P (2.1) The sum is for every pixel P in the image. • Residual e and the error e = (yP1 − sP1 , . . . , yPN − sPN )T , (2.2) = (yP1 − sP1 , . . . , yPN − sPN )T . • The discrepancy measure (2.1) is R = E||e − ||2 = E||e||2 − 2E( T e) + E|| ||2 . (2.3) 7 An estimator of R (1) An unbiased estimator of E||e||2 is ||e||2 , which can be calculated directly once the ﬂow estimate u is obtained. 2 2 (2) Since E( P ) = 0, E( P ) = E( P ) − [E( P )]2 ≡ σ 2 , we have E( 2 ) = E( 2 P ) = N · E( 2 P ) = N σ2 . P (3) As usual, we must work on the middle term T E( e). Calculation of the middle term Stein’s Lemma Given the Gaussian assumption (A1 & A2) and u is weakly diﬀerentiable with re- spect to the data y, we have T 2 ∂e E( e) = σ E[trace( )]. (2.4) ∂y Proof The distribution density is p(y|u) ≡ p( ) ≡ p( 1 , . . . , N ) N = p( i ) by independent i=1 N 2 (2.5) = (2πσ 2 )−1/2 exp(− i 2σ 2 ) i=1 1 = (2πσ 2 )−N/2 exp[−( 2σ2 )|| ||2 ], 8 Furthermore, since || ||2 = 2 , ∂p ∂y = p(y|u)[− 2σ2 (2 )] = − p(y|u) , 1 σ2 (2.6) ⇒ ∂p ( ∂y )T e = − p(y|u) ( T e). σ2 Thus, T E( e) = [p(y|u)( T e)]dy ∂p = −σ 2 [( ∂y )T e] dy N 2 ∂ei =σ [p(y|u) ∂yi ]dy i=1 (integration by parts & p(±∞) = 0) ∂e = σ 2 [p(y|u)trace( ∂y )] dy ∂e = σ 2 E[trace( ∂y )]. (by the deﬁnition of E) (2.7) The calculation of ∂e/∂y ∂eP ∂ ∂ sP ∂yP = ∂yP (yP − sP ) = 1 − ∂yP ∂ T T ∂ uP =1− ∂yP (gP uP ) = 1 − gP ∂yP (2.8) T ∂ −1 = 1 − gP ∂y [MP yQ gQ ] P Q∈NP T = 1 − gP M −1 gP . from (1.5B) 9 From the above, an unbiased estimate of the risk is given by: ∂e R ≈ ||e||2 − 2σ 2 trace( ∂y ) + N σ 2 T = [(yP − sP )2 − 2σ 2 (1 − gP M −1 gP ) + σ 2 ] P (by (2.2), (2.8)) = 2 [(yP − 2yP sP + s2 ) P P T +2σ 2 gP M −1 gP − σ 2 ] (2.9) 2 We can drop yP and −σ 2 since both terms do not depend on the neighborhood size. So we can replace R by R as follows: R= RP , P (2.11) T −1 RP = −2yP sP + s2 + 2σ 2 gP MP gP . P 10 CONCLUSION The Optical ﬂow equation is ill-posed. An extra constraints should impose on the optical ﬂow vector u The smoothing parameter for the local constant Lucas-Kanade method is the neighbourhood size n which is tradeoﬀ. Stein’s Unbiased Risk Estimator can be used to ﬁnd the optimal value of the tuning parameter n. 11 Questions? Comments? Constructive Criticism? Suggestions? Thank you 12

OTHER DOCS BY lindahy

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.