Docstoc

Computer Vision

Document Sample
Computer Vision Powered By Docstoc
					From Pixels to Features:
   Review of Part 1

      COMP 4900C
       Winter 2008
Topics in part 1 – from pixels to features
• Introduction
  • what is computer vision? It’s applications.
• Linear Algebra
  • vector, matrix, points, linear transformation, eigenvalue,
    eigenvector, least square methods, singular value decomposition.
• Image Formation
  • camera lens, pinhole camera, perspective projection.
• Camera Model
  • coordinate transformation, homogeneous coordinate,
    intrinsic and extrinsic parameters, projection matrix.
• Image Processing
  • noise, convolution, filters (average, Gaussian, median).
• Image Features
  • image derivatives, edge, corner, line (Hough transform), ellipse.
General Methods
• Mathematical formulation
   • Camera model, noise model

• Treat images as functions
              I  f ( x, y)
• Model intensity changes as derivatives f  [ I x , I y ]T
   • Approximate derivative with finite difference.

• First-order approximation
   I (i  u, j  v)  I (i, j )  I x u  I y v  I (i, j )  u v f
• Parameter fitting – solving an optimization
  problem
 Vectors and Points
We use vectors to represent points in 2 or 3 dimensions


    y
              v
                    Q(x2,y2)
                                          x2  x1 
 P(x1,y1)                       v QP           
                                          y2  y1 
                       x


 The distance between the two points:

            D  Q  P  ( x2  x1 )  ( y2  y1 )
                                      2             2
 Homogeneous Coordinates
Go one dimensional higher:
                     wx             wx 
             x            x  
                              y    wy 
             y    wy       wz 
             
                    w
                           z  
                              
                                     w
w is an arbitrary non-zero scalar, usually we choose 1.

From homogeneous coordinates to Cartesian coordinates:

                              x1 
        x1                  x   x1 / x4 
        x    x1 / x3     2    x2 / x4 
        2   x2 / x3 
                            x3            
        x3 
                            x   x3 / x4 
                                              
                              4
2D Transformation with Homogeneous Coordinates

2D coordinate transformation:

                    cos  sin    p x  Tx 
            p' '                  p   T 
                    sin  cos    y   y 


2D coordinate transformation using homogeneous coordinates:


         p x ' '  cos       sin  Tx   p x 
         p ' '   sin      cos  Ty   p y 
         y                             
         1   0
                               0   1  1 
                                          
Eigenvalue and Eigenvector

We say that x is an eigenvector of a square matrix A if

                      Ax  x
  is called eigenvalue and   x   is called eigenvector.
The transformation defined by A changes only the
magnitude of the vector x

Example:

3 2 1 5      1     3 2  2   4       2
1 4 1  5  51 and 1 4  1   2  2 1
                                     
                             1     2
5 and 2 are eigenvalues, and 1 and 1 are eigenvectors.
                                    
Symmetric Matrix
We say matrix A is symmetric if

                     AT  A
 Example:   BT B is symmetric for any B, because
            ( B T B )T  B T ( B T )T  B T B
A symmetric matrix has to be a square matrix


 Properties of symmetric matrix:
    •has real eignvalues;
    •eigenvectors can be chosen to be orthonormal.
    • BT B has positive eigenvalues.
Orthogonal Matrix

A matrix A is orthogonal if

                                           1
            A A I
             T
                            or     A AT


The columns of A are orthogonal to each other.

 Example:

    cos         sin           1cos         sin  
 A                             A 
    sin        cos 
                                     sin       cos   
Least Square

When m>n for an m-by-n matrix A,      Ax  b   has no solution.
 In this case, we look for an approximate solution.
 We look for vector x such that

                     Ax  b
                               2


is as small as possible.

This is the least square solution.
 Least Square

Least square solution of linear system of equations

                        Ax  b
 Normal equation:   A Ax  A b
                        T          T


   T
 A A     is square and symmetric


                                           1
The Least square solution   x  ( A A) A b
                                       T        T


          Ax  b
                    2
 makes                  minimal.
SVD: Singular Value Decomposition
An mn matrix A can be decomposed into:

                   A  UDV     T


U is mm, V is nn, both of them have orthogonal columns:

            U U I
               T
                               V V I
                                   T


D is an mn diagonal matrix.

Example:

            2 0  1 0 0 2 0
            0  3  0  1 0 0 3 1 0
                                0 1 
             0 0  0 0 1   0 0  
                               
                                           
Pinhole Camera
Why Lenses?




              Gather more light from each scene
              point
 Four Coordinate Frames
                                                            Pw
                                        pixel
                                        frame
                                  xim                            Yw
                                                    Zw
                                  pim      yim
              Yc            y

                   Xc                                    world
                                                                      Xw
     camera
     frame                                               frame
                          image     x
                   Zc
                          plane
       optical            frame
       center                           principal
                                        point




                              transformation 
Camera model:           pim                  Pw
                                  matrix     
Perspective Projection
                                      P
                                                             X      Y
                  p                                     x f   y f
              y                                              Z      Z
                      x

                  principal   principal
                                                        These are nonlinear.
    optical       point       axis
    center
                  image
                  plane



 Using homogenous coordinate, we have a linear relation:
                                                    X 
                      u   f            0    0 0  
                      v  0            f       Y 
                                               0 0
                                                 Z 
                       w  0
                                       0    1 0  
                                                   1
                                                     
                          x u/w              y  v/w
World to Camera Coordinate
Transformation between the camera and world coordinates:


                                 X c  RX w  T


                                  Xc              X w
                                  Y            T  Yw 
                                   c   R         
                                   Z c  0 
                                          
                                                   Z 
                                                 1 w
                                                   
             R,T                  1                1 
      Image Coordinates to Pixel Coordinates

                        x  (ox  xim ) s x y  (o y  yim ) s y
                        s x , s y : pixel sizes
      xim
yim          y

       x (ox,oy)       xim   1 / s x             0       ox   x 
                      y    0                   1/ s y   oy   y
                       im                                     
                      1  0
                                                  0       1  1 
                                                                 
Put All Together – World to Pixel
     x1   1 / s x     0        ox   u 
    x    0          1/ sy     oy   v 
     2                              
     x3   0
                       0        1   w
                                       
                                                         X 
           1 / s x      0        ox   f     0   0 0  c 
          0                      oy   0         0 0  c 
                                                          Y
                       1/ sy                f       Z 
           0
                         0        1  0
                                              0   1 0  c 
                                                        1
                                                          
                                                            X w
           1 / s x      0        ox   f    0 0
                                                0            
          0           1/ sy     oy   0         R T   Yw 
                                             f 0 0 
                                                       
           0                      1  0    0 1 0  0 1  Zw 
                         0                               
                                                             1 
                                                             Xw              X w
           f / s x       0         ox  1 0 0 0                          Y 
          0            f / sy         0 1 0 0  R T   Yw   K R   T  w 
                                    oy           0 1  Z                 Zw 
           0
                             0      1  0 0 1 0  
                                                 
                                                            w
                                                              
                                                              1               1 
                                                                                

      xim  x1 / x3           yim  x2 / x3
Camera Intrinsic Parameters

                f / s x       0       ox 
            K  0            f / sy   oy 
                                          
                0
                               0       1 

K is a 3x3 upper triangular matrix, called the
Camera Calibration Matrix.
There are five intrinsic parameters:
(a) The pixel sizes in x and y directions s x , s y
(b) The focal length f
(c) The principal point (ox,oy), which is the point
where the optic axis intersects the image plane.
Extrinsic Parameters

                                   X w     X w
                   x1            Y       Y 
            pim   x2   K R T  w   M  w 
                                  Zw      Zw 
                   x3 
                                          
                                    1       1 


[R|T] defines the extrinsic parameters.
The 3x4 matrix M = K[R|T] is called the projection matrix.
   Image Noise
Additive and random noise:
                       I  x, y   I  x, y   n  x, y 
                       ˆ


I(x,y) : the true pixel values
n(x,y) : the (random) noise at pixel (x,y)




                                        
Gaussian Distribution
Single variable


                   1     ( x   ) 2 / 2 2
         p( x)       e
                  2
Gaussian Distribution

Bivariate with zero-means and variance 2

                                   x 2  y 2  
            G x, y  
                          1
                               exp 
                                                
                                                 
                         2 2
                                      2  2
                                                 
 Gaussian Noise
Is used to model additive random noise
                                   n2
                                   2 2
•The probability of n(x,y) is e
•Each has zero mean
•The noise at each pixel is independent
  Impulsive Noise
   • Alters random pixels
   • Makes their values very different from the true ones



Salt-and-Pepper Noise:
   • Is used to model impulsive noise

                                                              I h, k        xl
                                        I sp h, k   
                                                       imin  yimax  imin  x  l
                                          x, y are uniformly distributed random
                                               variables
                                           l , im in,im axare constants
Image Filtering
Modifying the pixels in an image based on some
 function of a local neighbourhood of the pixels




      N(p)
  p
         10   30   10
                          f  p
         20   11   20                  5.7
         11   9    1
 Linear Filtering – convolution
The output is the linear combination of the neighbourhood pixels
                           m/2        m/2
 I A (i, j )  I * A        A(h, k ) I (i  h, j  k )
                         h m / 2 k  m / 2

 The coefficients come from a constant matrix A, called kernel.
 This process, denoted by ‘*’, is called (discrete) convolution.



   1    3      0          1       0         -1
   2    10     2         1       0.1 -1         =       5
   4    1      1          1       0         -1
       Image                   Kernel                Filter Output
Smoothing by Averaging




                       1   1    1
                   1                  
                 *     1   1    1
                   9
                       1   1    1


Convolution can be understood as weighted averaging.
Gaussian Filter


                       x 2  y 2  
G x, y  
               1
                   exp 
                                    
                                     
             2 2
                          2  2
                                     


 Discrete Gaussian kernel:
                                   h2 k 2
                   1           
 G (h, k )                e        2 2
               2     2



where Gh, k  is an element of an m  m array
Gaussian Filter




           *             




                   1
Gaussian Kernel is Separable
   IG  I  G 
                m/2          m/2
                 G(h, k ) I (i  h, j  k ) 
              h m / 2 k  m / 2
                 m/2            m/2        h2 k 2
                                       
                 e
               h m / 2 k  m / 2
                                            2 2
                                                      I (i  h, j  k ) 
                            2
                m/2  h            m/2                k2
                                                 
                e
              h m / 2
                          2 2
                                      e
                                    k  m / 2
                                                     2 2
                                                              I (i  h, j  k )

                          h2 k 2                h2               k2
                                                            
  since                    2
                                    e          2                2 2
                                2                     2
                  e                                       e
 Gaussian Kernel is Separable
Convolving rows and then columns with a 1-D Gaussian kernel.


                      1
     I                    1 9 18 9     1    =        Ir
                      38

                            1
                            9
                      1
     Ir                   18     =         result
                      38
                            9
                            1


The complexity increases linearly with m instead of with m 2
                                                           .
Gaussian vs. Average




  Gaussian Smoothing   Smoothing by Averaging
Nonlinear Filtering – median filter


Replace each pixel value I(i, j) with the median of the values
found in a local neighbourhood of (i, j).
Median Filter




  Salt-and-pepper noise   After median filtering
 Edges in Images
Definition of edges
 •   Edges are significant local changes of intensity in an image.
 •   Edges typically occur on the boundary between two different regions in an image.
Images as Functions

2-D




 Red channel intensity   I  f ( x, y)
Finite Difference – 2D
Continuous function:

       f x, y         f x  h, y   f x, y 
                   lim
          x        h 0            h
       f  x, y         f  x, y  h   f  x, y 
                    lim
          y         h 0             h

Discrete approximation:                                 Convolution kernels:

             f x, y 
        Ix              f i 1, j  f i , j              1 1
                x
             f x, y                                       1
        Iy              f i , j 1  f i , j              1
                y                                           
Image Derivatives




                    I x  I *  1 1




      Image I




                              1
                     Iy  I * 
                             1
Edge Detection using Derivatives


1-D image f (x)



1st derivative f ' ( x)


 f ' ( x) threshold


Pixels that pass the
threshold are
edge pixels
Image Gradient
                 gradient
                                   fx 
                                    
                             f   f 
                                   y 
                 magnitude

                             f          
                                         f 2
                                         x
                                                  f 2
                                                  y



                 direction
                             arctan(fy / fx )
                                         
 Finite Difference for Gradient
Discrete approximation:                               Convolution kernels:

                       f
         I x (i, j )      f i 1, j  f i , j          1 1
                       x
                      f                                   1
        I y (i, j )      f i , j 1  f i , j           1
                      y                                   

magnitude       G(i, j )  I (i, j )  I (i, j )
                                       2
                                       x
                                                  2
                                                  y


aprox. magnitude              G(i, j )  I x  I y

 direction        arctan( I y / I x )
  Edge Detection Using the Gradient
Properties of the gradient:
   • The magnitude of gradient
     provides information about the
     strength of the edge
   • The direction of gradient is
     always perpendicular to the
     direction of the edge


Main idea:
   • Compute derivatives in x and y directions
   • Find gradient magnitude
   • Threshold gradient magnitude
   Edge Detection Algorithm



               1 1   Ix
          *                                         edges
                              Ix  Iy
                               2    2


                                        Threshold
Image I

          *     1
               1       Iy
                
    Edge Detection Example

                     Ix


I




                    Iy
    Edge Detection Example

        G(i, j )  I x (i, j )  I y (i, j )
                     2             2




I




          G(i, j )  Threshold  
Finite differences responding to noise




      Increasing noise ->
      (this is zero mean additive gaussian noise)
Solution: smooth first




Where is the edge? Look for peaks in
 Sobel Edge Detector
Approximate derivatives with
central difference                            Convolution kernel

   I x (i, j ) 
                 f
                     f i 1, j  f i 1, j     1   0  1
                 x

                                               1 0  1 
Smoothing by adding 3 column                    2 0  2
neighbouring differences and give                      
more weight to the middle one                  1 0  1 
                                                       


                                               1    2  1
Convolution kernel for I y                     0    0  0
                                                          
                                                1  2  1
                                                          
Sobel Operator Example
          a1    a2    a3            1 0  1 
          a4    a5    a6             2 0  2
                               *            
          a7    a8    a9            1 0  1 
                                            

          a1    a2    a3
                                    1    2  1
          a4    a5    a6            0       0
                               *         0     
          a7    a8    a9             1  2  1
                                               

The approximate gradient at a 5
       I x  (a1  a3 )  2(a4  a6 )  (a7  a9 )
       I y  (a1  a7 )  2(a2  a8 )  (a3  a9 )
   Sobel Edge Detector

              1 0  1 
               2 0  2
                            Ix
          *   1 0  1 
                                      Ix  Iy
                                         2    2
                                                              edges
                                                  Threshold
Image I

          *   1    2  1
              0                   Iy
                   0  0 
               1  2  1
                         
Edge Detection Summary

Input: an image I and a threshold .

1. Noise smoothing: I s  I  h
   (e.g. h is a Gaussian kernel)

2. Compute two gradient images I x and I y by convolving I s
with gradient kernels (e.g. Sobel operator).

3. Estimate the gradient magnitude at each pixel

         G(i, j )  I x (i, j )  I y (i, j )
                      2             2



4. Mark as edges all pixels (i, j ) such that G(i, j )  
Corner Feature

Corners are image locations that have large intensity changes
in more than one directions.

Shifting a window in any direction should give a large
  change in intensity
 Harris Detector: Basic Idea




  “flat” region:        “edge”:                “corner”:
  no change in          no change along        significant change
  all directions        the edge direction     in all directions


C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
  Change of Intensity

The intensity change along some direction can be quantified
by sum-of-squared-difference (SSD).

               D(u, v)   I (i  u, j  v)  I (i, j )
                                                           2

                          i, j


                                 u 
                                 v 
                                  
                                        I (i  u, j  v)


                           I (i, j )
 Change Approximation
 If u and v are small, by Taylor theorem:
             I (i  u, j  v)  I (i, j )  I x u  I y v
                 I               I
 where      Ix       and    Iy 
                 x               y

therefore

I (i  u, j  v)  I (i, j )        I (i, j )  I xu  I y v  I (i, j ) 
                                 2                                           2



                                      I xu  I y v 
                                                      2


                                      I x u 2  2 I x I y uv  I y v 2
                                         2                        2


                                             Ix 2
                                                            I x I y  u 
                                      u v                   2  
                                            I x I y
                                                            I y  v 
                                                                    
Gradient Variation Matrix

                     Ix 2
                                  I I          u 
   D(u, v)  u v                      x y
                                                
                    I x I y     I      2
                                         y     v 
                                               

This is a function of ellipse.

                  Ix 2
                                 I I    
              C                  x y
                                         
                 I x I y       I 2
                                   y    
                                         

 Matrix C characterizes how intensity changes
 in a certain direction.
   Eigenvalue Analysis

            Ix 2
                          I I             0
                                        T 1
        C                    x y
                                     Q       Q
           I x I y      I              0 2 
                                2
                               y    
                                     
 If either λ is close to 0, then this is not a corner, so look for
      locations where both are large.
                                   I 
                                     x
                                         
                                C    Ix   Iy   
                                    I y 


     I x 
              
C    I x I y  AT A
     I y                                            (max)-1/2
•C is symmetric
•C has two positive eigenvalues                                    (min)-1/2
Corner Detection Algorithm
Line Detection




The problem:
•How many lines?
•Find the lines.
Equations for Lines
    The slope-intercept equation of line

                 y  ax  b
    What happens when the line is vertical? The slope a goes
    to infinity.


     A better representation – the polar representation




                 x cos   y sin 
Hough Transform: line-parameter mapping

A line in the plane maps to a point in the - space.


                                       (,)
        


All lines passing through a point map to a sinusoidal curve in the
- (parameter) space.


                                 



                x cos   y sin 
Mapping of points on a line



                             



Points on the same line define curves in the parameter space
that pass through a single point.

Main idea: transform edge points in x-y plane to curves in the
parameter space. Then find the points in the parameter space
that has many curves passing through.
Quantize Parameter Space

                                    




                              m                                      
                                                                     m


Detecting Lines by finding maxima / clustering in parameter space.
Examples

 input image   Hough space               lines detected




                  Image credit: NASA Dryden Research Aircraft Photo Archive
Algorithm
1. Quantize the parameter space
   int P[0, max][0, max]; // accumulators

2. For each edge point (x, y) {
     For ( = 0;  <= max;  = +) {
         x cos   y sin  // round off to integer
       (P[][])++;
     }
   }

3. Find the peaks in P[][].
Equations of Ellipse

                           x2 y 2
                             2
                                2 1
                           r1 r2



             ax 2  bxy  cy 2  dx  ey  f  0

                  Let   x  [ x 2 , xy, y 2 , x, y,1]T

                        a  [a, b, c, d , e, f ]T
                  Then xT a  0
Ellipse Fitting: Problem Statement

            Given a set of N image points         p i  [ xi , yi ]T
            find the parameter vector a 0such that the ellipse

                           f (p, a)  xT a  0

             fits p i best in the least square sense:

                                N
                          min
                           a
                                 D(p i , a)2
                                i 1




        Where D(p i , a) is the distance from p i to the ellipse.
Euclidean Distance Fit


   pi                   D(pi , a)  pi  pi
                                    ˆ
        ˆ
        pi

                        ˆ
                       p i is the point on the ellipse that
                       is nearest to p i

                             f (p i , a)  0
                                ˆ

               p i  p i is normal to the ellipse at p i
               ˆ                                     ˆ
Compute Distance Function
Computing the distance function is a constrained optimization
problem:
           min p i  p i
                           2
             ˆ
             pi
               ˆ                subject to            f (p i , a)  0
                                                         ˆ

Using Lagrange multiplier, define:
              L( x, y,  )  pi  pi  2f (pi , a)
                                     2
                             ˆ              ˆ

 where p i  [ x, y ]T
       ˆ

Then the problem becomes: min L( x, y,  )
                                         ˆ
                                         pi

        L L                                 p i  p i  f (p i , a)
                                              ˆ                ˆ
 Set         0           we have
        x y
Ellipse Fitting with Euclidean Distance

 Given a set of N image points p i  [ xi , yi ]
                                                        T



 find the parameter vector a 0such that


                          N                     2
                                  f (p i , a)
                    min 
                                 f (p i , a)
                      a                             2
                          i 1




  This problem can be solved by using a numerical nonlinear
  optimization system.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:4/13/2013
language:English
pages:72