Stereo Vision by lJt41jZz


									Computer Vision

    Stereo Vision
Pinhole Camera

Bahadir K. Gunturk   2
Perspective Projection

                     x' y' f '
                        
                     x  y  z

Bahadir K. Gunturk               3
Stereo Vision

    Two cameras.
    Known camera positions.
    Recover depth.

                                         scene point

                                 p                     p’
                                      image plane

                     optical center

Bahadir K. Gunturk                                          4

       p             p’

Bahadir K. Gunturk        5
Matrix form of cross product

                   a y bz  az by   0         az    a2 
                                                        
          a  b   az bx  ax bz    az        0    ax  b     a  b
                   ax by  a y bz   a y      ax     0 
                                                        

                               a  (a  b)  0
                               b  (a  b)  0

Bahadir K. Gunturk                                                            6
The Essential Matrix

       pT Ep '  0

 Essential matrix     p  (u, v,1)T
                     p '  (u ', v ',1)T
Bahadir K. Gunturk                         7
     Stereo Constraints
                            Image plane             Epipolar Line

                     Y1            p
Z1                            X1

                                                         O2         Z2
                          Focal plane

     Bahadir K. Gunturk                                             8
   A Simple Stereo System

              LEFT CAMERA                     RIGHT CAMERA

          Left image:                              Right image:
          reference                                target

                                 Depth Z

                               Elevation Zw
Zw=0 K. Gunturk
  Bahadir                                                         9
  Stereo View

       Left View                 Right View

Bahadir K. Gunturk   Disparity                10
 Stereo Disparity
    The separation between two matching objects
     is called the stereo disparity.

Bahadir K. Gunturk                                 11
Parallel Cameras
                                         T x x T
                                                      l

                                           Z f   Z
                         Z                               T
         xl                     xr
                                                  Z f
                                                       x x
f              pl              pr
                                                              l       r

          Ol                        Or
                                          Disparity:       d  x x
                                                                  l        r

    T is the stereo baseline

Bahadir K. Gunturk                                                    12
Finding Correspondences

Bahadir K. Gunturk        13
      LEFT IMAGE               Approach
                                   (x , y )
                                         l   l

   For Each point (xl, yl) in the left image, define a window
    centered at the point
Bahadir K. Gunturk                                               14
      RIGHT IMAGE             Approach
                                  (x , y )
                                       l   l

   … search its corresponding point within a search region in
    the right image
Bahadir K. Gunturk                                          15
      RIGHT IMAGE      (xr, yr)   Approach
                                   dx (x , y )
                                        l   l

   … the disparity (dx, dy) is the displacement when the
    correlation is maximum
Bahadir K. Gunturk                                          16
Comparing Windows        =
                     f         g


Bahadir K. Gunturk                 17
Comparing Windows

   Minimize          Sum of Squared

   Maximize          Cross correlation

Bahadir K. Gunturk                 18
Correspondence Difficulties
    Why is the correspondence problem difficult?
           Some points in each image will have no
            corresponding points in the other image.
           (1) the cameras might have different fields of view.
           (2) due to occlusion.
    A stereo system must be able to determine
     the image parts that should not be matched.

Bahadir K. Gunturk                                                19
     Structured Light
   Structured lighting
        Feature-based methods are not applicable when the
         objects have smooth surfaces (i.e., sparse disparity
         maps make surface reconstruction difficult).
        Patterns of light are projected onto the surface of
         objects, creating interesting points even in regions
         which would be otherwise smooth.
        Finding and matching such
         points is simplified by
         knowing the geometry of the
         projected patterns.

Bahadir K. Gunturk                                              20
   Stereo results
          Data from University of Tsukuba

                Scene                        Ground truth

Bahadir K. Gunturk
                               (Seitz)                      21
   Results with window correlation

    Estimated depth of field             Ground truth
     (a fixed-size window)
Bahadir K. Gunturk                                      22
    Results with better method

           A state of the art method                                  Ground truth
Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
   International Conference on Computer Vision, September 1999.

    Bahadir K. Gunturk                              (Seitz)                          23

To top