Docstoc

Introduction to Multimedia Systems

Document Sample
Introduction to Multimedia Systems Powered By Docstoc
					Cameras and Projectors

         Hao Jiang
Computer Science Department
      Boston College
       Oct. 2, 2007


     CS335 Principles of Multimedia Systems
            Cameras and Projectors
 Cameras and projectors have been intensively used
  in many different multimedia applications.

 It is important to understand techniques to
  manipulate these devices.

 We will study basic methods about
   – calibration,
   – image warping and blending,
   – and other issues in building a camera/projector system.




                     CS335 Principles of Multimedia Systems
       Example Application: Projector
     Geometry Distortion Compensation




Automatically correcting projector geometry distortion using
a camera and projector system.
                     CS335 Principles of Multimedia Systems
   Camera / Projector Geometry Model
 Camera and projector can be modeled as a pinhole
  imaging system.




                                Optical Center             Object Point
                        Focal length
Optical
axis          Image
              point


          Image Plane


                  CS335 Principles of Multimedia Systems
           Camera Geometry Model
 A camera projects a 3D point onto a 2D point in a
  image.


                         y
               x
                                                     3D point (X, Y, Z)
                                                     in camera’s frame is
                                                      (X’,Y’,Z’)
         (Ox, Oy)

           f                       x = (f X’/Z’)/dx + Ox = fx X’/Z’ + Ox
                                   y = (f Y’/Z’)/dy + Oy = fy Y’/Z’ + Oy
                          dx and dy are width and height of an image
                          pixel
                    CS335 Principles of Multimedia Systems
           Homogenous Coordinate
 The homogenous coordinate of a 3D point (x,y,z) is
   (X,Y,Z,W) where X/W = X, Y/W=Y and Z/W=Z.


 The homogenous coordinate of a 2D point (x,y) is
  (X,Y,W) where X/W = x, Y/W=Y.

 We can convert a 3D point from one frame to
  another, by simply T*P, where T is a 4x4 matrix and
  P is the homogeneous coordinate of the 3D point.




                   CS335 Principles of Multimedia Systems
                   Camera Matrix
 Using homogenous coordinate, the camera projection
  can be represented as

                                                            X
        x          a11 a12 a13 a14
                                             P = A3x4       Y   =AP
     p= y     ~    a21 a22 a23 a24
                                                            Z
        w          a31 a32 a33 a34
                                                            W


 Where A3x4 is a 3 rows and 4 columns matrix, called
  camera matrix.


                   CS335 Principles of Multimedia Systems
Projection from a Plane to Another Plane
                                Z
                                       X
                              Y
                                         P




          p
                      Since         p=AP               We have
   O                           x   a11 a12 a14             X
                     p=        y ~ a21 a22 a24             Y = H p’
                               w   a31 a32 a34             1

                     H is defines a Homography.
              CS335 Principles of Multimedia Systems
               The Projector Model
 Projector shares similar model with a camera.

 The only difference is that a projector projects a flat
  object that emits lights based on a computer image
  onto an image plane, the screen.

 Based on the previous analysis about planar object
  projection, the image from the computer and the one
  projected on the screen are related by a
  homography.



                   CS335 Principles of Multimedia Systems
      The Camera-Projector System
                                           r


                                                           H2*r
          H1*p
                   Image on the screen


                                                                  q
            p                H2*H1*p


Image sent to projector                         Camera image

                  CS335 Principles of Multimedia Systems
                    Calibration
 We would like to compute H1 and H2.

 We project marker points on the screen and form
  equations

            a11Xn + a12Yn + a14
                                          = xn
            a31Xn + a32Yn + a34

            a21Xn + a22Yn + a24                            n = 1 .. N
                                           = yn
            a31Xn + a32Yn + a34                            a34=1

                  CS335 Principles of Multimedia Systems
                       Image Adjustment (I)

Assume that                                  Image on the screen
the viewer is
close to the camera.

                                                   H2*H1*p
                        H1*p



                       p                                q    The desired
                               Pre-warping                   Image.

            Projector image                  Camera image

   Each p is projected to the cameraofimage and the color is determined
                          CS335 Principles Multimedia Systems
   by color interpolation in the desired image.
             Image Adjustment (II)

                                        Image on the screen
The desired image
                                         Estimate H2 using
                                         screen corners projections.

                                               H2*H1*p
               H1*p




                       Pre-warping

      Project image                     Camera image

                 CS335 Principles of desired image
  Pre-warp image based on the Multimedia Systems on the screen.
    Projection onto Arbitrary Surfaces
 We have learned how to deal with projector distortion
  using a planar screen.

 We can extend the method into other types of
  surfaces, such as cylinder or spheres.

 We need a relative dense mesh grid to capture the
  local deformation model.




                  CS335 Principles of Multimedia Systems
Titled Large Screen Display




      CS335 Principles of Multimedia Systems
         Titled Large Screen Display
 LCD or CRT are still not easy to be made as large as
  a wall.

 Titling multiple projector images into a large screen
  display is flexible and relatively cheap.

 The shortcoming is we need to align the images from
  different projectors in both geometry and color.

 Manually adjusting the projectors is a tedious task.

 Camera project system can be used to solve the
  problem.
                   CS335 Principles of Multimedia Systems
                          Titled Images


                                 2                  3              4
Calibration
Patterns           1
(Projector to
Camera                            6             7
Homography            5                                             8
can be computed
based on these
patterns)
                  9                           11
                           10                                      12

                  Projector images

                                                The camera view
                          CS335 Principles of Multimedia Systems
projector image   Large Screen Projection
             3

        p          pcamera = Hqprojector


                                    2                  3           4          The
                      1                                q                      big
                                     6                                        image
                                                   7                8
                      5


                  9                              11
                              10                                   12

                  Projector images
                          CS335 Principles of Multimedia Systems   The camera view
          Edge and Color Alignment
 Colors of different projectors are usually different. A
  color calibration and adjustment procedure is
  needed.

 Blending

         Color(p) = a Color(2,p) + (1-a)Color(3,p)

                            2                 3
                                     p

        a is determined by the dominance of 2 or 3.
                   CS335 Principles of Multimedia Systems
                        Blending
 The size of the blending region has to be carefully
  selected
    a                                                 1-a




                     Blending region
 If the region is too small, large scale structures will
  show abrupt changes. If it is too big, small structures
  (edges, dots) will overlap in a big region and
  therefore will result in blur (ghost) images.
                   CS335 Principles of Multimedia Systems
               Multi-band Blending
 Multi-band blending can be used to address the
  problem.

 In multi-band blending,
   – images are filtered into different bands.
   – The mask is also low-pass filtered to generate mask for each
     band images.
   – Images are blending in each channel.
   – The blended images are summed up to get the final result.




                    CS335 Principles of Multimedia Systems
Image Pyramids




   The Gaussian Pyramid

 CS335 Principles of Multimedia Systems
Image Pyramids




 The Laplacian Pyramid

 CS335 Principles of Multimedia Systems
Image Pyramids




The Gaussian pyramid of the mask

 CS335 Principles of Multimedia Systems
Multi-band Blending




     Multi-band blending

   CS335 Principles of Multimedia Systems
Multi-band Blending




   CS335 Principles of Multimedia Systems
 System Issues of Large Screen Display
 Large screen projection needs multiple projectors
  working simultaneously.
   – The first structure uses share memory system such as SGI
     Oynx2, that employs a shared-memory model. A single
     program can have different threads writing OpenGL
     primitives into different pipes while reading from a single
     shared database and synchronizing display update over
     shared flags.

   – The second structure is PC cluster, in which each PC
     handles one projector. This framework is much cheaper but
     the synchronization is a challenge problem.




                     CS335 Principles of Multimedia Systems
                   Applications
 Visualization and Collaboration




   IEEE Computer Graphics and Applications, 2000
                  CS335 Principles of Multimedia Systems
               Immersive Reality
 The CAVE (University of Illinois at Chicago)




     IEEE Computer Graphics and Applications, 2000
                  CS335 Principles of Multimedia Systems
Projection for Augmented Environment




 Anton Treskunov and Jarrell Pair, PROJECTOR-CAMERA SYSTEMS FOR
 IMMERSIVE TRAINING, ASC06

                   CS335 Principles of Multimedia Systems
     Input Methods for Large Screens
 It presents challenges for traditional input methods.

 Possible HCI methods include:
   – Pointing devices, including 3D tracking, passive optical
     (video) tracking, ultrasonic tracking, mice, and tablet
     interfaces;
   – User tracking, for point-of-view rendering or for gaze directed
     interaction, via optical tracking or electromagnetic tracking;
   – Handheld devices, providing control interfaces that can be
     out of band from the display;
   – Voice commands with audio feedback;
   – And haptics interfaces.



                     CS335 Principles of Multimedia Systems
           Multiple Camera System
 Multiple camera system can capture video from
  different locations simultaneously.

 Stereo system has been widely used for inferring the
  “depth” of objects.




                  CS335 Principles of Multimedia Systems
3D reconstruction from Multiple Views
     CS335 Principles of Multimedia Systems
  Application of Multi-camera Imaging
 Image based rendering
 The Matrix




                CS335 Principles of Multimedia Systems
Cameras in the Matrix




   CS335 Principles of Multimedia Systems
       Camera Calibration Toolboxes
 OpenCV

 Matlab Calibration Toolbox
   – http://www.vision.caltech.edu/bouguetj/calib_doc/


 Multiple Camera Calibration
   – http://cmp.felk.cvut.cz/~svoboda/SelfCal/




                     CS335 Principles of Multimedia Systems

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:16
posted:7/16/2011
language:English
pages:36