# Introduction to Multimedia Systems

Document Sample

```					Cameras and Projectors

Hao Jiang
Computer Science Department
Boston College
Oct. 2, 2007

CS335 Principles of Multimedia Systems
Cameras and Projectors
 Cameras and projectors have been intensively used
in many different multimedia applications.

 It is important to understand techniques to
manipulate these devices.

 We will study basic methods about
– calibration,
– image warping and blending,
– and other issues in building a camera/projector system.

CS335 Principles of Multimedia Systems
Example Application: Projector
Geometry Distortion Compensation

Automatically correcting projector geometry distortion using
a camera and projector system.
CS335 Principles of Multimedia Systems
Camera / Projector Geometry Model
 Camera and projector can be modeled as a pinhole
imaging system.

Optical Center             Object Point
Focal length
Optical
axis          Image
point

Image Plane

CS335 Principles of Multimedia Systems
Camera Geometry Model
 A camera projects a 3D point onto a 2D point in a
image.

y
x
3D point (X, Y, Z)
in camera’s frame is
(X’,Y’,Z’)
(Ox, Oy)

f                       x = (f X’/Z’)/dx + Ox = fx X’/Z’ + Ox
y = (f Y’/Z’)/dy + Oy = fy Y’/Z’ + Oy
dx and dy are width and height of an image
pixel
CS335 Principles of Multimedia Systems
Homogenous Coordinate
 The homogenous coordinate of a 3D point (x,y,z) is
(X,Y,Z,W) where X/W = X, Y/W=Y and Z/W=Z.

 The homogenous coordinate of a 2D point (x,y) is
(X,Y,W) where X/W = x, Y/W=Y.

 We can convert a 3D point from one frame to
another, by simply T*P, where T is a 4x4 matrix and
P is the homogeneous coordinate of the 3D point.

CS335 Principles of Multimedia Systems
Camera Matrix
 Using homogenous coordinate, the camera projection
can be represented as

X
x          a11 a12 a13 a14
P = A3x4       Y   =AP
p= y     ~    a21 a22 a23 a24
Z
w          a31 a32 a33 a34
W

Where A3x4 is a 3 rows and 4 columns matrix, called
camera matrix.

CS335 Principles of Multimedia Systems
Projection from a Plane to Another Plane
Z
X
Y
P

p
Since         p=AP               We have
O                           x   a11 a12 a14             X
p=        y ~ a21 a22 a24             Y = H p’
w   a31 a32 a34             1

H is defines a Homography.
CS335 Principles of Multimedia Systems
The Projector Model
 Projector shares similar model with a camera.

 The only difference is that a projector projects a flat
object that emits lights based on a computer image
onto an image plane, the screen.

 Based on the previous analysis about planar object
projection, the image from the computer and the one
projected on the screen are related by a
homography.

CS335 Principles of Multimedia Systems
The Camera-Projector System
r

H2*r
H1*p
Image on the screen

q
p                H2*H1*p

Image sent to projector                         Camera image

CS335 Principles of Multimedia Systems
Calibration
 We would like to compute H1 and H2.

 We project marker points on the screen and form
equations

a11Xn + a12Yn + a14
= xn
a31Xn + a32Yn + a34

a21Xn + a22Yn + a24                            n = 1 .. N
= yn
a31Xn + a32Yn + a34                            a34=1

CS335 Principles of Multimedia Systems

Assume that                                  Image on the screen
the viewer is
close to the camera.

H2*H1*p
H1*p

p                                q    The desired
Pre-warping                   Image.

Projector image                  Camera image

Each p is projected to the cameraofimage and the color is determined
CS335 Principles Multimedia Systems
by color interpolation in the desired image.

Image on the screen
The desired image
Estimate H2 using
screen corners projections.

H2*H1*p
H1*p

Pre-warping

Project image                     Camera image

CS335 Principles of desired image
Pre-warp image based on the Multimedia Systems on the screen.
Projection onto Arbitrary Surfaces
 We have learned how to deal with projector distortion
using a planar screen.

 We can extend the method into other types of
surfaces, such as cylinder or spheres.

 We need a relative dense mesh grid to capture the
local deformation model.

CS335 Principles of Multimedia Systems
Titled Large Screen Display

CS335 Principles of Multimedia Systems
Titled Large Screen Display
 LCD or CRT are still not easy to be made as large as
a wall.

 Titling multiple projector images into a large screen
display is flexible and relatively cheap.

 The shortcoming is we need to align the images from
different projectors in both geometry and color.

 Camera project system can be used to solve the
problem.
CS335 Principles of Multimedia Systems
Titled Images

2                  3              4
Calibration
Patterns           1
(Projector to
Camera                            6             7
Homography            5                                             8
can be computed
based on these
patterns)
9                           11
10                                      12

Projector images

The camera view
CS335 Principles of Multimedia Systems
projector image   Large Screen Projection
3

p          pcamera = Hqprojector

2                  3           4          The
1                                q                      big
6                                        image
7                8
5

9                              11
10                                   12

Projector images
CS335 Principles of Multimedia Systems   The camera view
Edge and Color Alignment
 Colors of different projectors are usually different. A
color calibration and adjustment procedure is
needed.

 Blending

Color(p) = a Color(2,p) + (1-a)Color(3,p)

2                 3
p

a is determined by the dominance of 2 or 3.
CS335 Principles of Multimedia Systems
Blending
 The size of the blending region has to be carefully
selected
a                                                 1-a

Blending region
 If the region is too small, large scale structures will
show abrupt changes. If it is too big, small structures
(edges, dots) will overlap in a big region and
therefore will result in blur (ghost) images.
CS335 Principles of Multimedia Systems
Multi-band Blending
 Multi-band blending can be used to address the
problem.

 In multi-band blending,
– images are filtered into different bands.
– The mask is also low-pass filtered to generate mask for each
band images.
– Images are blending in each channel.
– The blended images are summed up to get the final result.

CS335 Principles of Multimedia Systems
Image Pyramids

The Gaussian Pyramid

CS335 Principles of Multimedia Systems
Image Pyramids

The Laplacian Pyramid

CS335 Principles of Multimedia Systems
Image Pyramids

The Gaussian pyramid of the mask

CS335 Principles of Multimedia Systems
Multi-band Blending

Multi-band blending

CS335 Principles of Multimedia Systems
Multi-band Blending

CS335 Principles of Multimedia Systems
System Issues of Large Screen Display
 Large screen projection needs multiple projectors
working simultaneously.
– The first structure uses share memory system such as SGI
Oynx2, that employs a shared-memory model. A single
program can have different threads writing OpenGL
primitives into different pipes while reading from a single
shared database and synchronizing display update over
shared flags.

– The second structure is PC cluster, in which each PC
handles one projector. This framework is much cheaper but
the synchronization is a challenge problem.

CS335 Principles of Multimedia Systems
Applications
 Visualization and Collaboration

IEEE Computer Graphics and Applications, 2000
CS335 Principles of Multimedia Systems
Immersive Reality
 The CAVE (University of Illinois at Chicago)

IEEE Computer Graphics and Applications, 2000
CS335 Principles of Multimedia Systems
Projection for Augmented Environment

Anton Treskunov and Jarrell Pair, PROJECTOR-CAMERA SYSTEMS FOR
IMMERSIVE TRAINING, ASC06

CS335 Principles of Multimedia Systems
Input Methods for Large Screens
 It presents challenges for traditional input methods.

 Possible HCI methods include:
– Pointing devices, including 3D tracking, passive optical
(video) tracking, ultrasonic tracking, mice, and tablet
interfaces;
– User tracking, for point-of-view rendering or for gaze directed
interaction, via optical tracking or electromagnetic tracking;
– Handheld devices, providing control interfaces that can be
out of band from the display;
– Voice commands with audio feedback;
– And haptics interfaces.

CS335 Principles of Multimedia Systems
Multiple Camera System
 Multiple camera system can capture video from
different locations simultaneously.

 Stereo system has been widely used for inferring the
“depth” of objects.

CS335 Principles of Multimedia Systems
3D reconstruction from Multiple Views
CS335 Principles of Multimedia Systems
Application of Multi-camera Imaging
 Image based rendering
 The Matrix

CS335 Principles of Multimedia Systems
Cameras in the Matrix

CS335 Principles of Multimedia Systems
Camera Calibration Toolboxes
 OpenCV

 Matlab Calibration Toolbox
– http://www.vision.caltech.edu/bouguetj/calib_doc/

 Multiple Camera Calibration
– http://cmp.felk.cvut.cz/~svoboda/SelfCal/

CS335 Principles of Multimedia Systems

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 16 posted: 7/16/2011 language: English pages: 36
How are you planning on using Docstoc?