debevec-facade

Document Sample
debevec-facade Powered By Docstoc
					Paul Debevec, 3D Photography




                    Modeling and Rendering Architecture
                             from Photographs
                                          Paul Debevec
                                   Computer Science Division
                                University of California at Berkeley


                                 SIGGRAPH 99 Course #28, 3D Photography
                                   Brian Curless and Steve Seitz, organizers
                                            August 9, 1999


                   http://www.cs.berkeley.edu/~debevec




SIGGRAPH 99 Course #28                              4-1
Paul Debevec, 3D Photography




                                                                                                  Stereo
                                                                                                  Image
                                                                                                  Pair




                   Stereo Image Capture                                              Depth
                            Rig                                                      Map

                     Immersion ‘94
                     Michael Naimark
                       John Woodfill
                       Paul Debevec                                                               Synthetic
                        Leo Villareal                                                             Views
                        Ramin Zabih
                     Interval Research
                        Corporation       Ramin Zabih and John Woodfill. Non-parametric local transforms
                                             for determining visual correspondence. ECCV, May 1994.




                               Structure from Motion
                    Tomasi and Kanade 1992




                      Image from sequence
                      Image from sequence            Recovered model
                                                     Recovered model              Scene viewed from
                                                                                  Scene viewed from
                                                                                    same position
                                                                                    same position




SIGGRAPH 99 Course #28                                     4-2
Paul Debevec, 3D Photography




                               Structure from Motion
                     Taylor and Kriegman 1995




                                                         Recovered model

                    Two of eight original images




                    Tomasi and Kanade 1992                Taylor and Kriegman 1995




                          Image from sequence                  One of eight images




                           Recovered model                      Recovered model




SIGGRAPH 99 Course #28                             4-3
Paul Debevec, 3D Photography




                         Façade’s Modeling Method:
                       The user represents the scene as a
                        collection of geometric primitives

                       The computer solves for the sizes and
                        positions of the blocks according to
                        user-supplied edge correspondences




                     Modeling and Rendering Architecure from Photographs
                               (Debevec, Taylor, and Malik 1996)




                          Block Model    User-
                                         User-Marked Edges   Recovered Model




SIGGRAPH 99 Course #28                       4-4
Paul Debevec, 3D Photography




                               Façade Blocks




                               Parameterized Block




SIGGRAPH 99 Course #28                 4-5
Paul Debevec, 3D Photography




                                                       Parameter
                                                       References




                                                    Model
                                                   Hierarchy


                  Relation can be:
                  * Arbitrary 6 DOF
                  * Fixed Rotation
                  * Fixed Translation
                  * Geometric Relationship




SIGGRAPH 99 Course #28                       4-6
Paul Debevec, 3D Photography




                                 Reconstruction Algorithm
                     An objective function O measures the
                      misalignment between the marked edges
                      and the corresponding projected edges of
                      the model
                     O is minimized with respect to the model
                      parameters and camera positions
                     An initial estimate is obtained by a separate
                      procedure




                                                               Completed
                                                             Reconstruction
                                                            and Reprojection


                     Projected         Marked Edge
                      Model             Model Edge

                                        Error Area




SIGGRAPH 99 Course #28                               4-7
Paul Debevec, 3D Photography




                       Algorithm with Initial Estimate Procedure

                     1. Solve for camera rotations, independently,
                       based on edge orientations
                     2. Hold camea rotations fixed; solve for other
                       parameters (often linear)
                     3. Perform full non-linear optimization,
                       starting from near the solution




                                        Video




SIGGRAPH 99 Course #28                     4-8
Paul Debevec, 3D Photography




                     Photogrammetric Modeling Summary
                     Modeling with blocks works because:
                    Convenient for architecture
                    Recovers Complete Models
                    Reduces number of model parameters, e.g.
                      Campanile model has:
                           2,896 parameters as independent edges
                              240 parameters as independent blocks
                               33 parameters as constrained blocks
                       • → Few marked features required
                       • → Easier to solve




                               Surfaces of Revolution




                         Photograph                       Synthetic View

                                        Recovered Model




SIGGRAPH 99 Course #28                        4-9
Paul Debevec, 3D Photography




                   Arches and
                   Surfaces of Revolution
                                                      Taj Mahal
                                                     modeled from
                                                    one photograph




                      Image-Based Modeling, Rendering,
                                and Lighting
                                                SIGGRAPH 99 Course #39
                                                SIGGRAPH 99 Course #39
                                       Tuesday, August 10, 1999
                                Room 152, Los Angeles Convention Center
                                           8:30am - 5:00pm

                           Paul Debevec                            Michael Cohen
                                  UC Berkeley                        Microsoft Research

                         Leonard McMillan                            Chris Bregler
                                     MIT                             Stanford University

                          Richard Szeliski                         François Sillion
                               Microsoft Research                  iMAGIS - GRAVIR/IMAG




SIGGRAPH 99 Course #28                                  4-10
Paul Debevec, 3D Photography




                          Rendering with Projective Texture Mapping




                                   View-
                                   View-Dependent Weighting
                                          Function




SIGGRAPH 99 Course #28                      4-11
Paul Debevec, 3D Photography




                                  Scene with Geometric Detail

                    Model-Based
                      Stereo


                                   Approximate Block Model




                               Model-Based Stereo
                     Given a key and an offset image,
                         • Project the offset image onto the model
                         • View the model through the key camera
                           → Warped offset image
                     Stereo becomes feasible between key and
                       warped offset images because:
                         • Disparities are small
                         • Foreshortening is greatly reduced




SIGGRAPH 99 Course #28                       4-12
Paul Debevec, 3D Photography




                       Key Image            Warped Offset Image   Offset Image




                                   Disparity Map




                    Synthetic Views
                   of Refined Model
                   Four images composited with
                     Model-Based Stereo and
                              VDTM




SIGGRAPH 99 Course #28                             4-13
Paul Debevec, 3D Photography




                         Application: Rouen Revisited
                               (Golan Levin and Paul Debevec)
                                        SIGGRAPH 96 Art Show
                                        SIGGRAPH 96 Art Show




                    Synthetic View:    Synthetic View:         Synthetic View:
                         1996               1896               Monet Painting
                                        (Uncalibrated Views)




                                          Video




SIGGRAPH 99 Course #28                        4-14
Paul Debevec, 3D Photography




                    Application: The Campanile Movie
                        Created by: George Borshukov, Yizhou Yu, Jason Luros, Vivian
                       Jiang, Chris Wright, Sami Khoury, Charles Benton, Tim Hawkins,
                                       Charles Ying, and Paul Debevec

                      Thanks to Jitendra Malik, Jeff Davis, Susan Marquez, Al Vera, Peter
                       Bosselman, Camillo Taylor, Eric Paulos, Michael Naimark, Dorrice
                      Pyle, Russell Bayba, Lindsay Krisel, Oliver Crow, and Peter Pletcher,
                      as well as Charlie and Thomas Benton, Linda Branagan, John Canny,
                      Magdalene Crowley, Brett Evans, Eva Marie Finney, Lisa Sardegna,
                                        Ellen Perry, and Camillo J. Taylor.

                          Additional thanks: the Berkeley Computer Vision Group, the
                         Berkeley Multimedia Research Center, the Berkeley Computer
                         Graphics Group, the ONR MURI Program, Interval Research
                                    Corporation, and Silicon Graphics, Inc.




                     Cris Benton: Kite Aerial Photography




                               http://www-archfp.ced.berkeley.edu/kap/




SIGGRAPH 99 Course #28                               4-15
Paul Debevec, 3D Photography




                     Cris Benton: Kite Aerial Photography




                               http://www-archfp.ced.berkeley.edu/kap/




SIGGRAPH 99 Course #28                          4-16
Paul Debevec, 3D Photography




                         Campanile
                               Model




SIGGRAPH 99 Course #28                 4-17
Paul Debevec, 3D Photography




                               Campus Model (Campanile + 40 buildings)




                     Terrain Modeling
                    - Delaunay triangulation of
                       building bases + other
                      recovered ground points
                     - Extension out to horizon




SIGGRAPH 99 Course #28                          4-18
Paul Debevec, 3D Photography




                               Video




                                       A view from too far away




SIGGRAPH 99 Course #28          4-19
Paul Debevec, 3D Photography




                   Comparison: Time-of-flight Laser Scanning
                                 Laser scan of Berkeley’s Campanile,
                                    courtesy of Cyra corporation




                               Application: The Matrix


                                                                www.mvfx.com




                        Courtesy of
                     George Borshukov
                      and John Gaeta,
                         MANEX
                       Entertainment




SIGGRAPH 99 Course #28                         4-20
Paul Debevec, 3D Photography




                                        Video




                     Commercial
                       Product:
                     Metacreations
                       Canoma




                               www.metacreations.com/canoma




SIGGRAPH 99 Course #28                     4-21
Paul Debevec, 3D Photography




                     Application: Inverse Global Illumination
                       Yizhou Yu, Paul Debevec, Jitendra Malik, Tim Hawkins
                       SIGGRAPH 99, Thursday, 11:50-12:15pm, West Hall A




                                     40 radiance maps of a room




                       Recovered Geometry and Viewpoints




SIGGRAPH 99 Course #28                          4-22
Paul Debevec, 3D Photography




                                  Real/Synthetic Comparison
                                 Same viewpoints, Same lighting, Same objects




                                 Real/Synthetic Comparison
                               New viewpoint, New lighting, New object




SIGGRAPH 99 Course #28                            4-23
Paul Debevec, 3D Photography




                                      Interior Illumination Model
                                           St. Peter’s Basilica




                     Interior
                      Model
                    (35 parameters)
                    (35 parameters)




                                                  Related Sketches
                                               The Making of “Fiat Lux”
                                  Wednesday 11 August, 5:25pm - 6:00pm, Room 151 / 152
                                 Image-Based Modeling, Rendering, and Lighting in “Fiat Lux”
                                   Friday 13 August, 11:40am - 12:15pm, Room 408AB




SIGGRAPH 99 Course #28                                  4-24
Paul Debevec, 3D Photography




                                      Thanks
                          George Borshukov, Christine Cheng,
                            H-P Duiker, Tal Garfinkel, Tim
                             Hawkins, Jenny Huang, Sami
                          Khoury, Jason Luros, Jitendra Malik,
                           Westley Sarokin, Camillo Taylor,
                               Chris Wright, Yizhou Yu




SIGGRAPH 99 Course #28                    4-25

				
DOCUMENT INFO