Docstoc

chris

Document Sample
chris Powered By Docstoc
					Next Generation 4D Distributed Modeling and Visualization




 Automated 3D Model
Construction for Urban
    Environments

                Christian Frueh
                John Flynn
                Avideh Zakhor
           University of California, Berkeley


                     June 13, 2002
Presentation Overview

  Introduction
  Ground based modeling
   - Mesh processing
  Airborne modeling
   - Aerial photos
   - Airborne laser scans
  3D Model Fusion
  Rendering
  Conclusion and Future Work
Introduction

 Goal: Generate 3D model of a city for virtual
 walk/drive/fly-thrus and simulations
       • Fast
       • Automated
       • Photorealistic

needed:
For Fly-Thru:               For Walk/Drive-Thru:
• 3D Model of terrain       • 3D model of street
  and buildings tops          scenery & building
  & sides                     façades
• coarse resolution         • highly detailed
Introduction

3D Model of terrain            3D model of building
and building tops              façades



Airborne Modeling              Ground Based Modeling
•Laser scans/images            • Laser scans & images
    from plane                 from acquisition vehicle




                      Fusion

             Complete 3D City Model
Airborne Modeling

 Goal:   Acquisition of terrain shape and
         top-view building geometry


 Available Data:
 • Aerial Photos
 • Airborne laser scans

 Texture:    from aerial photos
 Geometry:          2 approaches:
             I) stereo matching of photos
             II) airborne laser scans
Airborne Modeling
Approach I : Stereo Matching               (last year)
Stereo photo pairs from city/urban areas, ~ 60% overlap

                 Semi-Automatic

   Manual:                        Automated:
 • Segmentation             • Camera parameter
                              computation,
                            • Matching,
                            • Distortion reduction,
                            • Model generation
Stereo Matching




 Stereo pair from downtown Berkeley and the estimated
 disparity after removing perspective distortions
Stereo Matching Results




  Downtown Oakland
Airborne Modeling

 Approach II: Airborne Laser Scans

 Scanning city from plane




• Resolution 1 scan point/m2
• Berkeley: 40 Million scan points

                                     point cloud
Airborne Laser Scans
• Re-sampling point cloud
• Sorting into grid
• Filling holes




     Map-like height field
usable for:
• Monte Carlo Localization
• Mesh Generation
Textured Mesh Generation

1. Connecting grid vertices to mesh
2. Applying Q-slim simplification

3. Texture mapping:
   • Semi-automatic
   • Manual selection of few correspondence
     points: 10 mins/entire Berkeley
   • Automated camera pose estimation
   • Automated computation of texture for
     mesh
Airborne Model




     East Berkeley campus with campanile
Airborne Model

 Downtown Berkeley




http://www-video.eecs.berkeley.edu/~frueh/3d/airborne/
Ground Based Modeling

   Goal:      Acquisition of highly detailed
              3D building façade models


                                                              buildings




                                           2D laser   v

                                   truck
                                                                z
                                                          u
                                                                          y
                                                                   


Acquisition vehicle          Scanning setup                    x


Truck with rack:             • vertical 2D laser scanner
• 2 fast 2D laser scanners     for geometry capture
• digital camera             • horizontal scanner for pose
                               estimation
Scan Matching & Initial Path Computation

   Horizontal laser scans:
    • Continuously captured during vehicle motion
    • Overlap
      Relative position estimation by scan-to-scan matching


t = t0          Translation (u,v)     Adding relative
                Rotation              steps (ui, vi, i)
                                                 (ui, vi, i)
                                                  (ui-1, vi-1, i-1)

                                                    …
t = t1
                     (u, v)                (u2, v2, 2)
                                          (u1, v1, 1)

             Scan                         path (xi,yi,i)
             matching
                                 3 DOF pose (x, y, yaw)
6 DOF Pose Estimation From Images

 Scan matching cannot estimate vertical
  motion
  - Small bumps and rolls
  - Slopes in hill areas
 Full 6 DOF pose of the vehicle is
  important; affects:
  - Future processing of the 3D and intensity data
  - Texture mapping of the resulting 3D models
 Extend initial 3 DOF pose by deriving
  missing 3 DOF (z, pitch, roll) from images
6 DOF Pose Estimation From Images
  Central idea: photo-consistency




  • Each 3D scan point can be projected into
    images using initial 3 DOF pose
  • If pose estimate is correct, point should
    appear the same in all images
  • Use discrepancies in projected position
    of 3D points within multiple images to
    solve for the full pose
6 DOF Pose Estimation – Algorithm




 3DOF of laser as initial estimate
 Project scan points into both images
 If not consistent, use image correlation
  to find correct projection
 Ransac used for robustness
6 DOF Pose Estimation – Results



                with 3 DOF
                pose




                with 6 DOF
                pose
6 DOF Pose Estimation – Results


              Original
  Model      Number of   Completely     Less
 Number       Seams       Removed     noticable   No Change   Worse
    1           48           42           0           4         2
    2           15           8            0           7         0
    3           29           25           2           2         0
    4           24           18           1           5         0
    5            33          24           0          9          0
    6            18          15           0          3          0
    7            34          28           1          3          2
  Total         201         160           4          33         4
Percentage                   80           2          16         2
 Monte Carlo Localization (1)
Previously: Global 3 DOF pose correction using aerial
photography
a) path before MCL correction b) path after MCL correction




    After correction, points fit to edges of aerial image
Monte Carlo Localization (2)
             Extend MCL to work with airborne
  Now:
             laser data and 6 DOF pose

No perspective shifts of
building tops, no shadow lines
  •   Fewer particles necessary,
      increased computation speed
  •   Significantly higher accuracy
      near high buildings and tree
      areas

Use terrain shape to estimate
z coordinate of truck
  •   Correct additional DOF for
      vehicle pose (z, pitch, roll)
  •   Modeling not restricted to
      flat areas
Monte Carlo Localization (3)
Track global 3D position of vehicle to correct relative
6 DOF motion estimates




Resulting corrected path overlaid with airborne laser
height field
Path Segmentation
                          24 mins, 6769 meters

                         vertical scans: 107,082
                           scan points: ~ 15 million

                       Too large to process as
                       one block!


  • Segment path into quasi-
    linear pieces
  • Cut path at curves and
    empty areas
  • Remove redundant
    segments
Path Segmentation




Resulting path segments overlaid with
edges of airborne laser height map
Simple Mesh Generation
Simple Mesh Generation


                     Triangulate




   Point cloud                           Mesh
Problem:
• Partially captured
  foreground objects
• erroneous scan points
  due to glass reflection

    Side views look “noisy”

    Remove foreground: extract facades
Façade Extraction and Processing (1)
    1. Transform path segment into depth image




     2. Histogram analysis over vertical scans
                              split   main
depth value sn,υ for
                              depth   depth                        split        main
a scan point Pn,υ
                                                                   depth        depth
                                              25



                                              20

                                                       local minimum
                                              15



scanner                                       10



                                              5



                                              0
                                                   1      6   11   16      21     26    31    36
                  ground points                                                              depth

                                                                                                     depth
Façade Extraction and Processing (2)
3. Separate depth image into 2 layers:




                            foreground
                            =trees, cars etc.




                            background
                            =building facades
Façade Extraction and Processing (3)
4. Process background layer:
  • Detect and remove invalid
  scan points
 • Fill areas occluded by foreground
 objects by extending geometry
 from boundaries
     •Horizontal, vertical, planar
     interpolation, RANSAC
 • Apply segmentation
 • Remove isolated segments
 • Fill remaining holes in large
 segments
  • Final result: “clean” background
  layer
Façade Extraction – Examples (1)




 without processing    with processing
Façade Extraction – Examples (2)

                   without processing




                     with processing
Façade Extraction – Examples (3)



 without
 processing




 with
 processing
Facade Processing
Foreground Removal
Mesh Generation




Downtown Berkeley
Automatic Texture Mapping (1)
Camera calibrated and synchronized with laser scanners
    Transformation matrix between camera image and
    laser scan vertices can be computed




1. Project geometry into images
2. Mark occluding foreground objects in image
 3. For each background triangle:
   Search pictures in which triangle is not occluded,
   and texture with corresponding picture area
Automatic Texture Mapping (2)

Efficient representation: texture atlas
Copy texture of all triangles into “mosaic” image




   Typical texture reduction: factor 8..12
Automatic Texture Mapping (3)

Large foreground objects: Some of the filled-in
triangles are not visible in any image!
    “texture holes” in the atlas

Texture synthesis: preliminary

•   Mark holes
    corresponding to non-
    textured triangles in
    the atlas
• Search the image for areas matching the hole
  boundaries

• Fill the hole by copying missing pixels from
  these image
Automatic Texture Mapping (4)



  Texture
   holes
  marked




  Texture
   holes
   filled
Automatic Texture Mapping (5)
Ground Based Modeling - Results




   Façade models of downtown Berkeley
Ground Based Modeling - Results




   Façade models of downtown Berkeley
Model Fusion

        Fusion of ground based and airborne
Goal:
        model to one single model




    Façade model

                              Airborne model
 Model Fusion:
  1. Registration of models
  2. Combining the registered meshes
Registration of Models
Models are already registered with each
via Monte-Carlo-Localization !




     Which model to use where?
Preparing Ground Based Models
Intersect path segments with each other Remove
degenerated, redundant triangles in overlapping
areas



original
mesh



redundant
triangles
removed
Preparing Airborne Model

 Ground based model has 5-10 times
 higher resolution

     Remove facades in airborne model
      where ground based geometry is
      available
     Add ground based façades
     Fill remaining gaps with a “blend
      mesh” to hide model transitions
Preparing Airborne Model




   Initial airborne model
Preparing Airborne Model




    Remove facades where ground
    based geometry is available
Combining Models




   Add ground based façade models
Combining Models




   Fill remaining gaps with a “blend
   mesh” to hide model transitions
Model Fusion - Results
Rendering
Ground based models:
• Up to 270,000 triangles, 20
  MB texture per path
  segment
• 4.28 million triangles, 348
  MB texture for 4 downtown
  blocks
• Difficult to render interactively!

    Subdivide model and create
     multiple level-of-details (LOD)
    Generate scene graph,
     decide which LOD to render when
Multiple LODs for façade meshes

  Highest LOD

•Qslim mesh
simplification
•Texture
subsampling


  Lower LOD
• Geometry: 10%
• Texture: 25%
of original mesh
Façade Model Subdivision for Rendering

Subdivide 2 highest LODs of façade meshes along
cut planes




                    Sub-      Submesh   LOD 0
                    scene               LOD 1
         Path                           LOD 0
                              Submesh
         segment                        LOD 1
Global             LOD 2                LOD 0
                              Submesh
scene                                   LOD 1
Interactive Rendering




  Downtown blocks: Interactive rendering with
  web-based browser!
Future Work

 Resolution enhancement and post-
  processing of LIDAR data
 Devise new data acquisition system and
  algorithms to capture
  - both sides of street simultaneously
  - texture for upper parts of tall buildings
 Include foreground objects in model
 Add temporal component to dynamically
  update models
 Compact representation
 Interactive rendering

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:11
posted:8/23/2012
language:English
pages:57