Photo VR Editor A Panoramic and Spherical Environment Map

							 Photo VR Editor: A Panoramic and
Spherical Environment Map Authoring
 Tool for Image-Based VR Browsers


         Jyh-Kuen Horng, Ming Ouhyoung

           Communications and Multimedia Lab.
  Dept. of Computer Science and Information Engineering
    National Taiwan University, Taipei, Taiwan, R.O.C.
  Outline
 Introduction
 Related works
 System overview - two sub-systems
  • Manual editor
  • Automatic stitching method

 Conclusions & Future work
 Introduction
 Image-based rendering becomes more and more
  important
 Compare with geometry-based rendering
  • constant rendering time regardless of scene
    complexity
  • low computational power needed
  • photo-realistic

 How to construct a virtual environment?
Related Works
 Image warping
   – QuickTime VR by Apple Corp.
 Video clips
  – VideoBrush
  – rich frame information

 Hardware sensitive
  – IPIX
  – fisheye-lens camera
System Overview
              Editing Environment

            Load Source Photographs

                  Stitch Images

   Automatic Stitching       Manual Stitching

     Apply RM & LS          3D Transformation

  Form Cylindrical Image   Form Spherical Image
Manual Editing (1/3)
 Based on real 3D graphics model
  • Each photograph is taken as a texture of a 3D
    image plane
  • All kinds of affine transformation are allowed, such
    as translation, rotation, scaling
  • Pixel color is determined by multiple hit plane
     – ray casting
     – bilinear interpolation
Manual Editing (2/3)
 Intensity tuning
   • the aperture cannot be controlled




   - Before intensity tuning   - After intensity tuning
Manual Editing (3/3)
 Form panoramic image
   • Gap closing : f’ = (360 - g) * f / 360
      • f’ : adjusted focal length, f : original focal length
      • g : gap angle
      • easily propagate the error

   • Smoothing intensity discontinuity

                 x2
          x1          c2
                                                   2        2
            c1                              x1 * C1  x 2 * C 2
                           blending color =       2     2
                                                 x1  x 2
Automatic Stitching (1/5)


                          p

               x




    Camera   • 3D point p = (X, Y, Z)
             • image coordinates x = (x, y, 1)
Automatic Stitching (2/5)
                   p


                       xl
 Image k      xk
                                   Image l




                            • 3D point p = (X, Y, Z)
           Camera
                            • image coordinates xk, xkl
Automatic Stitching (3/5)
 The relationship between p and x
  • can be described using rotational model
                        1   1
    x ~ TVRp  p ~ R V x               ……(1)


 The mapping between image k and l is
     xk ~ Mxl
    M ~ Vk Rk Rl1Vl 1  Vk RklVl 1   ……(2)
Automatic Stitching (4/5)
 We wish to minimize the squared error metric

                                                                 
                                       2

    E (d )    I 1 ( xi'' )  I 0 ( xi )   giT J iT d  ei
                   ~                                              2
               
             i 
                                          
                                             i
                                                                      ……(3)


                       g : gradient, J : Jacobian matrix


 After d is solved, the matrix M is adapted by
    M '  ( I  D ) M                                                ……(4)

                       D is the deformation matrix defined by d
Automatic Stitching (5/5)
 To eliminate the ghosting
  • a local search pass is introduced
  • do 3-D search based on x-, y-, z-rotation
  • perform an incremental update to R
   R  ( I  X ()) * R                              ……(5)
   where
             0       Wz   Wy 
    X ()   Wz      0      Wx 
                                
              Wy
                    Wx       0     : angular velocity
Conclusions and Future Work (1/2)
  Performance
    Source image number   Total time elapsed   Time for stitching   Image size
                                                  two images
            16                29.06 sec           1.816 sec         320 x 240
            16                33.5 sec            2.094 sec         320 x 240



  Results
Conclusions and Future Work (2/2)
  Some topics are under investigation
   • An extension of the algorithm to construct
     spherical environment map automatically
   • A faster and more robust method
   • Other kind of image source (e.g. video clips)