Shape Retrieval from Silhouette Images

Document Sample
Shape Retrieval from Silhouette Images Powered By Docstoc
					3D Shape Retrieval from
   Silhouette Images




           Nat Duca
Overview (1)

Input:         Output:




 Edges:
Overview (2)

   Are silhouettes good descriptions of shape?
    –   One may not be… but a few are
    –   Can you see 3D structure in the shadow of a hand in the
        projector?


   Silhouettes are easily entered queries… draw the
    contour of your shape
   Also, they can be derived from an arbitrary sketch or
    image… or shadows, come to think of it
Approach Overview

   Sea of cameras: produce multiple views of an object

   A silhouette contour of a closed object is a circular
    function: can be made rotation-invariant

   Multiple silhouettes can be used to improve the
    accuracy of a query (i.e. get me the object that has
    these two silhouettes)
Sea of cameras

   The “sea” of cameras can be sampled over a
    hemisphere because silhouettes will be the same
    (mod flip)

   Regular distribution on a sphere over samples…
   Ideal case: random distribution of K camera points
    over a hemisphere that changes for each model

   My version: custom placed locations of 20 points on
    a hemisphere that is constant across all shapes
Contour extraction (1)

   Most practical shapes will have:
    –   Sampling artifacts in the silhouette image
    –   Interior contours (i.e. no one dominant contour)




        silhouette
                                        edges
Contour extraction (2)

   Perform morphological closing of silhouette
    image with a disk structuring element
   Use a laplacian-of-gaussian edge detector to
    extract contour pixels
Contour extraction (3)

   Using a maze walking algorithm, find a closed loop
    through the contour pixels to produce a single closed
    group of pixels
   This is done for all connected components
   I bet we could use all of the components to do partial
    matching on this shape
   … but for now, we chose the largest contour that
    results
Silhouette Parameterization

   Goals: rotation, scale, translation invariance
   RDF (radial distance function)
    –   f(c) = r where r is maximal object extent in
        direction c from centroid
    –   As in spherical harmonics
   Arc length parameterization:
    –   f(c) = r + k(s) i parameterization about centroid
    –   Where k(s) is the stretch required to fit the curve
        to the surface
Matching Metric

   Let dist(s1, s2) be the sum of squared distances
    between the descriptor for two contours (“samples”)

   Option 1: best sample from query and model
    –   Let dist(mi,q1…K) for some model mi and K samples q1…j be
        defined as
            The sample sd in model mi that is closest to any qi

   Option 2: all samples in query, best in model
            Sum over all k of qi
               –   The sample sd in model mi that is closest to qi
System Details (1)

   I start with Princeton Shape Benchmark geometry
    and classifications

   Silhouette sea is generated by a glut app in Linux
    using depth buffer readback
   Images are copied to a machine with Matlab where
    morphological closing and edge detection is done

   … and then copied back… grr…

   Useful stat: 622 models yields 1.5 GB of images, but
    only a 7.5mb database
System Details (2)

   Edge images are processed by a maze search algorithm to find
    the largest closed contour

   Contours are sampled evenly to 200 points using Gnu Scientific
    Library

   The parameterization is FFT’d with FFTw

   The resulting descriptors are stored in a binary database

   We can set an epsilon value that allows the same sample to be
    appear in multiple models if it is similar enough (compression!)

   Precision vs recall is generated internally by the database
Control tests: rotation invariance

   Match is somewhat sensitive to noise… e.g. higher frequency
    noise introduced by the sampling of the contours

                               Contour invariance with rotation: 400-element descriptor

                         3.5

                          3

                         2.5
     Measured distance




                          2

                         1.5

                          1

                         0.5

                          0
                           0




                                                                          0

                                                                                 0

                                                                                        0

                                                                                               0

                                                                                                      0

                                                                                                             0

                                                                                                                    0

                                                                                                                           0

                                                                                                                                  0
                               10

                                    20

                                         30

                                              40

                                                   50

                                                        60

                                                             70

                                                                  80

                                                                        90
                                                                       10

                                                                              11

                                                                                     12

                                                                                            13

                                                                                                   14

                                                                                                          15

                                                                                                                 16

                                                                                                                        17

                                                                                                                               18




                                                                       Rotation
Control tests: rotation invariance

   Tinkering with filtering and other stuff doesn’t
    help much… using the RDF does.
                                                    Contour Invariance with Rotation

                                    1.6
     Difference from Base Contour




                                    1.4
                                    1.2                                                Linear Interp
                                     1                                                 Cubic Interp
                                    0.8                                                Cubic + gaussian(1s)
                                    0.6                                                Cubic + gaussian(2s)
                                                                                       RDF + Linear
                                    0.4
                                    0.2
                                     0
                                      5




                                                            0
                                                            5
                                                            0
                                                            5
                                                            0
                                          20
                                               35
                                               50
                                                    65
                                                    80
                                                          95
                                                         11
                                                         12
                                                         14
                                                         15
                                                         17




                                                    Rotation Amount
Control tests: Scale invariance

                                                 Contour Invariance with Scale

                              1.6
 Distance from base contour




                              1.4
                              1.2
                                                                                 Linear Interp
                               1                                                 Cubic Interp
                              0.8                                                Cubic + gaussian(1s)
                              0.6                                                Cubic + gaussian(2s)

                              0.4                                                RDF + Linear

                              0.2
                               0
                                                          1
                                          4



                                                  7




                                                                  3



                                                                          6
                                 25



                                         55



                                                 85



                                                         15



                                                                 45



                                                                         75
                                       0.



                                               0.




                                                               1.



                                                                       1.
                              0.



                                      0.



                                              0.



                                                      1.



                                                              1.



                                                                      1.




                                                  Scale Factor
Circular parameterization and
Model match option 1

                                20 classes shown
                                  57 total classes
                                      622 models
Circular parameterization and
Model match option 1

                                20 classes shown
                                  57 total classes
                                      622 models
Circular parameterization and
Model match option 1

                                20 classes shown
                                  57 total classes
                                      622 models
Radial distance parameterization and
Model match option 1

            1 silhouette used in query   15 classes shown
                                           57 total classes
                                               622 models
Radial distance parameterization and
Model match option 1

            2 silhouettes used in query   15 classes shown
                                            57 total classes
                                                622 models
Radial distance parameterization and
Model match option 1

            8 silhouettes used in query   15 classes shown
                                            57 total classes
                                                622 models
Radial distance parameterization and
Model match option 2

                          1 silhouette used in query
                                  15 classes shown
                                     57 total classes
                                         622 models
Radial distance parameterization and
Model match option 2

                          4 silhouettes used in query
                                   15 classes shown
                                      57 total classes
                                          622 models
Radial distance parameterization and
Model match option 2

                          4 silhouettes used in query
                                   15 classes shown
                                      57 total classes
                                          622 models
Radial distance parameterization and
Model match option 2

                          8 silhouettes used in query
                                   15 classes shown
                                      57 total classes
                                          622 models
Radial distance parameterization and
Model match option 2

                          8 silhouettes used in query,
                                    57 classes shown
                                      57 total classes
                                          622 models

				
DOCUMENT INFO