2008.ISBI.Zhou.SCS

Document Sample
2008.ISBI.Zhou.SCS Powered By Docstoc
					Spatially Constrained Segmentation
             of Dermoscopy Images

  Howard Zhou1, Mei Chen2, Le Zou2, Richard Gass2,
   Laura Ferris3, Laura Drogowski3, James M. Rehg1




           1Schoolof Interactive Computing, Georgia Tech
                                2Intel Research Pittsburgh
      3Department of Dermatology, University of Pittsburgh




                                                             1
Skin cancer and melanoma
   Skin cancer : most common of all cancers




                                                                                                           2
                 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Skin cancer and melanoma
   Skin cancer : most common of all cancers
   Melanoma : leading cause of mortality (75%)




                                                                                                           3
                 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Skin cancer and melanoma
   Skin cancer : most common of all cancers
   Melanoma : leading cause of mortality (75%)
   Early detection significantly reduces mortality




                                                                                                            4
                  [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Dermoscopy
Clinical View view




                                                                                                      5
            [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Dermoscopy
   Improve diagnostic accuracy by 30% in the hands
    of trained physicians
   May require as much as 5 year experience to have
    the necessary training
   Motivation for Computer-aided diagnosis (CAD) in
    this area

    Clinical view              Dermoscopy view




                                                       6
First step of analysis:
Segmentation
   Separating lesions from surrounding skin
   Resulting border
       Gives lesion size and border irregularity
       Crucial to the extraction of dermoscopic features for
        diagnosis
   Previous Work :
       PDE approach – Erkol et al. 2005, …
       Histogram thresholding – Hintz-Madsen et al. 2001, …
       Clustering – Schmid 1999, Melli et al. 2006…
       Statistical region merging – Celebi et al. 2007, …


                                                                7
Domain specific constraints
   Spatial constraints
       Four corners are skin (Melli et al.2006, Celebi et al. 2007)
       Implicitly enforcing Local neighborhood constraints on image
        Cartesian coordinates (Meanshift)




                                                                       8
Domain specific constraints
   Spatial constraints
       Four corners are skin (Melli et al.2006, Celebi et al. 2007)
       Implicitly enforcing Local neighborhood constraints on image
        Cartesian coordinates (Meanshift)




                                                                       9
Meanshift (c = 32, s = 8)
We explore …
   Spatial constraints arise from the growth
    pattern of pigmented skin lesions




                                                10
Meanshift (c = 32, s = 8)
We explore …
   Spatial constraints arise from the growth
    pattern of pigmented skin lesions –
    radiating pattern




                                                11
Meanshift (c = 32, s = 8)
Embedding constraints
   Radiating pattern from lesion growth
   Embedding constraints as polar coords
    improves segmentation performance




                                                12
Meanshift (c = 32, s = 8)       Polar (k = 6)
  Embedding constraints
     Radiating pattern from lesion growth
     Embedding constraints as polar coords
      improves segmentation performance




                                                   13
Meanshift                  Polar   Polar (k = 6)
  Comparison to the Doctors
     Radiating pattern from lesion growth
     Embedding constraints as polar coords
      improves segmentation performance White: Dr. Ferris
                                              Red : Dr. Zhang
                                              Blue : computer




                                                       14
Meanshift                     Polar
Dermoscopy images
Common radiating appearance




                              15
Growth pattern of pigmented
skin lesions
   lesions grow in both radial and vertical direction
   Skin absorbs and scatters light.
   Appearance of pigmented cells varies with depth
       Dark brown  tan  blue-gray
   Common radiating appearance pattern on skin surface




                                                                                       16
                                       [ Image courtesy of “Dermoscopy : An Atlas of
                                       Surface Microscopy of Pigmented Skin Lesions]
Radiating growth pattern on
skin surface
   Difference in appearance: more significant
    along the radial direction than any other
    direction.




                                                 17
Radiating growth pattern on
skin surface
   Difference in appearance: more significant
    along the radial direction than any other
    direction.




                                                 18
Embedding spatial constraints
Feature vectors
   Each pixel  feature vector in R4
       3D: R,G,B or L, a, b in the color space
       1D: polar radius measured from the center of
        the image (normalized by w)

                     original


                 r

                {R, G, B}
                                                       19
Embedding spatial constraints
Grouping features
   Each pixel  feature vector in R4
   Clustering pixels in the feature space
   Replace pixels with mean for compact
    representation
                  original                   filtered


              r

             {R, G, B}
                                                 20
Radiating pattern
Dermoscopy vs. natural images




               …                    …
Derm dataset (216)   BSD dataset (300)
                                         21
Embedding spatial constraints
Grouping features        Cartesian
   Mean per-pixel residue:
    average per-pixel color
    difference of each pair
                              {Rc, Gc, Bc}

             original
                                   polar

          {Ro, Go, Bo}
                              {Rp, Gp, Bp}
                                        22
Dermoscopy vs. natural images
Polar vs. Cartesion
   Mean per-pixel residue (k-means++, k = 30)

Residue (Cartesian)             Residue (Cartesian)




              Residue (polar)                 Residue (polar)
                                                                23

Derm dataset (216)               BSD dataset (300)
Dermoscopy vs. natural images
Polar vs. Cartesion
   Mean per-pixel residue (k-means++, k = 30)




                                                 24
Polar vs. Cartesian
   The regions appear more blocky in the
    Cartesian case




      Polar (k = 30)           Cartesian (k = 30)   25
Six super-regions
   30 clusters  6 super clusters (K-means++)




      Polar (k = 6)            Cartesian (k = 6)   26
Final segmentation




    Polar            Cartesian   27
Polar vs. Meanshift
   The regions appear more blocky in the
    Meanshift case




      Polar (k = 6)        Meanshift (c = 32, s = 8)   28
Final segmentation




    Polar            Meanshift   29
Algorithm overview
   Given a dermoscopy image




                               30
Algorithm overview
   Given a dermoscopy image




    original




                               31
Algorithm overview
1. First round clustering: K-means++ (k = 30)




 original         30 clusters




                                                32
Algorithm overview
2. Second round: clusters(30) super-regions(6)




 original           30 clusters       6 Super-regions




                                                    33
Algorithm overview
3. Apply texture gradient filter (Martin, et al. 2004)




  original           30 clusters         6 Super-regions




                                                         34
                  Texture edge map
Algorithm overview
4. Find optimal boundary (color+texture)




 original         30 clusters      6 Super-regions




                                                   35
               Texture edge map   Final segmentation
1. First round clustering
   First round clustering: K-means++ (k = 30)
       Reduce noise
       Groups pixels into homogenous regions – a
        more compact representation of the image
       Artuhur and Vassilvitskii, 2007
   R4 : {L*a*b* (3D), w * polar radius (1D)}
         original



                                                    36
1. First round clustering
   First round clustering: K-means++ (k = 30)
       Reduce noise
       Groups pixels into homogenous regions – a
        more compact representation of the image
       Artuhur and Vassilvitskii, 2007
   R4 : {L*a*b* (3D), w * polar radius (1D)}
         original         30 clusters



                                                    37
2. Second round clustering
   K = 6 : clusters(30) super-regions(6)
       Account for intra-skin and intra-lesion variations
       Avoid a large k
   Super-regions correspond to meaningful
    regions such as skin, skin-lesion transition,
    and inner lesion, etc.
          original          30 clusters



                                                             38
2. Second round clustering
   K = 6 : clusters(30) super-regions(6)
       Account for intra-skin and intra-lesion variations
       Avoid a large k
   Super-regions correspond to meaningful
    regions such as skin, skin-lesion transition,
    and inner lesion, etc.
          original          30 clusters   6 super-regions



                                                             39
3. Color-texture integration
   Incorporating texture information can
    improve segmentation performance.
       Severely sun damaged skin; texture variations
        at boundaries in addition to color variations



         original



                                                        40
3. Color-texture integration
   Incorporating texture information can
    improve segmentation performance.
       Severely sun damaged skin; texture variations
        at boundaries in addition to color variations
   Apply texture gradient filter (Martin, et al. 2004)

         original



                                                          41
3. Color-texture integration
   Incorporating texture information can
    improve segmentation performance.
       Severely sun damaged skin; texture variations
        at boundaries in addition to color variations
   Apply texture gradient filter (Martin, et al. 2004)
   Texture edge map: pseudo-likelihood
         original   Texture edge map



                                                          42
4. Optimal boundary
   Optimal skin-lesion boundary
       Color: Earth Mover’s Distance (EMD) between every
        pair of super-regions




6 super-regions



                                                            43
4. Optimal boundary
   Optimal skin-lesion boundary
       Color: Earth Mover’s Distance (EMD) between every pair
        of super-regions
       Texture: Texture edge map




6 super-regions       Texture edge map



                                                                 44
4. Optimal boundary
   Optimal skin-lesion boundary
       Color: Earth Mover’s Distance (EMD) between every pair
        of super-regions
       Texture: Texture edge map
       Minimizing the integrated color-texture measure



6 super-regions       Texture edge map



                                                                 45
Validation and results
   Our collaborating dermatologist Dr. Ferris manually
    outline the lesions in 67 dermoscopy images
   The border error is given by




   Computer : binary image obtained by filling the automatic
    detected border
   ground-truth : obtained by filling in the boundaries
    outlined by Dr. Ferris

                                                                46
Typical segmentation result


            Error = 12.96%



                             White: Dr. Ferris
                             Red : Dr. Zhang
                             Blue : computer




                                             47
                   Comparison
                         Compared to ground-truth outlined by Dr. Ferris
                   35

                   30                                                                  Dr. Zhang
Percentage error




                                                                                       RGB
                   25                                                                  CIELAB
                          20.64             21.41
                                  19.49             20.13
                   20                                                                  Color + texture
                                                              16.92
                                                                      15.91 14.93
                   15
                                                                                    11.32
                   10

                    5

                    0
                            none            Cartesian           polar                       Dr. Zhang

                                               Spatial constraints
                   To account for inter-operator variation, we also asked Dr. Alex Zhang to
                   manually outline boundaries on the same dataset                                       48
Additional results   White: Dr. Ferris
                     Red : Dr. Zhang
                     Blue : computer




                        Error = 5.80%
                                         49
Additional results   White: Dr. Ferris
                     Red : Dr. Zhang
                     Blue : computer




                        Error = 13.61%
                                         50
Additional results   White: Dr. Ferris
                     Red : Dr. Zhang
                     Blue : computer




                        Error = 16.60%
                                         51
Additional results   White: Dr. Ferris
                     Red : Dr. Zhang
                     Blue : computer




                        Error = 34.09%
                                         52
Limitation
   Assumption that lesions appear relatively
    near the center may not hold
   Fairly low number of super regions (6) may
    limit the algorithm to perform well on lesions
    with more colors




                                                     53
Conclusion
   Growth pattern of pigmented skin lesions can be used to
    improve lesion segmentation accuracy in dermoscopy
    images.
   An unsupervised segmentation algorithm incorporating
    these spatial constraints
   We demonstrate its efficacy by comparing the
    segmentation results to ground-truth segmentations
    determined by an expert.




                                                          54
Future work
   Extend to meanshift?




                           55
Comparison to other methods

                          Compared to ground-truth outlined by Dr. Ferris
                   30
                          26.74

                   25
Percentage error




                                       20.43         20.77           20.13
                   20
                                                                                   14.93
                   15
                                                                                              11.32
                   10

                    5

                    0
                        Meanshift   JSEG (Celebi   SRM (Celebi   SCS Cartesian   SCS polar   Dr. Zhang
                                       2006)         2007)

                                                   Segmentation methods

                                                                                                         56
Color and texture cue integration
   Apply texture gradient filter (Martin, et al. 2004)
   Pseudo-likelihood map - edge caused by texture
    variation is present at a certain location




                                                          57

				
DOCUMENT INFO