Learning Center
Plans & pricing Sign in
Sign Out

Multi-Label Image Segmentation for Medical Applications Based on Graph- Cuts


IJCSIS invites authors to submit their original and unpublished work that communicates current research on information assurance and security regarding both the theoretical and methodological aspects, as well as various applications in solving real world information security problems.

More Info
									                                                        (IJCSIS) International Journal of Computer Science and Information Security,
                                                        Vol. 8, No. 5, August 2010

           Multi-Label Image Segmentation for Medical

                      R.Rajeswari,                                                                P. Anandhakumar,
        Research Scholar, Department of Information Technology                   Asst. Professor, Department of Information Technology
                   MIT campus, Anna University                                               MIT Campus, Anna University
                        Chennai-44,India                                                       Chennai-44, India

Abstract : Mathematical morphology is very attractive for                   segmentation by weighted aggregation algorithm, and apply
automatic image segmentation because it efficiently deals with              the technique to the task of detecting and segmenting brain
geometrical descriptions such as size, area, shape, or connectivity         tumor and edema in multichannel magnetic resonance (MR)
that can be considered as segmentation-oriented features. This              volumes.[3].
paper presents a new approach for some theoretical and practical
improvements of image segmentation based on graph cuts. The                 The flowchart of proposed system shown in Fig.1.The tumors
method is based on the use of the region adjacency graph                    segmentation is obtained through the computation of a
produced by the watershed transform from mathematical                       minimal graph cut of the region adjacency graph obtained on
morphology. Marker extraction identifies the presence of                    the watershed segmentation of the image. The user has to
homogeneous regions. The combination of morphological and
graph cuts segmentation permits us to speed up and define new
                                                                            interactively specify the location of the tumors as well as a
classes of energy functions that can be minimized using graph               marker to specify the background. A post-processing step is
cuts. The use of region graphs gives promising results and can              also proposed to smooth the segmentation result. This post-
potentially become a leading method for interactive medical                 processing step consists in a morphological opening of the
image segmentation.                                                         segmentation.

Keywords- Image segmentation, marker extraction, morphology,                                                   Original Image
                                                                                User specified
Graphs, Watershed Transform.                                                      Markers
                                                                                                            Watershed Transform
                 I.       INTRODUCTION
MAGNETIC resonance imaging (MRI) provides detailed
images of living tissues, and is used for both brain and body                                              Region Adjacency Graph
human studies. Data obtained from MR images is used for
detecting tissue deformities such as cancers and injuries. The                                            Tumor segmentation by
key to any automatic method is that it must be robust, so that it                                     Computation of Minimal Graph Cuts
produces reliable results on every image acquired from any
MR scanner using different relaxation times, slice thicknesses
and fields of view. More recently, computer-assisted methods                                                         Post processing
have been used for specific tasks such as extraction of MS
lesions from MRI brain scans [1], [2].
 A key problem in medical imaging is automatically
segmenting an image into its constituent heterogeneous
processes. Automatic segmentation has the potential to                                    Fig 1.Flow Chart of proposed method
positively impact clinical medicine by freeing physicians from
the burden of manual labeling and by providing robust,
quantitative measurements to aid in diagnosis and disease                                   II.        RELATED WORK
modeling. One such problem in clinical medicine is the
automatic segmentation and quantification of brain tumors.                    There are lot of work available related to the proposed
The model-aware affinities integrated into the multilevel                   work, in which some of them are significant. It includes

                                                                                                        ISSN 1947-5500
                                                     (IJCSIS) International Journal of Computer Science and Information Security,
                                                     Vol. 8, No. 5, August 2010

automatic and semiautomatic for segmentation of brain into               overall classification accuracy compared to pixel-by-pixel
different tissues, including MS lesions. These approaches                approaches.
include a variety of methods such as statistical, fuzzy, neural
                                                                         A very powerful segmentation method that has been widely
networks, and fuzzy neural networks.
                                                                         used in image segmentation problems is the watershed
Kamber et al. [4] developed a brain tissue model for                     transform (WT) [13]–[19]. Since its original application on
segmentation of MR images of patients with MS disease. The               grayscale images [13], [14], a very attractive computational
offered model was three-dimensional, voxel based, which                  form has been derived [15] and extended to color images [16].
provided a priori probabilities for white matter, gray matter            An extensive review of the watershed algorithms can be found
and CSF.                                                                 in [19].The watershed transform presents some advantages
                                                                         over other developed segmentation methods.
The survey by Clarke et al. of segmentation methods for MR
images [5] describes many useful image processing techniques             1) The watershed lines form closed and connected regions,
and discusses the important question of validation. The                  where edge based techniques usually define disconnected
various image processing techniques used for segmenting the              boundaries that need postprocessing to produce closed regions.
brain can be divided into several groups: those required to
                                                                         2) The watershed lines always correspond to obvious contours
perform a crude threshold-based extraction of the brain,
                                                                         of objects which appear in the image.
followed by refinement of brain contours; statistical methods
for brain segmentation, and region growing methods.                      The main problem of over-segmentation, can be usually
                                                                         overcome by the use of preprocessing or postprocessing,
The work by M. Kass, A. Witkins & D. Terzopoulos [6] in
                                                                         producing a segmentation that better reflects the arrangement
1988 is the basis for the active contour model, the Snake
                                                                         of objects within the image.
algorithm. The work by Mathews Jacob, Thierry Blu and
Michael unser [7] presents different solutions for improving
spline based snakes, and demonstrates the minimum curvature                       III.WATERSHED BASED IMAGE TRANSFORM
interpolation property used as an argument to get rid of the             The proposed system starts with the original image and
explicit smoothness constraint.                                          watershed transform of the image's gradient modulus is used
The work of El Naqa et al [8] proposes a variational methods             for segmentation purpose.
based on multivalued level set deformable models for                       The validation is done comparing our segmentation with the
simultaneous 2D or 3D segmentation of multimodality images               physician’s delineation. Fig 1 shows the flowchart of the
or multiple image sets from the same modality.                           proposed method.
A multidimensional segmentation and filtering methodology                A. Watershed Based on Dissimilarity Measures
for accurate blood flow velocity field reconstruction from               The watershed transform definitions can be slightly modified
phase-contrast magnetic resonance imaging (PC MRI)                       in such a way that it produces a segmentation according to
developed and validated by Kartik S. Sundareswaran, David                different path based criteria. For image segmentation purposes
H. Frakes[9].                                                      the watershed transform is computed on the image's gradient
Yuri Boykov, Olga Veksler and Ramin Zabih [10] proposes an               modulus, usually the morphological gradient. The estimation
expansion algorithm that finds a labeling within a known                 of the gradient has thus also an importance for the quality of
factor of the global minimum, while swap algorithm handles               the segmentation. A first extension of the classical watershed
more general energy functions. Both algorithms allow                     transform of the gradient image is based on the local
important cases of discontinuity preserving energies for image           dissimilarity between neighbor pixels. This approach, called
restoration, stereo and motion.                                          watershed by dissimilarity was originally proposed by Pard
                                                                         as[20] and Lotufo in [21].
Saracoglu et al. [11] modeled the problem using a method
consisting of three steps: image tessellation, clustering, and             Given a pixel adjacency graph G = (V;E;W), we consider
classification. The image was tessellated into regions with              the following edges weights mapping
similar properties using a region growing approach                         e i,j Є E , wi,j = (1/d(i,j ).|pi—pj) |+1)
(tessellation step).Based on the “average” color information of
the regions, clustering is performed.                                    where i and j are two nodes of the graph, pi and pj are the grey
                                                                         level values of neighbor pixels of the image and d(i, j) is the
 A novel method for segmentation and classification of M-                distance between the two pixels.
FISH chromosome images is presented by Petros S. Karvelis,
Alexandros T. Tzallas et al.[12]. The segmentation is based on           These edges weights represent the local estimation of the
the multichannel watershed transform in order to define                  image's gradient modulus. Instead of computing the
regions of similar spatial and spectral characteristics. Then, a         morphological gradient of the image at each pixel, the gradient
Bayes classifier task-specific on region classification, is              is here estimated "between" the pixels. This weight map
applied. The combination of the multichannel segmentation                especially provides a better detection of thin contrasted
and the region-based classification is found to improve the              objects.

                                                                                                    ISSN 1947-5500
                                                              (IJCSIS) International Journal of Computer Science and Information Security,
                                                              Vol. 8, No. 5, August 2010

This weight map has the main advantage to achieve a better                     B. Region Adjacency Graph
detection of small details compared to the classical                           A region adjacency graph can be obtained from low-level
morphological gradient.                                                        segmentation of the image by numerous methods. It can be
                                                                               obtained from a first unsupervised clustering of the image, for
     IV.         GRAPHS IN IMAGE SEGMENTATION                                  instance one can apply the watershed transform [22, 23], flat
This section presents the graphs commonly encountered in                       zones labeling [24, 25], or k-means clustering [26] to obtain
imaging applications. A first level is the so-called "pixel                    such low-level segmentations.
adjacency graph". In a second level, an unsupervised low level                 The minimal cut problem on region adjacency graph is used
segmentation of the image (i.e. a segmentation that provides                   instead of pixel adjacency graph since it is advantageously
much more regions than objects in the image) is used to build                  used to speed up and extend the presented techniques.
a "region adjacency graph”.
                                                                                    V.           IMPLEMENTATION AND RESULTS
A. Pixel Adjacency Graphs
                                                                               The region adjacency graph is obtained from the watershed
       In this case, the set of nodes of the graph is the set of
                                                                               transform computed from all minima of the morphological
pixels of the image, and the edges link neighboring pixels. For
each image and an adjacency system is needed to build the                      gradient of the original MRI image using Meyer's algorithm.
corresponding pixel adjacency graph. Different common                          [27]. Then a region adjacency graph is extracted and used for
adjacency systems are illustrated in figure.                                   the next optimization steps. The user specified markers are
                                                                               used to compute a minimal graph cut separating the markers
                                                                               specifying the myocardium and the external tissues. The
                                                                               markers that specify the myocardium denoted as the set of
                                                                               regions Mm. The markers specifying the tissues surrounding
                                                                               the myocardium are denoted by M ext.
                                                                               F(ri , rj) is defined as the set of edges of the pixel graph connecting
                                                                               two regions ri and rj of the low-level watershed segmentation.

            Figure 1. 4 neighborhood adjacency system (V4).                    F(ri , rj)={e m,n Є E | m Є ri , n Є rj }                                   (1)

                                                                               The strictly positive and decreasing function g used as an edge
                                                                               indicator for the a minimal surface is

                                                                               g(||I(p)||)= (1/(1+||I(p)||)k                                               (2)

                                                                               where the k parameter was set to 2.
                                                                               The edges weights of the region adjacency graph GR= (VR, E R,
                                                                               W R ) are then set such that the weight of a graph cut equals
            Figure 2. 8 neighborhood adjacency system (V8).                    the energy function of a surface defined as

                                                                                   r i , , rj = Σ (g( || I(p)||))                                          (3)

                                                                               The myocardium boundaries are finally extracted by
                                                                               computing a minimal graph cut of the region adjacency graph
                                                                               with weights given by (3). The minimal cut is obtained on the
                                                                               region adjacency graph with two additional nodes s and t,
                                                                               respectively connected to the markers of the myocardium and
                                                                               the markers of the external tissues. The edges weights of the
                                                                               graph are summarized in Table I.

     Figure 3.     6 neighborhood adjacency system for 3D

                                                                                                                    ISSN 1947-5500
                                                                            (IJCSIS) International Journal of Computer Science and Information Security,
                                                                            Vol. 8, No. 5, August 2010

                     TABLE I.            EDGE WEIGHTS OF GRAPH

      Edge                           Weight                             for

       s , ri                          +∞                        ri Є M m
         ri,t                          +∞                        rj Є M ext

       r i , , rj   Σ                          (g( || I(p)||))   ri Є V R , rj Є N r                           Figure 4c.       Minimal surface computed by proposed
                    (e m,n   Є   F(ri , rj))                                                                                             method

                                                                                             A slice of a 3D Heart MRI superposed with a hand made
                                                                                             segmentation is shown in figure 4a. The segmentation
A. Dataset                                                                                   highlights the aorta (in red), the superior vena cave (in dark
 The imaging modality in this application is an isotropic 3D                                 blue), the pulmonary artery, the right and left chamber (light
 MRI. This imaging technology provides 50 to 120 slices                                      blue) and the heart muscle (green and light red).
 covering the whole heart. The whole myocardium is not
 always visible on 3D MRI datasets, especially along the right                               C. Segmentation
 chamber, because the myocardium is too thin in this area.                                   For each image, a reference segmentation done by an
 Thus the left part of the myocardium which is outlined in                                   experienced radiologist is available.
 green in Fig is segmented in our proposed method. The other
 parts of the myocardium have been obtained with manual

                                                                                                  Figure 5a.        Liver tumors segmentation. (a) User specfied markers.

                       Figure 4a.               A slice of a 3D Heart MRI

                                                                                                       Figure 5b.        Results of proposed segmentation strategy

                                                                                             In order to evaluate the accuracy of the developed method,
                                                                                             four scores are computed from five different evaluation
                                                                                               1) Volumetric overlap: This score is computed as the
                         Figure 4b.              MRI heart image superposed with             number of voxels in the intersection of segmentation and
                                                      markers                                reference, divided by the number of voxels in the union of
                                                                                             segmentation and reference. This value is equal to 1 for a
                                                                                             perfect segmentation and is equal to 0 as the lowest possible

                                                                                                                            ISSN 1947-5500
                                                                        (IJCSIS) International Journal of Computer Science and Information Security,
                                                                        Vol. 8, No. 5, August 2010

value, when there is no overlap at all between the                                       creation for surgery planning and simulation. The challenging
segmentation and the reference.                                                          problems presented by medical applications forced us to
   2) Relative absolute volume difference: The total volume                              imagine innovative methods that can tackle the problems
of the segmentation is divided by the total volume of the                                linked with clinical applications. The method presented have
reference. From this number 1 is subtracted, the absolute value                          all the characteristics to be used in real life applications:
is taken and the result is multiplied by 100.                                            robustness, speed and precision.
   3) Average symmetric absolute surface distance: These
are defined as those voxels in the object that have at least one                                   VII           FUTURE ENHANCEMENT
neighbor that does not belong to the object. For each voxel in                           The minimal cuts can be extended to provide constrained
these sets, the closest voxel in the other set is determined. All                        segmentation models. These new constrained problems are
these distances are stored, for boundary voxels from both                                solved in a linear programming framework. The major
reference and segmentation. Taking the average of all these                              limitation of linear programming is the current inefficiency of
distances symmetric absolute surface distance is determined.                             generic solvers. An efficient linear program solver that can
   4) Symmetric RMS surface distance: The squared                                        handle graphs representing large images can be developed.
distances between the two sets of border voxels is taken and
the root is extracted and gives the symmetric RMS surface                                                                REFERENCES
distance.                                                                                [1]    Johnston, M. S. Atkins, B. Mackiewich, and M. Anderson,
                                                                                                “Segmentation of multiple sclerosis lesions in intensity corrected
D. Comparison of Results:                                                                       multispectral MRI,” IEEE Trans. Med. Imag., vol. 15. pp. 154–169, Apr.
  1) Training: The evaluation scores of our method                                       [2]    A. P. Zijdenbos, B. M. Dawant, R. A. Margolin, and A. C.
calculated on a set of MRI images presenting different tumors                                   Palmer,“Morphometric analysis of white matter lesions in MR images:
with known handmade segmentations. The evaluation scores                                        Method and validation,” IEEE Trans. Med. Imag., vol. 13, p. 4, pp. 716–
                                                                                                724,Dec. 1994.
compare our results with the radiologists segmentations. The
                                                                                         [3]    Jason J. Corso, Eitan Sharon, Shishir Dube, Suzie El-Saden, Usha Sinha,
important point is that these results have been obtained with                                   and Alan Yuille “Efficient Multilevel Brain Tumor Segmentation With
the knowledge of the handmade segmentations. The results                                        Integrated Bayesian Model Classification,”IEEE Trans. Med. Imag.,,
have been obtained such that the similarity between the two                                     vol. 27, No. 5, MAY , pp.629-640, 2000.
                                                                                         [4]    M. Kamber, R. Shinghal, D.L. Collins, G.S. Francis, A.C. Evans,
segmentations is visually satisfactory.                                                         “Model-based 3D segmentation of multiple sclerosis lesions in magnetic
                                                                                                resonance brain images,”IEEE Trans. Med. Imag. 14 (1995) 442–453.
                                                                                         [5]    L. P. Clarke, R. P. Velthuizen, M. A. Camacho, J. J. Heine,
                                                                                                M.Vaidyanathan, L. O. Hall, R. W. Thatcher, and M. L. Silbiger, “MRI
                                                                                                segmentation: Methods and applications,” Magn. Reson. Imag., vol.
                                                                                                13,no. 3, pp. 343–368, 1995.
                                                                                         [6]    M. Kass, A. Witkins and D. Rerzopoulos, “Snakes: Active Contour
           35                                                                                   Models”, International Journal of Computer Vision, vol.1, p. 321-331,
           30                                                         Training           [7]    Mathews Jacob, Thierry Blu and Michael Uhnser, “A unifying approach
           25                                                                                   and interface on Spline-Based Snakes,”,Proceedings of SPIE, vol.4332,
                                                                      Testing                   2001, pp 340-347.
           20                                                                            [8]    Issam El Naqa, Deshan Yang, Aditya Apte, Divya Khullar, Sasa Mutic,
                                                                                                Jie Zheng,Jeffrey D. Bradley, Perry Grigsby, and Joseph O. Deasy,

                                                                                                “Concurrent multimodality image segmentation by active contours for
           10                                                                                   radiotherapy treatment planning ,”Medical Physics, Vol. 34, No. 12,
                                                                                                December 2007.
            5                                                                            [9]    Kartik S. Sundareswaran, David H. Frakes, Mark A. Fogel,Dennis D.
                                                                                                Soerensen, John N. Oshinski, and Ajit P. Yoganathan, “Optimum Fuzzy
            0                                                                                   Filters for Phase-Contrast Magnetic Resonance Imaging Segmentation”,
                Overlap Error    Volume      Average    RMS Surface
                                                                                                Journal of Magnetic Resonance Imaging 29:155–165 (2009).
                                                                                         [10]   K. Saracoglu, J. Brown, L. Kearney, S. Uhrig, J. Azofeifa, C. Fauth,
                                Difference   Surface     Distance
                                                                                                M.Speicher, and R. Eils, “New concepts to improve resolution and
                                             Distance                                           sensitivity of molecular cytogenetic diagnostics by multicolor
                                                                                                fluorescence in situ hybridization,” Cytometry, vol. 44, no. 1, pp. 7–15,
                        Metrics                                                                 May 2001.
                                                                                         [11]   Petros S. Karvelis, , Alexandros T. Tzallas, Dimitrios I. Fotiadis, and
          Figure 1.         Comparison between Training and Testing Results                     Ioannis Georgiou, “A Multichannel Watershed-Based Segmentation
                                                                                                Method for Multispectral Chromosome Classification,” IEEE Trans.
                                                                                                Med. Imag., vol. 27, no. 5, pp.697–708, May 2008.
                      VI.             CONCLUSION
                                                                                         [12]   S. Beucher and C. Lantuejoul, “Use of watersheds in contour
The developed tools, based on low-level segmentation and                                        detection,”in Proc. Int. Workshop Image Process., Real-Time Edge
graph based methods, have been successfully used for various                                    Motion Detection/Estimation, Rennes, France, 1979, pp. 17–21.
medical applications including tumors delineation and model                              [13]    S. Beucher, “The watershed transformation applied to image
                                                                                                segmentation,”Scan. Micros. Intern., vol. 6, pp. 299–314, 1992.

                                                                                                                          ISSN 1947-5500
                                                                (IJCSIS) International Journal of Computer Science and Information Security,
                                                                Vol. 8, No. 5, August 2010

[14] L. Vincent and P. Soille, “Watershed in digital spaces: An efficient            [22] ___, “The morphological approach to segmentation: the watershed
     algorithm based on immersion simulations,” IEEE Trans. Pattern                       transformation, Mathematical Morphology in Image Processing” (E.R.
     Anal.Mach. Intell., vol. 13, no. 6, pp. 583–598, Jun. 1991.                          Dougherty, ed.), 1993, pp. 433-481.
[15] F. Meyer, “Color image segmentation,” in Proc. Int. Conf. Image                 [23] P. Salembier and J. Serra, Flat zones filtering, connected operators and
     Process. Appl., 1992, pp. 303–306.                                                   filters by reconstruction, IEEE Transactions on Image Processing 3
[16] V. Grau, A. U. Mewes, M. Alcaniz, R. Kikinis, and S. K.                              (1995), no. 8, 1153-1160.
     Warfield,“Improved watershed transform for medical image                        [24] J. Crespo, R. Schafer, J. Serra, C. Gratin, and F. Meyer, The flat zone
     segmentation using prior information,” IEEE Trans. Med. Imag., vol. 23,              approach: a general low-level region merging segmentation method,
     no. 4, pp.447–458, Apr. 2004.                                                        Signal Processing 62 (1997), 37-60.
[17] K. Haris, S. Efstratiadis, N. Maglaveras, C. Pappas, J. Gourassas, and          [25] J. McQueen, Some methods for classification and analysis of
     G. Louridas, “Model-based morphological segmentation and labeling of                 multivariate observations, In Proceedings of the Fifth Berkeley
     coronary angiograms,” IEEE Trans. Med. Imag., vol. 18, no. 10,                       Symposium on Mathematical Statistics and Probability 1 (1967),281-
     pp.1003–1015, Oct. 1999.                                                             297.
[18] J. Roerdink and A. Meijster, “The watershed transform:                          [26] F. Meyer and S. Beucher, Morphological segmentation, Journal of
     Definitions,algorithms and parallelization strategies,” Fundamenta                   Visual Communication and Image Representation 1 (1990), no. 1, 21-46.
     Informaticae,vol. 41, pp. 187–228, 2001.
[19] M. Pardas, Segmentacion morfologica de secuencias de imagenes:
     Aplicacion a la codificacion, PhD thesis, Universitat Politcnica de
     Catalunya,). Barcelona, Spain. (1995).
[20] Lotufo and A. Falcao, The ordered queue and the optimality of the
     watershed approaches, Proceedings of the International Symposium on
     Mathematical Morphology 2000, Palo Alto, CA, Jun 2000. (2000), 341-
[21] N. Biggs, E. Lloyd, and R. Wilson, Graph theory, 1736-1936, Oxford
     University Press, 1986.

                            AUTHORS PROFILE
Dr.P.AnandhaKumar is Currently working as AssistantProfessor in the Departm-
ent of Information Technology at Madras Institute of Technology, Anna University,
Chennai,India. He has published several papers in international, national journals
and conferences. His areas of interests includes content-based image indexing
 techniques and frameworks, image processing and analysis, video analysis ,fuzzy
logic, pattern recognition, knowledge management and semantic analysis

R.Rajeswari is currently doing Ph.D in the Department of Information Technology,
MIT Campus,AnnaUniversity Chennai,India

                                                                                                                    ISSN 1947-5500

To top