Docstoc Random Impulse Noise Removal From Color Image Sequences.TIP

Document Sample Random Impulse Noise Removal From Color Image Sequences.TIP Powered By Docstoc
					IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011                                                                                     959

                   Fuzzy Random Impulse Noise Removal
                        From Color Image Sequences
                                           Tom Mélange, Mike Nachtegael, and Etienne E. Kerre

   Abstract—In this paper, a new fuzzy filter for the removal of                     which allows a more gradual transition between belonging to
random impulse noise in color video is presented. By working with                   and not belonging to. Such gradual transition makes fuzzy sets
different successive filtering steps, a very good tradeoff between                   very useful for the processing of human knowledge in terms of
detail preservation and noise removal is obtained. One strong fil-
tering step that should remove all noise at once would inevitably                   linguistic variables (e.g., large, small, etc.). There is for example
also remove a considerable amount of detail. Therefore, the noise                   no need to use a threshold to decide whether a difference in color
is filtered step by step. In each step, noisy pixels are detected by                 component value between two pixels is large or not. Two differ-
the help of fuzzy rules, which are very useful for the processing of                ences that differ only one unit (which is not noticeable by the
human knowledge where linguistic variables are used. Pixels that
are detected as noisy are filtered, the others remain unchanged. Fil-
                                                                                    human eye) could then be respectively large and not large. It is
tering of detected pixels is done by blockmatching based on a noise                 better to allow a difference to be large to some intermediate de-
adaptive mean absolute difference. The experiments show that the                    gree. For a larger difference, this degree will be higher than that
proposed method outperforms other state-of-the-art filters both vi-                  of a smaller difference. For an illustration of the effectiveness
sually and in terms of objective quality measures such as the mean                  of fuzzy set theory in image processing, we refer to, e.g., [25].
absolute error (MAE), the peak-signal-to-noise ratio (PSNR) and
the normalized color difference (NCD).                                                 Most filters in literature, that are developed for video, are
                                                                                    intended for sequences corrupted by additive Gaussian noise
   Index Terms—Circuits and systems, computers and information
processing, computational and artificial intelligence, filtering, fil-
                                                                                    (e.g., [3]–[7]). Only few video filters for the impulse noise case
ters, fuzzy logic, image denoising, logic, nonlinear filters.                        can be found (e.g., [19]–[23], [30], [31], [45], [46]). However,
                                                                                    several impulse noise filters for still images exist. The best
                                                                                    known among them are the median based rank-order filters
                            I. INTRODUCTION                                         (e.g., [8]–[10], [32], [34]–[40], [48]. But also some fuzzy tech-
                                                                                    niques can be found [11]–[18], [33], [41]. Such 2-D filters could
     MAGES and videos belong to the most important infor-
I    mation carriers in today’s world (e.g., traffic observations,
surveillance systems, autonomous navigation, etc.). However,
                                                                                    be used to filter each of the frames of a video successively.
                                                                                    However, temporal inconsistencies will arise due to the neglec-
                                                                                    tion of the temporal correlation between successive frames.
the images are likely to be corrupted by noise due to bad acquisi-
                                                                                    A better alternative would be to use 3-D filtering windows,
tion, transmission or recording. Such degradation negatively in-
                                                                                    in which also pixels from neighboring frames are taken into
fluences the performance of many image processing techniques
                                                                                    account [19]–[23], [30], [31], [45], [46]. The main problem in
and a preprocessing module to filter the images is often required.
                                                                                    using neighboring frames is motion between them. Using pixels
   Among those filters, more and more fuzzy techniques start
                                                                                    at corresponding spatial positions in neighboring frames for
to appear in literature [7], [11]–[18], [33], [41], [43], [44]–[47],
                                                                                    noise removal may introduce ghosting artifacts in the presence
[49], [52], [53]. Fuzzy set theory was introduced by Zadeh in
                                                                                    of camera and object motion. In the method proposed in this
1965 [24] and is a generalization of classical set theory. A clas-
                                                                                    paper, we will therefore only in non-moving areas assign a
sical crisp set over a universe can be modelled by a
                                                                                    temporal impulse between two corresponding spatial positions
mapping (characteristic function): an element             belongs to
                                                                                    to noise (detection phase) and for the replacement of a noisy
the set or does not belong to it. Fuzzy sets are now modelled as
                                                                                    pixel (filtering phase) motion compensation will be applied to
            mappings (membership functions). So the character-
                                                                                    find the most reliable pixel in the previous frame.
istic function is extended to a membership function where also
                                                                                       Analogously, a distinction between filters intended for
membership degrees between zero and one are allowed. An el-
                                                                                    greyscale images and for color images needs to be made. Filters
ement            can now also belong to some degree to the set,
                                                                                    for greyscale images could be used for color images by applying
                                                                                    them on each of the color bands of the image separately. In this
   Manuscript received March 25, 2010; revised July 12, 2010; accepted August       paper, we consider the images to be modeled in the RGB color
31, 2010. Date of publication September 20, 2010; date of current version March
18, 2011. This work was supported by the GOA project B/04138/01 of Ghent            space and we thus have three color bands: red, green and blue.
University. The associate editor coordinating the review of this manuscript and     However, such approach will generally introduce many color
approving it for publication was Dr. Kenneth K. M. Lam.                             artefacts, especially in textured areas, due to the neglection of
   The authors are with the Fuzziness and Uncertainty Modeling Research Unit,
Department of Applied Mathematics and Computer Science, Ghent Univer-               the correlation between the different color bands. To incorpo-
sity, 9000 Ghent, Belgium (e-mail:; mike.nachtegael            rate this correlation, vector-based methods were introduced.;;                      Most of these methods are based on ordering the vectors in a
   Color versions of one or more of the figures in this paper are available online
at                                                      predefined filtering window. The output for a given color pixel
   Digital Object Identifier 10.1109/TIP.2010.2077305                                is then the pixel in the window around the given pixel, that has
                                                                 1057-7149/$26.00 © 2011 IEEE
MÉLANGE et al.: FUZZY RANDOM IMPULSE NOISE REMOVAL FROM COLOR IMAGE SEQUENCES                                                                                 969

                                                                                  is based on fuzzy rules in which information from spatial and
                                                                                  temporal neighbors as well as from the other color bands is used.
                                                                                  Detected noisy components are filtered based on blockmatching
                                                                                  where a noise adaptive mean absolute difference is used and
                                                                                  where the search region contains pixels blocks from both the
                                                                                  previous and current frame.
                                                                                     The experiments showed that the proposed method outper-
                                                                                  forms other state-of-the-art methods both in terms of objective
                                                                                  measures such as MAE, PSNR and NCD and visually.

                                                                                     [1] E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new effi-
                                                                                         cient approach for the removal of impulse noise from highly corrupted
                                                                                         images,” IEEE Trans. Image Process., vol. 5, no. 6, pp. 1012–1025,
                                                                                     [2] R. H. Chan, C. Hu, and M. Nikolova, “An iterative procedure for re-
                                                                                         moving random-valued impulse noise,” IEEE Signal Process. Lett., vol.
                                                                                         11, no. 12, pp. 921–924, 2004.
                                                                                     [3] S. M. M. Rahman, M. O. Ahmad, and M. N. S. Swamy, “Video de-
                                                                                         noising based on inter-frame statistical modelling of wavelet coeffi-
                                                                                         cients,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 2, pp.
                                                                                         187–198, 2007.
                                                                                     [4] L. Jovanov, A. Pizurica, V. Zlokolica, S. Schulte, P. Schelkens, A.
                                                                                         Munteanu, E. E. Kerre, and W. Philips, “Combined wavelet-domain
                                                                                         and motion-compensated video denoising based on video codec mo-
                                                                                         tion estimation methods,” IEEE Trans. Circuits Syst. Video Technol.,
                                                                                         vol. 19, no. 3, pp. 417–421, 2009.
                                                                                     [5] H. B. Yin, X. Z. Fang, Z. Wei, and X. K. Yang, “An improved mo-
                                                                                         tion-compensated 3-D LLMMSE filter with spatio-temporal adaptive
                                                                                         filtering support,” IEEE Trans. Circuits Syst. Video Technol., vol. 17,
                                                                                         no. 12, pp. 1714–1727, 2007.
Fig. 9. 20th frame of the “Deadline” sequence (top-left to bottom-right): orig-      [6] L. Guo, O. C. Au, M. Ma, and Z. Liang, “Temporal video denoising
inal, noisy (p = 25%), INRC, AVMF, VAVDMF and Proposed.                                  based on multihypothesis motion compensation,” IEEE Trans. Circuits
                                                                                         Syst. Video Technol, vol. 17, no. 10, pp. 1423–1429, 2007.
                                                                                     [7] T. Mélange, M. Nachtegael, E. E. Kerre, V. Zlokolica, S. Schulte, V. De
                             TABLE III                                                   Witte, A. Pizurica, and W. Philips, “Video denoising by fuzzy motion
           AVERAGE RUNNING TIME (SECONDS PER FRAME) FOR                                  and detail adaptive averaging,” J. Electron. Imaging, vol. 17, no. 4, pp.
             THE PROCESSING OF THE “SALESMAN” SEQUENCE                                   043005–, 2008.
                                                                                     [8] T. Chen, K. K. Ma, and L. H. Chen, “Tri-state median filter for image
                                                                                         denoising,” IEEE Trans. Image Process., vol. 8, pp. 1834–1838, 1999.
                                                                                     [9] R. C. Hardie and C. G. Boncelet, “LUM filters: A class of
                                                                                         rank-order-based filters for smoothing and sharpening,” IEEE Trans.
                                                                                         Signal Process., vol. 41, pp. 1061–1076, 1993.
                                                                                    [10] S. J. Ko and Y. H. Lee, “Center weighted median filters and their ap-
                                                                                         plications to image enhancement,” IEEE Trans. Circuits Syst., vol. 38,
                                                                                         pp. 984–993, 1991.
                                                                                    [11] S. M. Guo, C. S. Lee, and C. Y. Hsu, “An intelligent image agent based
                                                                                         on soft-computing techniques for color image processing,” Expert Sys-
                                                                                         tems With Appl., vol. 28, pp. 483–494, 2005.
                                                                                    [12] S. Schulte, V. De Witte, M. Nachtegael, D. Van der Weken, and E. E.
                                                                                         Kerre, “Fuzzy random impulse noise reduction method,” Fuzzy Sets
                                                                                         Syst., vol. 158, pp. 270–283, 2007.
stage could be sped up by using fast motion estimation tech-                        [13] S. Schulte, M. Nachtegael, V. De Witte, D. Van der Weken, and E. E.
                                                                                         Kerre, “A fuzzy impulse noise detection and reduction method,” IEEE
niques such as those presented in [27]–[29]. Next, for higher                            Trans. Image Process., vol. 15, no. 5, pp. 1153–1162, 2006.
noise levels, it might be useful to do the block matching for                       [14] F. Russo, “Fire operators for image processing,” Fuzzy Sets Syst., vol.
blocks of pixels and to filter each of the noisy pixels in the blocks                     103, no. 2, pp. 265–275, 1999.
                                                                                    [15] F. Russo, “Hybrid neuro-fuzzy filter for impulse noise removal,” Pat-
at the same time instead of applying the block matching for each                         tern Recognit., vol. 32, pp. 1843–1855, 1999.
noisy pixel separately. Further, in each of the successive steps                    [16] H. K. Kwan, “Fuzzy filters for noise reduction in images,” in Fuzzy
                                                                                         Filters for Image Processing, M. Nachtegael, D. Van der Weken, D.
in the algorithm, the detection and filtering of a pixel does not                         Van De Ville, and E. E. Kerre, Eds. Heidelberg, Germany: Springer,
depend on the detection and filtering of the other pixels in the                          2003, pp. 54–71.
frame, such that the algorithm could be further sped up by per-                     [17] J. H. Wang, W. J. Liu, and L. D. Lin, “An histogram-based fuzzy filter
                                                                                         for image restoration,” IEEE Trans. Syst. Man Cybern. B, Bern., vol.
forming this detection and filtering for several pixels in parallel.                      32, no. 2, pp. 230–238, 2002.
                                                                                    [18] H. Xu, G. Zhu, H. Peng, and D. Wang, “Adaptive fuzzy switching filter
                                                                                         for images corrupted by impulse noise,” Pattern Recognit. Lett., vol. 25,
                            IV. CONCLUSION                                               pp. 1657–1663, 2004.
                                                                                    [19] F. Cocchia, S. Carrato, and G. Ramponi, “Design and real-time imple-
   In this paper, we have presented a new filtering framework                             mentation of a 3-D rational filter for edge preserving smoothing,” IEEE
for color videos corrupted with random valued impulse noise.                             Trans. Consum. Electron., vol. 43, no. 4, pp. 1291–1300, 1997.
                                                                                    [20] J.-S. Kim and H. W. Park, “Adaptive 3-D median filtering for
In order to preserve the details as much as possible, the noise is                       restoration of an image sequence corrupted by impulse noise,” Signal
removed step by step. The detection of noisy color components                            Process.: Image Commun., vol. 16, pp. 657–668, 2001.
970                                                                                         IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011

      [21] P. S. Windyga, “Fast impulsive noise removal,” IEEE Trans. Image                [48] Z. Xu, H. R. Wu, B. Qiu, and X. Yu, “Geometric features based filtering
           Process., vol. 10, no. 1, pp. 173–179, 2001.                                         for suppression of impulse noise in color images,” IEEE Trans. Image
      [22] R. Lukac and S. Marchevsky, “LUM smoother with smooth control for                    Process., vol. 18, no. 8, pp. 1742–1759, 2009.
           noisy image sequences,” EURASIP J. Appl. Signal Process., vol. 2001,            [49] C.-C. Kang and W.-J. Wang, “Fuzzy reasoning-based directional me-
           no. 2, pp. 110–120, 2001.                                                            dian filter design,” Signal Process., vol. 89, no. 3, pp. 344–351, 2009.
      [23] M. El Hassouni, H. Cherifi, and D. Aboutajdine, “HOS-based image                 [50] V. I. Ponomaryov, “Real-time 2D-3D filtering using order statistics
           sequence noise removal,” IEEE Trans. Image Process., vol. 15, no. 3,                 based algorithms,” J. Real-Time Image Process., vol. 1, no. 3, pp.
           pp. 572–581, 2006.                                                                   173–194, 2007.
      [24] L. A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp.                 [51] J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Some im-
           338–353, 1965.                                                                       provements for image filtering using peer group techniques,” Image
      [25] , E. E. Kerre and M. Nachtegael, Eds., “Fuzzy techniques in image                    Vis. Comput., vol. 28, no. 1, pp. 188–201, 2010.
           processing,” in Studies in Fuzziness and Soft Computing. Heidelberg,            [52] S. Morillas, V. Gregori, and G. Peris-Fajarns, “Isolating impulsive
           Germany: Physica-Verlag, 2000, vol. 52.                                              noise pixels in color images by peer group techniques,” Comput. Vis.
      [26] S. Weber, “A general concept of fuzzy connectives, negations and im-                 Image Underst., vol. 110, no. 1, pp. 102–116, 2008.
           plications based on t-norms and t-conorms,” Fuzzy Sets Syst., vol. 11,          [53] S. Morillas, V. Gregori, G. Peris-Fajarns, and A. Sapena, “Local
           no. 2, pp. 115–134, 1983.                                                            self-adaptive fuzzy filter for impulsive noise removal in color images,”
      [27] L. M. Po and W. C. Ma, “A novel four-step search algorithm for fast                  Signal Process., vol. 88, no. 2, pp. 390–398, 2008.
           block motion estimation,” IEEE Trans. Circuits Syst. Video Technol.,            [54] Q. Ma, L. Zhang, and B. Wang, “New strategy for image and video
           vol. 6, no. 3, pp. 313–317, 1996.                                                    quality assessment,” J. Electron. Imaging, vol. 19, no. 011019, 2010.
      [28] S. Zhu and K. K. Ma, “A new diamond search algorithm for fast block-            [55] H. R. Sheikh and A. C. Bovik, “A visual information fidelity approach
           matching motion estimation,” IEEE Trans. Image Process., vol. 9, no.                 to video quality assessment,” in Proc. 1st Int. Workshop on Video
           2, pp. 287–290, 2000.                                                                Process. and Quality Metrics for Consumer Electron., 2005, pp.
      [29] C. Zhu, X. Lin, and L. P. Chau, “Hexagon-based search pattern for fast               23–25.
           block motion estimation,” IEEE Trans. Circuits Syst. Video Technol.,
           vol. 12, no. 5, pp. 349–355, 2002.
      [30] V. Ponomaryov, A. Rosales-Silva, and V. Golikov, “Adaptive and
           vector directional processing applied to video colour images,” Elec-
           tron. Lett., vol. 42, no. 11, pp. 623–624, 2006.
      [31] R. Lukac, “Vector order-statistics for impulse detection in noisy color
           image sequences,” in Proc. 4th EURASIP-IEEE Region 8 Int. Symp.                                         Tom Mélange was born in Kortrijk, Belgium, in
           Video/Image Process. Multimedia Commun., Zadar, Croatia, 2002.                                          1984. He received the M.Sc. degree in mathematics
      [32] R. Lukac, “Adaptive vector median filtering,” Pattern Recognit. Lett.,                                   from Ghent University, Ghent, Belgium, in 2006.
           vol. 24, pp. 1889–1899, 2003.                                                                           In October 2006, he joined the Department of
      [33] S. Schulte, S. Morillas, V. Gregori, and E. E. Kerre, “A new fuzzy                                      Applied Mathematics and Computer Science, Ghent
           color correlated impulse noise reduction method,” IEEE Trans. Image                                     University, where he is a member of the Fuzziness
           Process., vol. 16, no. 10, pp. 2565–2575, 2007.                                                         and Uncertainty Modeling Research Unit. In 2010
      [34] R. Lukac, K. N. Plataniotis, A. N. Venetsanopoulos, and B. Smolka,                                      he received the Ph.D. degree with a thesis on
           “A statistically-switched adaptive vector median filter,” J. Intell. Robot.                              fuzzy techniques for noise reduction in video and
           Syst., vol. 42, no. 4, pp. 361–391, 2005.                                                               interval-valued fuzzy mathematical morphology,
      [35] J. Camacho, S. Morillas, and P. Latorre, “Efficient impulse noise sup-                                   under Prof. Dr. E. E. Kerre.
           pression based on statistical confidence limits,” J. Imag. Sci. Technol.,
           vol. 50, no. 5, pp. 427–436, 2006.
      [36] S. Hore, B. Qiu, and H. R. Wu, “Improved vector filtering for color
           images using fuzzy noise detection,” Opt. Eng., vol. 42, no. 6, pp.
           1656–1664, 2003.                                                                                        Mike Nachtegael was born on February 16, 1976. He
      [37] R. Lukac and K. N. Plataniotis, “A taxonomy of color image filtering                                     received the M.Sc. degree in mathematics from Ghent
           and enhancement solutions,” Adv. Imag. Electron. Phys., vol. 140, pp.                                   University, Ghent, Belgium, in 1998. In the same year
           187–264, 2006.                                                                                          he joined the Department of Applied Mathematics
      [38] R. Lukac, “Adaptive color image filtering based on center-weighted                                       and Computer Science, where he is a member of the
           vector directional filters,” Multidimen. Syst. Signal Process., vol. 15,                                 Fuzziness and Uncertainty Modeling Research Unit.
           no. 2, pp. 169–196, 2004.                                                                               In May 2002 he received the Ph.D. in mathematics,
      [39] Z. H. Ma, H. R. Wu, and B. Qiu, “A robust structure-adaptive hybrid                                     on the topic of fuzzy techniques in image processing.
           vector filter for color image restoration,” IEEE Trans. Image Process.,                                     In 2002, he became an Assistant Professor in his
           vol. 14, no. 12, pp. 1990–2001, 2005.                                                                   Department and since 2008 he has held the position
      [40] V. Chatzis and I. Pitas, “Fuzzy scalar and vector median filters based                                   of Guest Professor.
           on fuzzy distances,” IEEE Trans. Image Process., vol. 8, no. 5, pp.
           731–734, 1999.
      [41] S. Morillas, V. Gregori, G. Peris-Fajarns, and P. Latorre, “A fast im-
           pulsive noise color image filter using fuzzy metrics,” Real-Time Imag.,                                   Etienne E. Kerre was born in Zele, Belgium, on May
           vol. 11, no. 5–6, pp. 417–428, 2005.
                                                                                                                    8, 1945. He received the M.Sc. degree in mathematics
      [42] S. J. Sangwine and R. E. N. Horne, The Colour Image Processing
                                                                                                                    and the Ph.D. in mathematics from Ghent University,
           Handbook. London, U.K.: Chapman & Hall, 1998.
                                                                                                                    Ghent, Belgium, in 1967 and 1970, respectively.
      [43] J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Fast detection
           and removal of impulsive noise using peer groups and fuzzy metrics,”                                        Since 1984, he has been a Lector, and since 1991,
           J. Vis. Commun. Image Represent., vol. 19, no. 1, pp. 20–29, 2008.                                       a full Professor at Ghent University. He is a referee
      [44] S. Morillas, V. Gregori, and A. Hervs, “Fuzzy peer groups for reducing                                   for more than 30 international scientific journals, and
           mixed Gaussian-impulse noise from color images,” IEEE Trans. Image                                       a member of the editorial board of international jour-
           Process., vol. 18, no. 7, pp. 1452–1466, 2009.                                                           nals and conferences on fuzzy set theory. He has been
      [45] V. Ponomaryov, A. Rosales, and F. Gallegos, “3D filtering of colour                                       an honorary chairman at various international confer-
           video sequences using fuzzy logic and vector order statistics,” in Proc.                                 ences. In 1976, he founded the Fuzziness and Uncer-
           Advanced Concepts for Intelligent Vision Systems, LNCS 5807, 2001,           tainty Modeling Research Unit (FUM) and since then his research has been
           pp. 210–221.                                                                 focused on the modeling of fuzziness and uncertainty, and has resulted in a
      [46] V. F. Kravchenko, V. I. Ponomaryov, and V. I. Pustovoit, “Three-di-          great number of contributions in fuzzy set theory and its various generalizations.
           mensional filtration of multichannel video sequences on the basis of          Especially the theories of fuzzy relational calculus and of fuzzy mathematical
           fuzzy-set theory,” Doklady Phys., vol. 55, no. 2, pp. 58–63, 2010.           structures owe a great deal to him. Over the years he has also been a promotor
      [47] V. Ponomaryov, F. Gallegos-Funes, and A. Rosales-Silva, “Fuzzy               of 29 Ph.D.s on fuzzy set theory. His current research interests include fuzzy
           directional (FD) filter to remove impulse noise from colour images,”          and intuitionistic fuzzy relations, fuzzy topology, and fuzzy image processing.
           IEICE Trans. Fundament. Electron., Commun. Comput. Sci., vol.                He has authored or co-authored 25 books, and more than 450 papers in interna-
           E93-A, no. 2, pp. 570–572, 2010.                                             tional refereed journals and proceedings.