IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011 959 Fuzzy Random Impulse Noise Removal From Color Image Sequences Tom Mélange, Mike Nachtegael, and Etienne E. Kerre Abstract—In this paper, a new fuzzy ﬁlter for the removal of which allows a more gradual transition between belonging to random impulse noise in color video is presented. By working with and not belonging to. Such gradual transition makes fuzzy sets different successive ﬁltering steps, a very good tradeoff between very useful for the processing of human knowledge in terms of detail preservation and noise removal is obtained. One strong ﬁl- tering step that should remove all noise at once would inevitably linguistic variables (e.g., large, small, etc.). There is for example also remove a considerable amount of detail. Therefore, the noise no need to use a threshold to decide whether a difference in color is ﬁltered step by step. In each step, noisy pixels are detected by component value between two pixels is large or not. Two differ- the help of fuzzy rules, which are very useful for the processing of ences that differ only one unit (which is not noticeable by the human knowledge where linguistic variables are used. Pixels that are detected as noisy are ﬁltered, the others remain unchanged. Fil- human eye) could then be respectively large and not large. It is tering of detected pixels is done by blockmatching based on a noise better to allow a difference to be large to some intermediate de- adaptive mean absolute difference. The experiments show that the gree. For a larger difference, this degree will be higher than that proposed method outperforms other state-of-the-art ﬁlters both vi- of a smaller difference. For an illustration of the effectiveness sually and in terms of objective quality measures such as the mean of fuzzy set theory in image processing, we refer to, e.g., . absolute error (MAE), the peak-signal-to-noise ratio (PSNR) and the normalized color difference (NCD). Most ﬁlters in literature, that are developed for video, are intended for sequences corrupted by additive Gaussian noise Index Terms—Circuits and systems, computers and information processing, computational and artiﬁcial intelligence, ﬁltering, ﬁl- (e.g., –). Only few video ﬁlters for the impulse noise case ters, fuzzy logic, image denoising, logic, nonlinear ﬁlters. can be found (e.g., –, , , , ). However, several impulse noise ﬁlters for still images exist. The best known among them are the median based rank-order ﬁlters I. INTRODUCTION (e.g., –, , –, . But also some fuzzy tech- niques can be found –, , . Such 2-D ﬁlters could MAGES and videos belong to the most important infor- I mation carriers in today’s world (e.g., trafﬁc observations, surveillance systems, autonomous navigation, etc.). However, be used to ﬁlter each of the frames of a video successively. However, temporal inconsistencies will arise due to the neglec- tion of the temporal correlation between successive frames. the images are likely to be corrupted by noise due to bad acquisi- A better alternative would be to use 3-D ﬁltering windows, tion, transmission or recording. Such degradation negatively in- in which also pixels from neighboring frames are taken into ﬂuences the performance of many image processing techniques account –, , , , . The main problem in and a preprocessing module to ﬁlter the images is often required. using neighboring frames is motion between them. Using pixels Among those ﬁlters, more and more fuzzy techniques start at corresponding spatial positions in neighboring frames for to appear in literature , –, , , , –, noise removal may introduce ghosting artifacts in the presence , , . Fuzzy set theory was introduced by Zadeh in of camera and object motion. In the method proposed in this 1965  and is a generalization of classical set theory. A clas- paper, we will therefore only in non-moving areas assign a sical crisp set over a universe can be modelled by a temporal impulse between two corresponding spatial positions mapping (characteristic function): an element belongs to to noise (detection phase) and for the replacement of a noisy the set or does not belong to it. Fuzzy sets are now modelled as pixel (ﬁltering phase) motion compensation will be applied to mappings (membership functions). So the character- ﬁnd the most reliable pixel in the previous frame. istic function is extended to a membership function where also Analogously, a distinction between ﬁlters intended for membership degrees between zero and one are allowed. An el- greyscale images and for color images needs to be made. Filters ement can now also belong to some degree to the set, for greyscale images could be used for color images by applying them on each of the color bands of the image separately. In this Manuscript received March 25, 2010; revised July 12, 2010; accepted August paper, we consider the images to be modeled in the RGB color 31, 2010. Date of publication September 20, 2010; date of current version March 18, 2011. This work was supported by the GOA project B/04138/01 of Ghent space and we thus have three color bands: red, green and blue. University. The associate editor coordinating the review of this manuscript and However, such approach will generally introduce many color approving it for publication was Dr. Kenneth K. M. Lam. artefacts, especially in textured areas, due to the neglection of The authors are with the Fuzziness and Uncertainty Modeling Research Unit, Department of Applied Mathematics and Computer Science, Ghent Univer- the correlation between the different color bands. To incorpo- sity, 9000 Ghent, Belgium (e-mail: email@example.com; mike.nachtegael rate this correlation, vector-based methods were introduced. @ugent.be; firstname.lastname@example.org; http://www.fuzzy.ugent.be). Most of these methods are based on ordering the vectors in a Color versions of one or more of the ﬁgures in this paper are available online at http://ieeexplore.ieee.org. predeﬁned ﬁltering window. The output for a given color pixel Digital Object Identiﬁer 10.1109/TIP.2010.2077305 is then the pixel in the window around the given pixel, that has 1057-7149/$26.00 © 2011 IEEE MÉLANGE et al.: FUZZY RANDOM IMPULSE NOISE REMOVAL FROM COLOR IMAGE SEQUENCES 969 is based on fuzzy rules in which information from spatial and temporal neighbors as well as from the other color bands is used. Detected noisy components are ﬁltered based on blockmatching where a noise adaptive mean absolute difference is used and where the search region contains pixels blocks from both the previous and current frame. The experiments showed that the proposed method outper- forms other state-of-the-art methods both in terms of objective measures such as MAE, PSNR and NCD and visually. REFERENCES  E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new efﬁ- cient approach for the removal of impulse noise from highly corrupted images,” IEEE Trans. Image Process., vol. 5, no. 6, pp. 1012–1025, 1996.  R. H. Chan, C. Hu, and M. Nikolova, “An iterative procedure for re- moving random-valued impulse noise,” IEEE Signal Process. Lett., vol. 11, no. 12, pp. 921–924, 2004.  S. M. M. Rahman, M. O. Ahmad, and M. N. S. Swamy, “Video de- noising based on inter-frame statistical modelling of wavelet coefﬁ- cients,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 2, pp. 187–198, 2007.  L. Jovanov, A. Pizurica, V. Zlokolica, S. Schulte, P. Schelkens, A. Munteanu, E. E. Kerre, and W. Philips, “Combined wavelet-domain and motion-compensated video denoising based on video codec mo- tion estimation methods,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 3, pp. 417–421, 2009.  H. B. Yin, X. Z. Fang, Z. Wei, and X. K. Yang, “An improved mo- tion-compensated 3-D LLMMSE ﬁlter with spatio-temporal adaptive ﬁltering support,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 12, pp. 1714–1727, 2007. Fig. 9. 20th frame of the “Deadline” sequence (top-left to bottom-right): orig-  L. Guo, O. C. Au, M. Ma, and Z. Liang, “Temporal video denoising inal, noisy (p = 25%), INRC, AVMF, VAVDMF and Proposed. based on multihypothesis motion compensation,” IEEE Trans. Circuits Syst. Video Technol, vol. 17, no. 10, pp. 1423–1429, 2007.  T. Mélange, M. Nachtegael, E. E. Kerre, V. Zlokolica, S. Schulte, V. De TABLE III Witte, A. Pizurica, and W. Philips, “Video denoising by fuzzy motion AVERAGE RUNNING TIME (SECONDS PER FRAME) FOR and detail adaptive averaging,” J. Electron. Imaging, vol. 17, no. 4, pp. THE PROCESSING OF THE “SALESMAN” SEQUENCE 043005–, 2008.  T. Chen, K. K. Ma, and L. H. Chen, “Tri-state median ﬁlter for image denoising,” IEEE Trans. Image Process., vol. 8, pp. 1834–1838, 1999.  R. C. Hardie and C. G. Boncelet, “LUM ﬁlters: A class of rank-order-based ﬁlters for smoothing and sharpening,” IEEE Trans. Signal Process., vol. 41, pp. 1061–1076, 1993.  S. J. Ko and Y. H. Lee, “Center weighted median ﬁlters and their ap- plications to image enhancement,” IEEE Trans. Circuits Syst., vol. 38, pp. 984–993, 1991.  S. M. Guo, C. S. Lee, and C. Y. Hsu, “An intelligent image agent based on soft-computing techniques for color image processing,” Expert Sys- tems With Appl., vol. 28, pp. 483–494, 2005.  S. Schulte, V. De Witte, M. Nachtegael, D. Van der Weken, and E. E. Kerre, “Fuzzy random impulse noise reduction method,” Fuzzy Sets Syst., vol. 158, pp. 270–283, 2007. stage could be sped up by using fast motion estimation tech-  S. Schulte, M. Nachtegael, V. De Witte, D. Van der Weken, and E. E. Kerre, “A fuzzy impulse noise detection and reduction method,” IEEE niques such as those presented in –. Next, for higher Trans. Image Process., vol. 15, no. 5, pp. 1153–1162, 2006. noise levels, it might be useful to do the block matching for  F. Russo, “Fire operators for image processing,” Fuzzy Sets Syst., vol. blocks of pixels and to ﬁlter each of the noisy pixels in the blocks 103, no. 2, pp. 265–275, 1999.  F. Russo, “Hybrid neuro-fuzzy ﬁlter for impulse noise removal,” Pat- at the same time instead of applying the block matching for each tern Recognit., vol. 32, pp. 1843–1855, 1999. noisy pixel separately. Further, in each of the successive steps  H. K. Kwan, “Fuzzy ﬁlters for noise reduction in images,” in Fuzzy Filters for Image Processing, M. Nachtegael, D. Van der Weken, D. in the algorithm, the detection and ﬁltering of a pixel does not Van De Ville, and E. E. Kerre, Eds. Heidelberg, Germany: Springer, depend on the detection and ﬁltering of the other pixels in the 2003, pp. 54–71. frame, such that the algorithm could be further sped up by per-  J. H. Wang, W. J. Liu, and L. D. Lin, “An histogram-based fuzzy ﬁlter for image restoration,” IEEE Trans. Syst. Man Cybern. B, Bern., vol. forming this detection and ﬁltering for several pixels in parallel. 32, no. 2, pp. 230–238, 2002.  H. Xu, G. Zhu, H. Peng, and D. Wang, “Adaptive fuzzy switching ﬁlter for images corrupted by impulse noise,” Pattern Recognit. Lett., vol. 25, IV. CONCLUSION pp. 1657–1663, 2004.  F. Cocchia, S. Carrato, and G. Ramponi, “Design and real-time imple- In this paper, we have presented a new ﬁltering framework mentation of a 3-D rational ﬁlter for edge preserving smoothing,” IEEE for color videos corrupted with random valued impulse noise. Trans. Consum. Electron., vol. 43, no. 4, pp. 1291–1300, 1997.  J.-S. Kim and H. W. Park, “Adaptive 3-D median ﬁltering for In order to preserve the details as much as possible, the noise is restoration of an image sequence corrupted by impulse noise,” Signal removed step by step. The detection of noisy color components Process.: Image Commun., vol. 16, pp. 657–668, 2001. 970 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011  P. S. Windyga, “Fast impulsive noise removal,” IEEE Trans. Image  Z. Xu, H. R. Wu, B. Qiu, and X. Yu, “Geometric features based ﬁltering Process., vol. 10, no. 1, pp. 173–179, 2001. for suppression of impulse noise in color images,” IEEE Trans. Image  R. Lukac and S. Marchevsky, “LUM smoother with smooth control for Process., vol. 18, no. 8, pp. 1742–1759, 2009. noisy image sequences,” EURASIP J. Appl. Signal Process., vol. 2001,  C.-C. Kang and W.-J. Wang, “Fuzzy reasoning-based directional me- no. 2, pp. 110–120, 2001. dian ﬁlter design,” Signal Process., vol. 89, no. 3, pp. 344–351, 2009.  M. El Hassouni, H. Cheriﬁ, and D. Aboutajdine, “HOS-based image  V. I. Ponomaryov, “Real-time 2D-3D ﬁltering using order statistics sequence noise removal,” IEEE Trans. Image Process., vol. 15, no. 3, based algorithms,” J. Real-Time Image Process., vol. 1, no. 3, pp. pp. 572–581, 2006. 173–194, 2007.  L. A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp.  J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Some im- 338–353, 1965. provements for image ﬁltering using peer group techniques,” Image  , E. E. Kerre and M. Nachtegael, Eds., “Fuzzy techniques in image Vis. Comput., vol. 28, no. 1, pp. 188–201, 2010. processing,” in Studies in Fuzziness and Soft Computing. Heidelberg,  S. Morillas, V. Gregori, and G. Peris-Fajarns, “Isolating impulsive Germany: Physica-Verlag, 2000, vol. 52. noise pixels in color images by peer group techniques,” Comput. Vis.  S. Weber, “A general concept of fuzzy connectives, negations and im- Image Underst., vol. 110, no. 1, pp. 102–116, 2008. plications based on t-norms and t-conorms,” Fuzzy Sets Syst., vol. 11,  S. Morillas, V. Gregori, G. Peris-Fajarns, and A. Sapena, “Local no. 2, pp. 115–134, 1983. self-adaptive fuzzy ﬁlter for impulsive noise removal in color images,”  L. M. Po and W. C. Ma, “A novel four-step search algorithm for fast Signal Process., vol. 88, no. 2, pp. 390–398, 2008. block motion estimation,” IEEE Trans. Circuits Syst. Video Technol.,  Q. Ma, L. Zhang, and B. Wang, “New strategy for image and video vol. 6, no. 3, pp. 313–317, 1996. quality assessment,” J. Electron. Imaging, vol. 19, no. 011019, 2010.  S. Zhu and K. K. Ma, “A new diamond search algorithm for fast block-  H. R. Sheikh and A. C. Bovik, “A visual information ﬁdelity approach matching motion estimation,” IEEE Trans. Image Process., vol. 9, no. to video quality assessment,” in Proc. 1st Int. Workshop on Video 2, pp. 287–290, 2000. Process. and Quality Metrics for Consumer Electron., 2005, pp.  C. Zhu, X. Lin, and L. P. Chau, “Hexagon-based search pattern for fast 23–25. block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 5, pp. 349–355, 2002.  V. Ponomaryov, A. Rosales-Silva, and V. Golikov, “Adaptive and vector directional processing applied to video colour images,” Elec- tron. Lett., vol. 42, no. 11, pp. 623–624, 2006.  R. Lukac, “Vector order-statistics for impulse detection in noisy color image sequences,” in Proc. 4th EURASIP-IEEE Region 8 Int. Symp. Tom Mélange was born in Kortrijk, Belgium, in Video/Image Process. Multimedia Commun., Zadar, Croatia, 2002. 1984. He received the M.Sc. degree in mathematics  R. Lukac, “Adaptive vector median ﬁltering,” Pattern Recognit. Lett., from Ghent University, Ghent, Belgium, in 2006. vol. 24, pp. 1889–1899, 2003. In October 2006, he joined the Department of  S. Schulte, S. Morillas, V. Gregori, and E. E. Kerre, “A new fuzzy Applied Mathematics and Computer Science, Ghent color correlated impulse noise reduction method,” IEEE Trans. Image University, where he is a member of the Fuzziness Process., vol. 16, no. 10, pp. 2565–2575, 2007. and Uncertainty Modeling Research Unit. In 2010  R. Lukac, K. N. Plataniotis, A. N. Venetsanopoulos, and B. Smolka, he received the Ph.D. degree with a thesis on “A statistically-switched adaptive vector median ﬁlter,” J. Intell. Robot. fuzzy techniques for noise reduction in video and Syst., vol. 42, no. 4, pp. 361–391, 2005. interval-valued fuzzy mathematical morphology,  J. Camacho, S. Morillas, and P. Latorre, “Efﬁcient impulse noise sup- under Prof. Dr. E. E. Kerre. pression based on statistical conﬁdence limits,” J. Imag. Sci. Technol., vol. 50, no. 5, pp. 427–436, 2006.  S. Hore, B. Qiu, and H. R. Wu, “Improved vector ﬁltering for color images using fuzzy noise detection,” Opt. Eng., vol. 42, no. 6, pp. 1656–1664, 2003. Mike Nachtegael was born on February 16, 1976. He  R. Lukac and K. N. Plataniotis, “A taxonomy of color image ﬁltering received the M.Sc. degree in mathematics from Ghent and enhancement solutions,” Adv. Imag. Electron. Phys., vol. 140, pp. University, Ghent, Belgium, in 1998. In the same year 187–264, 2006. he joined the Department of Applied Mathematics  R. Lukac, “Adaptive color image ﬁltering based on center-weighted and Computer Science, where he is a member of the vector directional ﬁlters,” Multidimen. Syst. Signal Process., vol. 15, Fuzziness and Uncertainty Modeling Research Unit. no. 2, pp. 169–196, 2004. In May 2002 he received the Ph.D. in mathematics,  Z. H. Ma, H. R. Wu, and B. Qiu, “A robust structure-adaptive hybrid on the topic of fuzzy techniques in image processing. vector ﬁlter for color image restoration,” IEEE Trans. Image Process., In 2002, he became an Assistant Professor in his vol. 14, no. 12, pp. 1990–2001, 2005. Department and since 2008 he has held the position  V. Chatzis and I. Pitas, “Fuzzy scalar and vector median ﬁlters based of Guest Professor. on fuzzy distances,” IEEE Trans. Image Process., vol. 8, no. 5, pp. 731–734, 1999.  S. Morillas, V. Gregori, G. Peris-Fajarns, and P. Latorre, “A fast im- pulsive noise color image ﬁlter using fuzzy metrics,” Real-Time Imag., Etienne E. Kerre was born in Zele, Belgium, on May vol. 11, no. 5–6, pp. 417–428, 2005. 8, 1945. He received the M.Sc. degree in mathematics  S. J. Sangwine and R. E. N. Horne, The Colour Image Processing and the Ph.D. in mathematics from Ghent University, Handbook. London, U.K.: Chapman & Hall, 1998. Ghent, Belgium, in 1967 and 1970, respectively.  J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Fast detection and removal of impulsive noise using peer groups and fuzzy metrics,” Since 1984, he has been a Lector, and since 1991, J. Vis. Commun. Image Represent., vol. 19, no. 1, pp. 20–29, 2008. a full Professor at Ghent University. He is a referee  S. Morillas, V. Gregori, and A. Hervs, “Fuzzy peer groups for reducing for more than 30 international scientiﬁc journals, and mixed Gaussian-impulse noise from color images,” IEEE Trans. Image a member of the editorial board of international jour- Process., vol. 18, no. 7, pp. 1452–1466, 2009. nals and conferences on fuzzy set theory. He has been  V. Ponomaryov, A. Rosales, and F. Gallegos, “3D ﬁltering of colour an honorary chairman at various international confer- video sequences using fuzzy logic and vector order statistics,” in Proc. ences. In 1976, he founded the Fuzziness and Uncer- Advanced Concepts for Intelligent Vision Systems, LNCS 5807, 2001, tainty Modeling Research Unit (FUM) and since then his research has been pp. 210–221. focused on the modeling of fuzziness and uncertainty, and has resulted in a  V. F. Kravchenko, V. I. Ponomaryov, and V. I. Pustovoit, “Three-di- great number of contributions in fuzzy set theory and its various generalizations. mensional ﬁltration of multichannel video sequences on the basis of Especially the theories of fuzzy relational calculus and of fuzzy mathematical fuzzy-set theory,” Doklady Phys., vol. 55, no. 2, pp. 58–63, 2010. structures owe a great deal to him. Over the years he has also been a promotor  V. Ponomaryov, F. Gallegos-Funes, and A. Rosales-Silva, “Fuzzy of 29 Ph.D.s on fuzzy set theory. His current research interests include fuzzy directional (FD) ﬁlter to remove impulse noise from colour images,” and intuitionistic fuzzy relations, fuzzy topology, and fuzzy image processing. IEICE Trans. Fundament. Electron., Commun. Comput. Sci., vol. He has authored or co-authored 25 books, and more than 450 papers in interna- E93-A, no. 2, pp. 570–572, 2010. tional refereed journals and proceedings.
Pages to are hidden for
"0.1.15.1.2011.04.ARO.Fuzzy Random Impulse Noise Removal From Color Image Sequences.TIP"Please download to view full document