IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011 959
Fuzzy Random Impulse Noise Removal
From Color Image Sequences
Tom Mélange, Mike Nachtegael, and Etienne E. Kerre
Abstract—In this paper, a new fuzzy ﬁlter for the removal of which allows a more gradual transition between belonging to
random impulse noise in color video is presented. By working with and not belonging to. Such gradual transition makes fuzzy sets
different successive ﬁltering steps, a very good tradeoff between very useful for the processing of human knowledge in terms of
detail preservation and noise removal is obtained. One strong ﬁl-
tering step that should remove all noise at once would inevitably linguistic variables (e.g., large, small, etc.). There is for example
also remove a considerable amount of detail. Therefore, the noise no need to use a threshold to decide whether a difference in color
is ﬁltered step by step. In each step, noisy pixels are detected by component value between two pixels is large or not. Two differ-
the help of fuzzy rules, which are very useful for the processing of ences that differ only one unit (which is not noticeable by the
human knowledge where linguistic variables are used. Pixels that
are detected as noisy are ﬁltered, the others remain unchanged. Fil-
human eye) could then be respectively large and not large. It is
tering of detected pixels is done by blockmatching based on a noise better to allow a difference to be large to some intermediate de-
adaptive mean absolute difference. The experiments show that the gree. For a larger difference, this degree will be higher than that
proposed method outperforms other state-of-the-art ﬁlters both vi- of a smaller difference. For an illustration of the effectiveness
sually and in terms of objective quality measures such as the mean of fuzzy set theory in image processing, we refer to, e.g., .
absolute error (MAE), the peak-signal-to-noise ratio (PSNR) and
the normalized color difference (NCD). Most ﬁlters in literature, that are developed for video, are
intended for sequences corrupted by additive Gaussian noise
Index Terms—Circuits and systems, computers and information
processing, computational and artiﬁcial intelligence, ﬁltering, ﬁl-
(e.g., –). Only few video ﬁlters for the impulse noise case
ters, fuzzy logic, image denoising, logic, nonlinear ﬁlters. can be found (e.g., –, , , , ). However,
several impulse noise ﬁlters for still images exist. The best
known among them are the median based rank-order ﬁlters
I. INTRODUCTION (e.g., –, , –, . But also some fuzzy tech-
niques can be found –, , . Such 2-D ﬁlters could
MAGES and videos belong to the most important infor-
I mation carriers in today’s world (e.g., trafﬁc observations,
surveillance systems, autonomous navigation, etc.). However,
be used to ﬁlter each of the frames of a video successively.
However, temporal inconsistencies will arise due to the neglec-
tion of the temporal correlation between successive frames.
the images are likely to be corrupted by noise due to bad acquisi-
A better alternative would be to use 3-D ﬁltering windows,
tion, transmission or recording. Such degradation negatively in-
in which also pixels from neighboring frames are taken into
ﬂuences the performance of many image processing techniques
account –, , , , . The main problem in
and a preprocessing module to ﬁlter the images is often required.
using neighboring frames is motion between them. Using pixels
Among those ﬁlters, more and more fuzzy techniques start
at corresponding spatial positions in neighboring frames for
to appear in literature , –, , , , –,
noise removal may introduce ghosting artifacts in the presence
, , . Fuzzy set theory was introduced by Zadeh in
of camera and object motion. In the method proposed in this
1965  and is a generalization of classical set theory. A clas-
paper, we will therefore only in non-moving areas assign a
sical crisp set over a universe can be modelled by a
temporal impulse between two corresponding spatial positions
mapping (characteristic function): an element belongs to
to noise (detection phase) and for the replacement of a noisy
the set or does not belong to it. Fuzzy sets are now modelled as
pixel (ﬁltering phase) motion compensation will be applied to
mappings (membership functions). So the character-
ﬁnd the most reliable pixel in the previous frame.
istic function is extended to a membership function where also
Analogously, a distinction between ﬁlters intended for
membership degrees between zero and one are allowed. An el-
greyscale images and for color images needs to be made. Filters
ement can now also belong to some degree to the set,
for greyscale images could be used for color images by applying
them on each of the color bands of the image separately. In this
Manuscript received March 25, 2010; revised July 12, 2010; accepted August paper, we consider the images to be modeled in the RGB color
31, 2010. Date of publication September 20, 2010; date of current version March
18, 2011. This work was supported by the GOA project B/04138/01 of Ghent space and we thus have three color bands: red, green and blue.
University. The associate editor coordinating the review of this manuscript and However, such approach will generally introduce many color
approving it for publication was Dr. Kenneth K. M. Lam. artefacts, especially in textured areas, due to the neglection of
The authors are with the Fuzziness and Uncertainty Modeling Research Unit,
Department of Applied Mathematics and Computer Science, Ghent Univer- the correlation between the different color bands. To incorpo-
sity, 9000 Ghent, Belgium (e-mail: firstname.lastname@example.org; mike.nachtegael rate this correlation, vector-based methods were introduced.
@ugent.be; email@example.com; http://www.fuzzy.ugent.be). Most of these methods are based on ordering the vectors in a
Color versions of one or more of the ﬁgures in this paper are available online
at http://ieeexplore.ieee.org. predeﬁned ﬁltering window. The output for a given color pixel
Digital Object Identiﬁer 10.1109/TIP.2010.2077305 is then the pixel in the window around the given pixel, that has
1057-7149/$26.00 © 2011 IEEE
MÉLANGE et al.: FUZZY RANDOM IMPULSE NOISE REMOVAL FROM COLOR IMAGE SEQUENCES 969
is based on fuzzy rules in which information from spatial and
temporal neighbors as well as from the other color bands is used.
Detected noisy components are ﬁltered based on blockmatching
where a noise adaptive mean absolute difference is used and
where the search region contains pixels blocks from both the
previous and current frame.
The experiments showed that the proposed method outper-
forms other state-of-the-art methods both in terms of objective
measures such as MAE, PSNR and NCD and visually.
 E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new efﬁ-
cient approach for the removal of impulse noise from highly corrupted
images,” IEEE Trans. Image Process., vol. 5, no. 6, pp. 1012–1025,
 R. H. Chan, C. Hu, and M. Nikolova, “An iterative procedure for re-
moving random-valued impulse noise,” IEEE Signal Process. Lett., vol.
11, no. 12, pp. 921–924, 2004.
 S. M. M. Rahman, M. O. Ahmad, and M. N. S. Swamy, “Video de-
noising based on inter-frame statistical modelling of wavelet coefﬁ-
cients,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 2, pp.
 L. Jovanov, A. Pizurica, V. Zlokolica, S. Schulte, P. Schelkens, A.
Munteanu, E. E. Kerre, and W. Philips, “Combined wavelet-domain
and motion-compensated video denoising based on video codec mo-
tion estimation methods,” IEEE Trans. Circuits Syst. Video Technol.,
vol. 19, no. 3, pp. 417–421, 2009.
 H. B. Yin, X. Z. Fang, Z. Wei, and X. K. Yang, “An improved mo-
tion-compensated 3-D LLMMSE ﬁlter with spatio-temporal adaptive
ﬁltering support,” IEEE Trans. Circuits Syst. Video Technol., vol. 17,
no. 12, pp. 1714–1727, 2007.
Fig. 9. 20th frame of the “Deadline” sequence (top-left to bottom-right): orig-  L. Guo, O. C. Au, M. Ma, and Z. Liang, “Temporal video denoising
inal, noisy (p = 25%), INRC, AVMF, VAVDMF and Proposed. based on multihypothesis motion compensation,” IEEE Trans. Circuits
Syst. Video Technol, vol. 17, no. 10, pp. 1423–1429, 2007.
 T. Mélange, M. Nachtegael, E. E. Kerre, V. Zlokolica, S. Schulte, V. De
TABLE III Witte, A. Pizurica, and W. Philips, “Video denoising by fuzzy motion
AVERAGE RUNNING TIME (SECONDS PER FRAME) FOR and detail adaptive averaging,” J. Electron. Imaging, vol. 17, no. 4, pp.
THE PROCESSING OF THE “SALESMAN” SEQUENCE 043005–, 2008.
 T. Chen, K. K. Ma, and L. H. Chen, “Tri-state median ﬁlter for image
denoising,” IEEE Trans. Image Process., vol. 8, pp. 1834–1838, 1999.
 R. C. Hardie and C. G. Boncelet, “LUM ﬁlters: A class of
rank-order-based ﬁlters for smoothing and sharpening,” IEEE Trans.
Signal Process., vol. 41, pp. 1061–1076, 1993.
 S. J. Ko and Y. H. Lee, “Center weighted median ﬁlters and their ap-
plications to image enhancement,” IEEE Trans. Circuits Syst., vol. 38,
pp. 984–993, 1991.
 S. M. Guo, C. S. Lee, and C. Y. Hsu, “An intelligent image agent based
on soft-computing techniques for color image processing,” Expert Sys-
tems With Appl., vol. 28, pp. 483–494, 2005.
 S. Schulte, V. De Witte, M. Nachtegael, D. Van der Weken, and E. E.
Kerre, “Fuzzy random impulse noise reduction method,” Fuzzy Sets
Syst., vol. 158, pp. 270–283, 2007.
stage could be sped up by using fast motion estimation tech-  S. Schulte, M. Nachtegael, V. De Witte, D. Van der Weken, and E. E.
Kerre, “A fuzzy impulse noise detection and reduction method,” IEEE
niques such as those presented in –. Next, for higher Trans. Image Process., vol. 15, no. 5, pp. 1153–1162, 2006.
noise levels, it might be useful to do the block matching for  F. Russo, “Fire operators for image processing,” Fuzzy Sets Syst., vol.
blocks of pixels and to ﬁlter each of the noisy pixels in the blocks 103, no. 2, pp. 265–275, 1999.
 F. Russo, “Hybrid neuro-fuzzy ﬁlter for impulse noise removal,” Pat-
at the same time instead of applying the block matching for each tern Recognit., vol. 32, pp. 1843–1855, 1999.
noisy pixel separately. Further, in each of the successive steps  H. K. Kwan, “Fuzzy ﬁlters for noise reduction in images,” in Fuzzy
Filters for Image Processing, M. Nachtegael, D. Van der Weken, D.
in the algorithm, the detection and ﬁltering of a pixel does not Van De Ville, and E. E. Kerre, Eds. Heidelberg, Germany: Springer,
depend on the detection and ﬁltering of the other pixels in the 2003, pp. 54–71.
frame, such that the algorithm could be further sped up by per-  J. H. Wang, W. J. Liu, and L. D. Lin, “An histogram-based fuzzy ﬁlter
for image restoration,” IEEE Trans. Syst. Man Cybern. B, Bern., vol.
forming this detection and ﬁltering for several pixels in parallel. 32, no. 2, pp. 230–238, 2002.
 H. Xu, G. Zhu, H. Peng, and D. Wang, “Adaptive fuzzy switching ﬁlter
for images corrupted by impulse noise,” Pattern Recognit. Lett., vol. 25,
IV. CONCLUSION pp. 1657–1663, 2004.
 F. Cocchia, S. Carrato, and G. Ramponi, “Design and real-time imple-
In this paper, we have presented a new ﬁltering framework mentation of a 3-D rational ﬁlter for edge preserving smoothing,” IEEE
for color videos corrupted with random valued impulse noise. Trans. Consum. Electron., vol. 43, no. 4, pp. 1291–1300, 1997.
 J.-S. Kim and H. W. Park, “Adaptive 3-D median ﬁltering for
In order to preserve the details as much as possible, the noise is restoration of an image sequence corrupted by impulse noise,” Signal
removed step by step. The detection of noisy color components Process.: Image Commun., vol. 16, pp. 657–668, 2001.
970 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 4, APRIL 2011
 P. S. Windyga, “Fast impulsive noise removal,” IEEE Trans. Image  Z. Xu, H. R. Wu, B. Qiu, and X. Yu, “Geometric features based ﬁltering
Process., vol. 10, no. 1, pp. 173–179, 2001. for suppression of impulse noise in color images,” IEEE Trans. Image
 R. Lukac and S. Marchevsky, “LUM smoother with smooth control for Process., vol. 18, no. 8, pp. 1742–1759, 2009.
noisy image sequences,” EURASIP J. Appl. Signal Process., vol. 2001,  C.-C. Kang and W.-J. Wang, “Fuzzy reasoning-based directional me-
no. 2, pp. 110–120, 2001. dian ﬁlter design,” Signal Process., vol. 89, no. 3, pp. 344–351, 2009.
 M. El Hassouni, H. Cheriﬁ, and D. Aboutajdine, “HOS-based image  V. I. Ponomaryov, “Real-time 2D-3D ﬁltering using order statistics
sequence noise removal,” IEEE Trans. Image Process., vol. 15, no. 3, based algorithms,” J. Real-Time Image Process., vol. 1, no. 3, pp.
pp. 572–581, 2006. 173–194, 2007.
 L. A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp.  J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Some im-
338–353, 1965. provements for image ﬁltering using peer group techniques,” Image
 , E. E. Kerre and M. Nachtegael, Eds., “Fuzzy techniques in image Vis. Comput., vol. 28, no. 1, pp. 188–201, 2010.
processing,” in Studies in Fuzziness and Soft Computing. Heidelberg,  S. Morillas, V. Gregori, and G. Peris-Fajarns, “Isolating impulsive
Germany: Physica-Verlag, 2000, vol. 52. noise pixels in color images by peer group techniques,” Comput. Vis.
 S. Weber, “A general concept of fuzzy connectives, negations and im- Image Underst., vol. 110, no. 1, pp. 102–116, 2008.
plications based on t-norms and t-conorms,” Fuzzy Sets Syst., vol. 11,  S. Morillas, V. Gregori, G. Peris-Fajarns, and A. Sapena, “Local
no. 2, pp. 115–134, 1983. self-adaptive fuzzy ﬁlter for impulsive noise removal in color images,”
 L. M. Po and W. C. Ma, “A novel four-step search algorithm for fast Signal Process., vol. 88, no. 2, pp. 390–398, 2008.
block motion estimation,” IEEE Trans. Circuits Syst. Video Technol.,  Q. Ma, L. Zhang, and B. Wang, “New strategy for image and video
vol. 6, no. 3, pp. 313–317, 1996. quality assessment,” J. Electron. Imaging, vol. 19, no. 011019, 2010.
 S. Zhu and K. K. Ma, “A new diamond search algorithm for fast block-  H. R. Sheikh and A. C. Bovik, “A visual information ﬁdelity approach
matching motion estimation,” IEEE Trans. Image Process., vol. 9, no. to video quality assessment,” in Proc. 1st Int. Workshop on Video
2, pp. 287–290, 2000. Process. and Quality Metrics for Consumer Electron., 2005, pp.
 C. Zhu, X. Lin, and L. P. Chau, “Hexagon-based search pattern for fast 23–25.
block motion estimation,” IEEE Trans. Circuits Syst. Video Technol.,
vol. 12, no. 5, pp. 349–355, 2002.
 V. Ponomaryov, A. Rosales-Silva, and V. Golikov, “Adaptive and
vector directional processing applied to video colour images,” Elec-
tron. Lett., vol. 42, no. 11, pp. 623–624, 2006.
 R. Lukac, “Vector order-statistics for impulse detection in noisy color
image sequences,” in Proc. 4th EURASIP-IEEE Region 8 Int. Symp. Tom Mélange was born in Kortrijk, Belgium, in
Video/Image Process. Multimedia Commun., Zadar, Croatia, 2002. 1984. He received the M.Sc. degree in mathematics
 R. Lukac, “Adaptive vector median ﬁltering,” Pattern Recognit. Lett., from Ghent University, Ghent, Belgium, in 2006.
vol. 24, pp. 1889–1899, 2003. In October 2006, he joined the Department of
 S. Schulte, S. Morillas, V. Gregori, and E. E. Kerre, “A new fuzzy Applied Mathematics and Computer Science, Ghent
color correlated impulse noise reduction method,” IEEE Trans. Image University, where he is a member of the Fuzziness
Process., vol. 16, no. 10, pp. 2565–2575, 2007. and Uncertainty Modeling Research Unit. In 2010
 R. Lukac, K. N. Plataniotis, A. N. Venetsanopoulos, and B. Smolka, he received the Ph.D. degree with a thesis on
“A statistically-switched adaptive vector median ﬁlter,” J. Intell. Robot. fuzzy techniques for noise reduction in video and
Syst., vol. 42, no. 4, pp. 361–391, 2005. interval-valued fuzzy mathematical morphology,
 J. Camacho, S. Morillas, and P. Latorre, “Efﬁcient impulse noise sup- under Prof. Dr. E. E. Kerre.
pression based on statistical conﬁdence limits,” J. Imag. Sci. Technol.,
vol. 50, no. 5, pp. 427–436, 2006.
 S. Hore, B. Qiu, and H. R. Wu, “Improved vector ﬁltering for color
images using fuzzy noise detection,” Opt. Eng., vol. 42, no. 6, pp.
1656–1664, 2003. Mike Nachtegael was born on February 16, 1976. He
 R. Lukac and K. N. Plataniotis, “A taxonomy of color image ﬁltering received the M.Sc. degree in mathematics from Ghent
and enhancement solutions,” Adv. Imag. Electron. Phys., vol. 140, pp. University, Ghent, Belgium, in 1998. In the same year
187–264, 2006. he joined the Department of Applied Mathematics
 R. Lukac, “Adaptive color image ﬁltering based on center-weighted and Computer Science, where he is a member of the
vector directional ﬁlters,” Multidimen. Syst. Signal Process., vol. 15, Fuzziness and Uncertainty Modeling Research Unit.
no. 2, pp. 169–196, 2004. In May 2002 he received the Ph.D. in mathematics,
 Z. H. Ma, H. R. Wu, and B. Qiu, “A robust structure-adaptive hybrid on the topic of fuzzy techniques in image processing.
vector ﬁlter for color image restoration,” IEEE Trans. Image Process., In 2002, he became an Assistant Professor in his
vol. 14, no. 12, pp. 1990–2001, 2005. Department and since 2008 he has held the position
 V. Chatzis and I. Pitas, “Fuzzy scalar and vector median ﬁlters based of Guest Professor.
on fuzzy distances,” IEEE Trans. Image Process., vol. 8, no. 5, pp.
 S. Morillas, V. Gregori, G. Peris-Fajarns, and P. Latorre, “A fast im-
pulsive noise color image ﬁlter using fuzzy metrics,” Real-Time Imag., Etienne E. Kerre was born in Zele, Belgium, on May
vol. 11, no. 5–6, pp. 417–428, 2005.
8, 1945. He received the M.Sc. degree in mathematics
 S. J. Sangwine and R. E. N. Horne, The Colour Image Processing
and the Ph.D. in mathematics from Ghent University,
Handbook. London, U.K.: Chapman & Hall, 1998.
Ghent, Belgium, in 1967 and 1970, respectively.
 J. G. Camarena, V. Gregori, S. Morillas, and A. Sapena, “Fast detection
and removal of impulsive noise using peer groups and fuzzy metrics,” Since 1984, he has been a Lector, and since 1991,
J. Vis. Commun. Image Represent., vol. 19, no. 1, pp. 20–29, 2008. a full Professor at Ghent University. He is a referee
 S. Morillas, V. Gregori, and A. Hervs, “Fuzzy peer groups for reducing for more than 30 international scientiﬁc journals, and
mixed Gaussian-impulse noise from color images,” IEEE Trans. Image a member of the editorial board of international jour-
Process., vol. 18, no. 7, pp. 1452–1466, 2009. nals and conferences on fuzzy set theory. He has been
 V. Ponomaryov, A. Rosales, and F. Gallegos, “3D ﬁltering of colour an honorary chairman at various international confer-
video sequences using fuzzy logic and vector order statistics,” in Proc. ences. In 1976, he founded the Fuzziness and Uncer-
Advanced Concepts for Intelligent Vision Systems, LNCS 5807, 2001, tainty Modeling Research Unit (FUM) and since then his research has been
pp. 210–221. focused on the modeling of fuzziness and uncertainty, and has resulted in a
 V. F. Kravchenko, V. I. Ponomaryov, and V. I. Pustovoit, “Three-di- great number of contributions in fuzzy set theory and its various generalizations.
mensional ﬁltration of multichannel video sequences on the basis of Especially the theories of fuzzy relational calculus and of fuzzy mathematical
fuzzy-set theory,” Doklady Phys., vol. 55, no. 2, pp. 58–63, 2010. structures owe a great deal to him. Over the years he has also been a promotor
 V. Ponomaryov, F. Gallegos-Funes, and A. Rosales-Silva, “Fuzzy of 29 Ph.D.s on fuzzy set theory. His current research interests include fuzzy
directional (FD) ﬁlter to remove impulse noise from colour images,” and intuitionistic fuzzy relations, fuzzy topology, and fuzzy image processing.
IEICE Trans. Fundament. Electron., Commun. Comput. Sci., vol. He has authored or co-authored 25 books, and more than 450 papers in interna-
E93-A, no. 2, pp. 570–572, 2010. tional refereed journals and proceedings.