Preprocessing of video image with unconstrained background for Drowsy Driver Detection

Document Sample
Preprocessing of video image with unconstrained background for Drowsy Driver Detection Powered By Docstoc
					                                                                   (IJCSIS) International Journal of Computer Science and Information Security,
                                                                   Vol. 8, No. 2, May 2010

   Preprocessing of video image with unconstrained
      background for Drowsy Driver Detection
                                                   M.Moorthi1, Dr. M.Arthanari2, M.Sivakumar3
                               Assistant Professor, Kongu Arts and Science College, Erode – 638 107, Tamil Nadu, India
                   Prof. & Head, Tejaa Sakthi Institute of Technology for Women, Coimbatore – 641 659, Tamil Nadu, India
                                        Doctoral Research Scholar, Anna University, Coimbatore, Tamil Nadu, India

                           Email:,, , Email:

                                        Abstract                                    of the problem and illumination changes, robustness and
                                                                                    preprocessing steps of these approaches are still a problem.
            The     face       recognition     includes   enhancement   and         Most commonly, natural face feature templates taken from real
segmentation of face image, detection of face boundary and facial                   person are used for a template matching algorithm [1],[2].
features, matching of extracted features, and finally recognition of the            These templates have to satisfy a set of requirements like
face. Though a number of algorithms are devised for face recognition,               orientation, size and illumination. Therefore preprocessing step
the technology is not matured enough to recognize the face of a person
                                                                                    is necessary for at least aligning and size changes. A wavelets
since the algorithm deal with significant amount of illumination
                                                                                    based approach is described in [3]. Face images and face
variation in image. We propose a new image preprocessing algorithm
                                                                                    features from a database have to be aligned in orientation and
that compensates for the problem. The proposed algorithm enhances
the contrast of images by transforming the values in an intensity                   size in preprocessing step. Both previous described methods
image, so that the histogram of the output image is approximately                   are limited by the used template and face database.
uniformly distributed on pixel. Our algorithm does not require any
training steps or reflective surface models for illumination                                 In this paper we propose a novel low cost method
compensation. We apply the algorithm to face images prior to                        designed for preprocessing. The preprocessing has three steps.
recognition. Simulation is done using seventy five web camera images                In first steps modified histogram equalization is used to
using Mat lab 7.0.
                                                                                    enhance the brightness and contrast of the images. In steps two,
                                                                                    median filter is used to remove salt and pepper noise. Third,
Keywords: Facial recognition, Facial features extraction, Eye
                                                                                    Binary image are obtained through the thresholding.
                                 1. Introduction
                                                                                             This paper is organized as follows. Literature surveys
            The preprocessing of real image is a crucial aspect in
                                                                                    are given in section 2. In section 3 we will devote ourselves to
many useful applications like video coding of faces for video
                                                                                    discussing the preprocessing method in detail. Experimental
phony, animation of synthetic faces, driver behaviors analysis,
                                                                                    results are reported in section 4. Conclusions will be drawn in
word visual recognition, expression and emotion analysis,
                                                                                    section 5.
tracing and recognition of faces. The detection of facial
features has been approached by many researchers and a
variety of methods exist. Nevertheless, due to the complexity

                                                                                                               ISSN 1947-5500
                                                         (IJCSIS) International Journal of Computer Science and Information Security,
                                                         Vol. 8, No. 2, May 2010

                     2. Literature Survey                                 enhanced method of histogram equalization is used to
                                                                          preprocess the image.
         Besides pose variation, illumination is the most
significant factor affecting the appearance of faces. Ambient                                     3. Preprocessing
lighting changes greatly within and between days and among                         In order to obtain appropriately-segmented binary
indoor and outdoor environments. Due to the 3D shape of the               images, an image preprocessing is applied. To compensate for
face, a direct lighting source can cast strong shadows that               illumination variations and to obtain more image details,
accentuate or diminish certain facial features. Evaluations of            modified histogram equalization is used to enhance the
face recognition algorithms consistently show that state-of-the-          brightness and the contrast of the images. Then a median filter
art systems can not deal with large differences in illumination           is used to remove the noise. Binary images are obtained
conditions between gallery and probe images [1-3].                        through thresholding. The preprocessing steps are shown in Fig
                                                                          1.Let we see the steps of preprocessing one by one.
         The face detection algorithms are based on either gray
level template matching or computation of geometric                       3.1. Capturing image
relationships among facial features. In recent years many                          The required images are taken from the video image
appearance-based algorithms have been proposed to deal with               using web camera.
the problem [4-7]. Belhumeur showed [5] that the set of images            3.2 Enhancing the image
of an object in fixed pose but under varying illumination forms                    Histogram equalization is a method in image
a convex cone in the space of images. The illumination cones              processing of contrast adjustment using the image's histogram.
of human faces can be approximated well by low-dimensional                This method usually increases the global contrast of many
linear subspaces [8]. The linear subspaces are typically                  images, especially when the usable data of the image is
estimated from training data, requiring multiple images of the            represented by close contrast values. Through this adjustment,
object under different illumination conditions. Alternatively,            the intensities can be better distributed on the histogram. This
model-based approaches have been proposed to address the                  allows for areas of lower local contrast to gain a higher contrast
problem. Blanz et al. [9] fit a previously constructed morphable          without affecting the global contrast. Histogram equalization
3D model to single images. The algorithm works well across                accomplishes this by effectively spreading out the most
pose and illumination, however, the computational expense is              frequent intensity values.
very high.
                                                                                   The method is useful in images with backgrounds and
         In general, an image I(x; y) is regarded as product I(x;         foregrounds that are both bright or both dark. A key advantage
y) = R(x; y)L(x; y) where R(x; y) is the reflectance and L(x; y)          of the method is that it is a fairly straightforward technique and
is the illuminance at each point (x; y) [10]. Computing the               an invertible operator. So in theory, if the histogram
reflectance and the illuminence fields from real images is, in            equalization function is known, then the original histogram can
general, an ill-posed problem. Therefore, various assumptions             be recovered. The calculation is not computationally intensive.
and simplifications about L, or R, or both are proposed in order
to attempt to solve the problem. A common assumption is that                       Histogram equalization often produces unrealistic
L varies slowly while R can change abruptly. For example,                 effects in photographs; however it is very useful for scientific
Homomorphic filtering [11] uses this assumption to extract R              images like thermal, satellite or x-ray images, often the same
by high-pass filtering the logarithm of the image. In this paper,         class of images that user would apply false-color to

                                                                                                       ISSN 1947-5500
                                                           (IJCSIS) International Journal of Computer Science and Information Security,
                                                           Vol. 8, No. 2, May 2010

                                                                           contrast. Examples of such methods include adaptive histogram

                         Input                                             equalization    and   contrast   limiting      adaptive      histogram
                        Video                                              equalization.

                                                                                    Histogram equalization also seems to be used in
                       video                                               biological neural networks so as to maximize the output firing
                       image                                               rate of the neuron as a function of the input statistics. This has

                       Median                                              been proved in particular in the fly retina. Histogram
                       filtering                                           equalization is a specific case of the more general class of
                                                                           histogram remapping methods. These methods seek to adjust
                                                                           the image to make it easier to analyze or improve visual quality

                                                                           3.3. Proposed Modification
                                                                                    While the results of a standard histogram equalization
                                                                           filtering over the whole image just described give promising
                   Fig. 1: Preprocessing steps.
                                                                           results, we wanted to see if the results could be further
         . Also histogram equalization can produce undesirable             improved. Many well-known enhancement algorithms such as
effects (like visible image gradient) when applied to images               histogram equalization and homomorphic filtering are global in
with low color depth. For example if applied to 8-bit image                nature and are intended to enhance an image and deal with it as
displayed with 8-bit gray-scale palette it will further reduce             a whole. We tried to split the original image in sub-images and
color depth (number of unique shades of gray) of the image.                filter each sub-image individually. First we decided to try and
Histogram equalization will work the best when applied to                  split the image into two halves vertically (thus obtaining two
images with much higher color depth than palette size, like                sub-images of the original image) and then apply the filter to
continuous data or 16-bit gray-scale images.                               each half individually. Second idea was to split the image
                                                                           horizontally and again apply the filter to each half individually.
         There are two ways to think about and implement                   Encouraged by the good results obtained with both these
histogram equalization, either as image change or as palette               methods (see Section 4 for details) we further tried to combine
change. The operation can be expressed as P(M(I)) where I is               the filtering results into a joint representation. Let IHEV(x,y) be
the original image, M is histogram equalization mapping                    the image split vertically and each half filtered with histogram
operation and P is a palette. If we define new palette as                  equalization filter individually, let IHEH(x,y) be the same for
P'=P(M) and leave image I unchanged then histogram                         horizontally split images and let IHEMOD(x,y) be our proposed
equalization is implemented as palette change. On the other                modification:
hand if palette P remains unchanged and image is modified to
I'=M(I) then the implementation is by image change. In most                         IHEMOD(x, y) = 0.5[ IHEV(x, y) + .70 IHEH(x,y)]
cases palette change is better as it preserves the original data.
                                                                                    Since IHEV scored higher results than IHEH in our
         Generalizations     of    this   method    use   multiple         tests we decided to keep the whole IHEV and multiply IHEH
histograms to emphasize local contrast, rather than overall                with a constant of 0.70 (chosen based on experimental results),
                                                                                                       ISSN 1947-5500
                                                          (IJCSIS) International Journal of Computer Science and Information Security,
                                                          Vol. 8, No. 2, May 2010

to lower its influence on the final representation. This                   eliminated by the conventional connected components labeling
combination produced highest results in our experiments and                process.
was kept as a final representation. We will show in the
following section that our method yields superior results, and             3.6 Binary image
therefore justifies further research of the histogram equalization                    Binary images are obtained through the thresholding.
filtering variations as a means of simple yet efficient image              Then the final feature image is obtained, as shown in Fig. 4 (c).
                                                                                               4. Experimental Results
         As shown in Fig. 4 (a), the input image has low                              The proposed method was tested on the real video
contrast due to illumination; segmentation results, therefore,             images. The video image of [480 x 640 pixels] of 75 different
are unlikely to be good. Fig. 4 (b) demonstrates the image                 test persons and has been recorded during several sessions at
enhanced by modified histogram equalization the contrast is                different places. This set features a larger variety of
improved, and the details in the face region are enhanced which            illumination, background and face size. It stresses real world
are discussed in detail in the following section.                          constraints. So it is believed to be more difficult than other
                                                                           datasets containing images with uniform illumination and
3.4 Median filtering                                                       background. The facial image can be preprocessed successfully
                                                                           in most cases, no matter whether face patterns are in different
         The intensity in the eye region and other facial                  scale, expression, and illumination conditions. Typical results
features is dark in a grey-level facial image. The image has               of preprocessing with the proposed approach are shown in
been enhanced through modified histogram equalization. In                  Fig.4. The input images vary greatly in background, scale,
image processing it is usually necessary to perform a high                 expression and illumination, the images also including partial
degree of noise reduction in an image before performing                    face occlusions and glasses wearing.
higher-level processing steps, such as edge detection. The
median filter is a non-linear digital filtering technique, often           4.1 Method Tested
used to remove noise from images or other signals.                                    No enhancement (NE). For this test we only
                                                                           geometrically normalized the images (actually, images were
         Median filtering is a common step in image
                                                                           geometrically normalized in all subsequent tests as well). No
processing. It is particularly useful to reduce speckle noise and
                                                                           filtering or histogram equalization is used.
salt and pepper noise. Its edge-preserving nature makes it
useful in cases where edge blurring is undesirable.
                                                                           Standard histogram equalization (HE): Images were
                                                                           geometrically     normalized     and     a    standard      histogram
3.5 Thresholding
                                                                           equalization (HE) technique was employed. HE enhances the
                                                                           contrast of images by transforming the values in an intensity
         After median filtering threshold is set to 128, so that
                                                                           image, so that the histogram of the output image is
only dark pixels remain, including eye pair structure. Then, a
                                                                           approximately uniformly distributed on pixel intensities of 0 to
binary image is obtained, which obviously contains the facial
structure. Taking into account that the nonface area can
influence the speed and the results of template matching, the
                                                                           HE vertical (HEV): Histogram equalization filtering of two
oversize black area, which is useless in the binary image, is
                                                                           sub images are obtained by vertically dividing the input image

                                                                                                       ISSN 1947-5500
                                                                                                              (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                              Vol. 8, No. 2, May 2010

into two halves prior to filtering and then filtering each of
them. The resulting image is obtained by concatenating the two                                                                 The fig 4 shows the implementation results of image preprocessing.

filtered halves.                                                                                                             The fig 4.a and fig 4.b are the results of enhanced image and binary

HE horizontal (HEH): The same procedure as in HEV is used                                                                    image. By looking at the extremely low recognition rate on NE images
                                                                                                                             just 4.15%, the proposed method is better. In our experiment the
with the exception of an image being horizontally divided.
                                                                                                                             standard preprocessing HE which yielded only 48.20%. HEV and
                                                                                                                             HEMOD give significant improvement with 60% and 60.20%
HE modified (HEMOD): Method proposed in Section 3.3,
                                                                                                                             respectively. Therefore, we can see clearly that our proposed method
consisting in combining results from HEV and HEH                                                                             is superior to all other methods and recognition rate is 12% is higher
                                                                                                                             than the standard HE. The superiority of the proposed method is
                Table 1, Results of applying all the techniques on                                                           further confirmed in Fig. 3 where the cumulative match score curve
video images The numbers in the table represent rank 1                                                                       for the standard method and proposed method could seen.

recognition rate (RR) in percentages of correctly recognized
images over the whole probe set.

      Method                        NE                  HE                HEV           HEH              HEMOD
      RR %                          4.15                48.20             60            58.30            60.20                        (a)                    (b)                           (c)

                Table 1: Recognition rate in percentages
  4.2 Results
  The fig. 2 shows the proposed preprocessing method gives
better results for finding the correct eye than other method
since the recognition rate of the eye here is 60.2%.                                                                                  (a)                    (b)                           (c)

                                               Ey e R e c ogn i t i on R a t e                                               Fig. 4: An example of preprocessing (a) Original Image (b)

                                                                                                                             Enhanced image (c) Binary image
                       60                                            60          58.3         60.2
                       50                           48.2
                       30                                                                                                                              5. Conclusions
                               NE              HE              HEV         HEH          HEMOD

                                                           M e t hods                                                                 We     introduced     a      simple    image-preprocessing
                                                                                                                             algorithm for compensation of illumination variations in
                            Fig. 2 Comparison of various methods                                                             images. The algorithm enhances the contrast of images by
                                                                                                                             transforming the values in an intensity image so that the
                       90                                                                                                    histogram of the output image is approximately uniformly
                                                                                                                             distributed on pixel intensities. The algorithm delivers large
         e o nt o ae
        Rc g i i nRt

                       65                                                                                                    performance improvements for standard face recognition
                                                                                        HEM OD                               algorithms. Experiments demonstrated the robustness of the
                               1           2        3      4         5     6       7      8          9   10                  method with several images captured from web camera.

  Fig. 3 Cumulative Match score curves for HE and proposed
                                                                                                                                                          ISSN 1947-5500
                                                                         (IJCSIS) International Journal of Computer Science and Information Security,
                                                                         Vol. 8, No. 2, May 2010


1.   Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The FERET
     evaluation methodology for Face                      recognition                     Mr. M. Moorthi received MCA degree from Bharathiar
     algorithms. IEEE PAMI 22 (2000) 1090-1104                                            University,   Coimbatore,      and      M.Phil.,      Degree       from
2.   Blackburn,       D.,     Bone,    M.,         Philips,       P.:     Facial          Manonmaniam Sundaranar University and doing PhD Degree
     recognition vendor test                2000: evaluation              report          in R&D centre, Bharathiar University, Coimbatore, TN, India
     (2000)                                                                               . He is currently the Lecturer (SG) in Kongu Arts and Science
3.   Gross,    R.,    Shi,    J.,    Cohn,        J.:     Quo     vadis     face          College, Erode, TN, India. He has 11 years of teaching and
     recognition?     In:    Third    Workshop             on       Empirical             6 years of research experience. He has guided nine M.Phil
     Evaluation Methods in Computer Vision. (2001)                                        students in the area of Computer Science. He has presented
4.   Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.:                                     papers in National and International Conference and has
     Eigen faces vs. Fisherfaces: Recognition using class                                 published an article in National Journal. He is a member of
     specific linear projection. IEEE PAMI 19                       (1997)                ISCA and working as Associate Editor in Canadian Research
     711-720                                                                              & Development Center of Science and Cultures – Advances in
5.   Belhumeur, P., Kriegman, D.: What is the set of images                               Natural Science and Management Engineering – ISSN 1913-
     of   an    object       under    all     possible                   lighting         0341.His interests and expertise are in the area of Image
     conditions. Int. J. of Computer Vision 28 (1998) 245-260                             Processing, Data Mining, Multimedia, Computer Graphics and
6.   Georghiades, A., Kriegman, D., Belhumeur, P.: From                                   Networks.
     few to many: Generative models for                   recognition under               E-mail ID:
     variable pose and         illumination. IEEE PAMI (2001)
7.   Riklin-Raviv, T., Shashua, A.: The Quotient                    image:
     class-based re-rendering and recognition                       with
     varying illumination conditions. In: IEEE PAMI. (2001)
8.   Georghiades,      A.,     Kriegman,           D.,      Belhumeur,        P.:
     Illumination cones for recognition                      under variable               Dr Arthanari holds a Ph.D in Mathematics from Madras
     lighting: Faces. In: Proc. IEEE Conf. on CVPR. (1998)                                University as well as Masters degree in Computer Science from
9.   Blanz,     V.,      Romdhani,          S.,         Vetter,    T.:      Face          BITS, Pilani. He holds patent issued by the Govt. of India for
     identification         across      different               poses        and          the invention in the field of Computer Science. He has directed
     illumination with a 3D morphable model. In: IEEE Conf.                               teams of Ph.D researchers and industry experts for developing
     on Automatic Face and Gesture Recognition. (2002)                                    patentable products. He teaches strategy, project management,
10. Horn, B.: Robot Vision. MIT Press (1986)                                              creative problem solving, innovation and integrated new
11. Stockam, T.: Image processing in the context of a                     visual          product development for last 35 years.
     model. Proceedings of the IEEE 60                    (1972) 828-842
12 A. V. Oppenheim, R. W. Schafer, and T. G. S. Jr,
     “Nonlinear filtering of multiplied and convolved signals”
     IEEE Proc., vol. 56, no. 8,            pp. 1264–1291, 1968.

                                                                                                                      ISSN 1947-5500
                                                          (IJCSIS) International Journal of Computer Science and Information Security,
                                                          Vol. 8, No. 2, May 2010

Mr. Sivakumar M ( has 10+ years of
experience   in   the   software   industry   including   Oracle
Corporation. He received his Bachelor degree in Physics and
Masters in Computer Applications from the Bharathiar
University, India. He holds patent for the invention in
embedded technology. He is technically certified by various
professional bodies like ITIL, IBM Rational Clearcase
Administrator, OCP - Oracle Certified Professional 10g and

                                                                                                     ISSN 1947-5500