PCA-Based face recognition system

							                  PCA-Based face recognition system
                                       by Giuseppe Cotugno

Introduction
Face recognition is an interesting topic in computer vision that is arising much attention for its
possible applications in many fields of IT, security and entertainment. While is still quite
challenging to master this technique efficiently, many progresses have been done from the past. In
this work we will focus mainly on a well-know and universally recognized method for face
identification, the principal component analysis. In the next paragraph we will describe our system's
structure, then we will illustrate some results of our software.

Some basic assumptions
In order to let this software work correctly, some assumptions have to be made. Is known that PCA
is sensible to illumination variance, alignment, facial expressions and head pose. The system can
circumvent illumination variances and alignment but is sensible to different head poses. We assume
that every person, in our database of known subjects, have a mug-shot picture and is only matched
with other mug-shot pictures. Moreover, we assume that the picture encloses the whole subject's
face, and that the scale difference among the faces is not too wide. Finally, we assume that any
subject doesn't show extremely wired facial expressions.

Classification Step
Briefly speaking, the system performs some image optimization and then execute the PCA on every
subject of the database. In addition, a multi-resolution corner detection is performed, in order to
improve the effectiveness of the PCA recognition.
The software creates a database of rendered
images, starting from a collection of JPEG files,
and then it matches user selected subjects among
the stored ones. Finally it chooses N candidates
sorted by likeness. The figures illustrate the
schemes.
At first, an histogram equalization is performed,
in order to neutralize the negative effects of
illumination. Actually, PCA is sensible to changes
in illumination, so equalizing any image to a
constant histogram reduces this sensibility
significantly.
Next step is face detection. The software search in
the image, in order to find the subject's face, and
then it extrapolates the result from the figure. If
no faces are found,the application proceeds to the
next image. The motivation to this step is the
reduction of the search space for the following
PCA, in order to optimize the performances and
reduce the errors. The face detection is performed
by a third party software[1], embedded in our
software, that extrapolates the face starting from
the detection's results. Unfortunately, the third
party software code is proprietary so we were unable to do any optimization.
Sometimes face detection produces images of different size, so a padding step is required in order to
standardize the dimensions and be able to perform the PCA. While the database is being
constructed, every image is adapted to the biggest dimensions found. During the recognition step, if
the test image is bigger than the database ones, the test image is reduced by equally cutting among
the borders. However, this eventuality is quite rare.
A gaussian filter is applied to any image in order to blur the noise. The images used were all
suffering some random noise, that was reducing the quality of identification, especially in the corner
detection step. Empirical trials had convinced us to implement the denoising step as a default step.
Actually, is quite rare to find images completely noise-free and a small amount of noise increases
the number of outliers found in the corner detection; blurring the images generally improve the
overall quality of identification in any case. The gaussian filtering was not performed before,
because it doesn't increase significantly the output quality of the previous step and apply filtering
now, means work directly with small faces instead of big portraits, so the time complexity
performances are improved.
Before apply the PCA, a multi-resolution corner detection is performed. The aiming of this step is
to offer to the system more informations about each subject, in order to identify the correct match
using more evidences. For each image, a gaussian pyramid is constructed, then for any level of the
pyramid is performed an harris corner detection using a third party software[2]. The detection will
produce many corner points, the bigger is the image, the more points will be found. We are interest
only in resolution-invariant corner point, to find them is executed an iterative selection among
pyramid levels. Given a level k of the pyramid with m points and a level k-1 of the pyramid with n
points, we know that the dimensions of the level k-1 are the half of the level k. The consequence is
that the total values available for the coordinates in level k-1 are half of the one available at level k
(if level k is 120x160, x coordinates can be found in a [1,160] space at level k, while at level k-1 x
can assume values included in [1,80] ). The intuition is to double the coordinates of the n points in
order to project them in the same domain of the m points. However the image dimensions cannot be
equally divided by 2 for every level. Sometimes the division returns a non-integer value so an
approximation have to be made. In this case, the result of the division is increased to the highest
integer. In the worst case, if the pyramid has L levels, we make L approximations for each
dimension when we shrink the image till the last level,
and 2L approximation when we double the last level
point's coordinate till the first level domain. A total
approximation of L2L is gained between the coordinates
at level L and the correspondent coordinates at level 1.
This approximation factor is unacceptable so we are
forced to iteratively match the n points of level k-1 only
with the m points of level k-1. This is performed by
doubling the coordinates of all the n points and finding
the closest neighbor in terms of euclidean distance.
Doing so, in the worst case the approximation level is
much reduced, because we match points only between
two adjacent levels. Please note that the histogram
equalization increases the number of outliers founded,
due to the increased contrast. The gaussian filtering
reduces the loss of precision and the face extrapolation
reduces significantly the space of search and so the
outliers number.
Finally, the PCA is performed. While database is being constructed, PCA is executed on every face
recovering eigenfaces and the mean face. At first, every image is vectorized and a face matrix is
constructed. From the face image, a mean face is computed and it is subtracted from the face
matrix, obtaining a centered face matrix. The transpose centered face matrix is multiplied with its
original version and finally its eigenvectors are recovered, using the build-in matlab function.
Eigfaces are computed by multiplying the founded eigenvectors with the centered face matrix.
Please note that the method used is not exactly the Turk and Pentland's because no threshold is
calculated after the eigenvectors. The classical PCA-based recognition uses the threshold to
discriminate among new subjects, known subjects and noise. Due the previous face detection step,
this discrimination became, in practice, superfluous. The code used is a third party one[3], that was
optimized and adopted to satisfy the needing of the application.
PCA is last step in the construction of the face database. It is stored in a 2x1 cell array containing
an header struct and a Nx1 face struct. The header contains common informations for the subjects,
while every element of the face struct contains the calculated data for each of the N subjects plus an
identity number. After the data structure is created, it is saved to a mat file so that, next time, the
program can optionally load it, instead of calculating everything from scratch.



Recognition step
The recognition step makes the same image pre-processing for a user selected image. The picture is
equalized, then the face is extrapolated and eventually padded. This time, if the program cannot find
any face, the recognition is aborted.
When image is ready, PCA recognition is performed selecting T faces that best match the subject.
We achieve this by projecting the vectorized and centered subject face in the face space and
selecting the T closest faces in term of euclidean distance. Thanks to the face detection step, we
circumvent the problem of understanding if a given image is a face or not, but we can't decide if a
given face doesn't exists in the database. Each of the
T candidates have a similitude ranking that is his
euclidean distance normalized on the smallest
euclidean distance of the T candidates.
Then, for each of the T candidates we evaluate the
similitude with the subject in terms of corner points.
The same subject should have similar corner points
in different pictures, so we compare the corner
points of the T candidates with the subject's corner
points. Again we use a multi-resolution approach
based on gaussian pyramid and harris corner
detection. At first, we calculate the resolution-
invariant corner points for the subject using the
same method adopted during the construction of the
face database. Obtained the corner points, we have
to match them among the corner points of every
candidate and then assign a score to each match.
Given a subject's point and a candidate's point
vector, we calculate the euclidean distance between
the subject point and every point in the vector. The
index of the smallest distance indicates the element
of the vector that matches this point. We repeat this
calculus for every subject's point and we account the
total distance between the two vector points (that is the sum of the partial distances for every point).
At the end, we will have a vector Tx1 where every element I indicates the euclidean distance
between the I candidate and the subject. Finally, the distance vector is normalized on the smallest
value. Please note that the candidates and the subject can have a different amount of points, so for
each extra point the candidate receive a penalty on its final distance.
Last step of recognition is calculating the total similitude between the candidates and the subject.
This is achieved by summing the result of the PCA and the resolution-invariant analysis, please note
that, due to the normalization every result counts the same for identification. At the end candidates
are sorted on a decreasing order, using bubble sort algorithm, and showed to the final user.

Testing
We have created two set of images starting from the Essex Face Database [4]: an easy set and a
difficult one. The easy set holds 10
subjects randomly selected from the
easiest group available at Essex Face
Database, while the difficult set
holds 16 subjects randomly selected
from the average group of the
original database. Every subject is
young and can be male or female,
black or white, with or without
glasses. The differences between the
two sets are mainly illumination,
and, sometimes, scale variance
within subjects of the same set. We
had evaluate empirically the optimal
parameters and we didn't change
them in different test. Parameters are
saved in the file options.mat and the
user can modify them easily using a
simple form.
The overall performances for the easy set were generally good, the unknown subject was
recognized most of times in the first four or the first six candidates. Recognition of subjects in the
difficult set was much more challenging, and the system actually recognizes only a small amount of
people.
Conclusions
Overally, even is the performances are not exciting, the system illustrates an interesting structure,
useful to circumvent the classical PCA limitations, while still doesn't beat the most tricky problems,
like head orientation. Of course the software can be improved, implementing a smarter histogram
equalization, that can adapt itself to different distributions of pixels in the image. Moreover, a skin
recognition can be performed, by manually extracting a skin sample from each candidate and then
executing a template matching technique based on convolution, using the skin texture as the
template to match. Finally, PCA can be extended, using a threshold to discriminate new subjects.
Unfortunately, these features could not be implemented due to a time constraint, but they will not
modify the main motivation of the recognition system, that is support PCA with pre-processing and
calculation of extra data of the subjects.

References


   1. Face Detection in Matlab, by Mikael Nilsson, available at
      http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?
      objectId=13701&objectType=file
   2. Harris Corner Detector, by Peter Kovesi, available at
      http://www.csse.uwa.edu.au/~pk/Research/MatlabFns/index.html
   3. PCA-based face recognition system, by Amir Hossein Omidvarnia, availabe at
      http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?
      objectId=17032&objectType=file
   4. Essex Face Database, by Libor Spacek, available at
      http://cswww.essex.ac.uk/mv/allfaces/index.html