# Face detection and face recognition

Document Sample

Face detection

KH Wong

face interface v.1_2c   1
Introduction

   Face interface
Face database
   Face detection
   Face recognition
Output:
Mr.Chan
Face detection                 Face recognition
Prof..Cheng

face interface v.1_2c                            2
Face detection [1]

   To detect faces in an image (Not recognize it yet)
   Challenges
   A picture has 0,1 or many faces
   Faces are not the same: with spectacles, mustache etc
   Sizes of faces vary
   Available in most digital cameras nowadays
   The simple method
   Slide a window across the window and detect faces
 Too slow, pictures have too many pixels
(1280x1024=1.3M pixels)

face interface v.1_2c              3
Evaluation of face detection
   Detection rate
 Give positive results in locations where faces exist

 Should be high > 95%

   False positive rate
 The detector output is positive but it is false (there is actually no
face).Definition of False positive: A result that is erroneously positive when a situation is normal.
An example of a false positive: a particular test designed to detect cancer of the toenail is positive
but the person does not have toenail cancer.
(http://www.medterms.com/script/main/art.asp?articlekey=3377)
   Should be low <10-6
   A good system has
 High detection rate

 Low false positive rate

face interface v.1_2c   False positive result                     4
Exercise

   What are the detection rate and false
detection rate here?
9 faces in the picture, 8
are correctly detected.
   Answer
   detection rate=(8/9)*100%
   false detection rate=(1/9)*100%
1 window reported
to have face is in
fact not a face         False positive result

face interface v.1_2c                           5
The Viola and Jones method [1]

   The most famous method
   Training may need weeks
   Recognition is very fast, e.g. real-time for
digital cameras.
   Techniques
   Integral image for feature extraction
   Ada-Boost for feature selection
   Attentional cascade for fast rejection of non-face
sub-windows

face interface v.1_2c             6
Image Features ref[3]

“Rectangle filters”

Rectangle_Feature_value f=
∑ (pixels in white area) –
∑ (pixels in shaded area)

face interface v.1_2c   7
Exercise

   Find the                                      1      2   3   3
Rectangle_Feature_val
ue (f) of the box
3      0   1   3
enclosed by the dotted
line
   Rectangle_Feature_value f=
5      8   7   1
   ∑ (pixels in white area) –
∑ (pixels in shaded area)
   f=(8+7)-(0+1)
   =15-1= 14                                     0      2   3   6

face interface v.1_2c               8
Example: A simple face detection method
using one feature
Rectangle_Feature_value f
f= ∑(pixels in white area) – ∑ (pixels in shaded area)

If (f) is large it is face ,i.e.
if (f)>threshold, then
 face                                                       Result
Else
 non-face

This is a face:T he eye-part is dark,
the nose-part is bright
So f is large, hence it is face

This is not a face.             face interface v.1_2c                                9
Because f is small
How to find features faster
Integral images fast calculation method [Lazebnik09 ]

   The integral image = sum
of all pixel values above
(x,y)
and to the left of (x,y)
   Can be found very quickly

face interface v.1_2c            10
Computing the integral image [Lazebnik09 ]

face interface v.1_2c     11
Computing the integral image [Lazebnik09 ]

ii(x, y-1)
s(x-1, y)

i(x, y)

   Cumulative row sum: s(x, y) = s(x–1, y) + i(x, y)
   Integral image: ii(x, y) = ii(x, y−1) + s(x, y)
   MATLAB: ii = cumsum(cumsum(double(i)), 2);

face interface v.1_2c             12
Calculate sum within a rectangle
   A,B,C,D are the values of the
integral images at the corners of
the rectangle R.                                D                      B
   The sum of image values inside R
is:                                                 Rectangle
Area_R = A – B – C + D                              R
Area= Area_R
   If A,B,C,D are found , only 3                                      A
additions are needed to find                    C
Area_R

face interface v.1_2c                              13
Why do we need to find pixel sum of rectangles?
Answer: We want to get face features

   You may consider these                   
features as face features
   Two eyes= (Area_A-Area_B)
A
B
   Nose =(Area_C+Area_E-Area_D)                        C
D
   Mouth =(Area_F+Area_H-
Area_G)
E
F
G
   They can be different sizes,                                H
polarity and aspect ratios

face interface v.1_2c           14
Face feature and example                     Pixel values inside
The areas
10     20      4
Shaded area
7      45     7
-1
Integral     216 102 78
White area
+2
Image       129 210 111
F=Feat_val =
pixel sum in shared area - pixel sum in white
area
Example
•Pixel sum in white area=
216+102+78+129+210+111=846

•Pixel sum in shared area=
10+20+4+7+45+7=93

Feat_val=F=846-93
A face         If F>threshold,
feature=+1
Else
feature=-1 End if;
We can choose threshold =768 , so feature is
15
face
+1. interface v.1_2c
Exercise 1?
Definition: Area at X =pixel sum of the area from top-left corner to X=
Area_X                         Top-left corner
    Find the feature output of this
image.                                      1              2   3   3
    Area_D=1
    Area_B=1+2+3=6
    Area_C =1+3=4                                          D           B
    Area_A=1+2+3+3+4+6=19                       3              4   6   3
    Area_E=? 1+3+5=9
    Area_F=? 1+2+3+3+4+6+5+2+4=30
    Pixel sum of the area inside the box                   C           A
ABDC=
    Area_A - Area_B - Area_C +Area_D            5              2   4   1
=? 19-6-4+1=10
    Pixel sum of the area inside the box
EFAC=                                                  E           F
    ?Area_F-Area_A-area_E+Area_C=
    30-19-9+4=6                                 0              2   3   6
    Feature of EFBD=
    (white area-shaded area)=?

face interface v.1_2c                   16
4 basic types of features
for white_area-gray_area
   Type) Rows x columns                Each basic type can
   Type 1) 1x2                          have difference sizes
and aspect ratios.
   Type 2) 2x1

   Type 3) 1x3

   Type 4) 3x1

   Type 5) 2x2

face interface v.1_2c                      17
Feature selection [Lazebnik09 ]
Some examples and their types
   For a 24x24 detection           Fill in the types for the 2nd, 3rd rows
2        3   5   1   4
region, the number of
possible rectangle
features is ~160,000!
   Exercise
   Name the types of the
feature (type 1,2,3,4,5)
Type
of the features in the                            1)

left figure                                       2)

3)

4)

face interface v.1_2c       5)                   18
Features in a 24x24 window [stackoverflow ]
#include <stdio.h>
int main()
{
int i, x, y, sizeX, sizeY, width, height,
    Why there are                                    count, c;

~160,000 features in a                               /* All five shape types */
const int features = 5;
24x24 window?                                        const int feature[][2] = {{2,1}, {1,2},
{3,1}, {1,3}, {2,2}};

    features=162,336                                     const int frameSize = 24;

count = 0;
/* Each shape */
for (i = 0; i < features; i++) {
sizeX = feature[i][0];
sizeY = feature[i][1];
printf("%dx%d shapes:\n", sizeX, sizeY);

/* each size (multiples of basic shapes)
*/
for (width = sizeX; width <= frameSize;
width+=sizeX) {
for (height = sizeY; height <=
frameSize; height+=sizeY) {
printf("\tsize: %dx%d => ",
width, height);

face interface v.1_2c
http://stackoverflow.com/questions/1707620/viola-jones-face-detection-claims-180k-features        19
Matlab [AdaBoost & Its Applications by [Yu tm]
   temp=0; %------matlab program to find number of features in Viola-Jones face detection cascaded Adaboost algorithm------------
   %%%% 2x1 shape : (2 rows one column) , 2 types
   width=36
   %width=24

   for w=1:width/2
      for h=1:width
       temp=(width-2*w+1)*(width-h+1)+temp;
      end
   end
   N_Type1=temp
   N_Type2=N_Type1 %same because the window is a square


   %3x1 shape (3 rows one column)
   temp=0; %--------------------------------------------------------------------
   for w=1:width/3
      for h=1:width
       temp=(width-3*w+1)*(width-h+1)+temp;
      end
   end
   N_Type3=temp
   N_Type4=N_type3 %same because the window is a square


   %2x2 shape (2 rows and 2 columns)
   temp=0; %--------------------------------------------------------------------
   for w=1:width/2
      for h=1:width/2
       temp=(width-2*w+1)*(width-2*h+1)+temp;
      end
   end
   N_Type5=temp
   'total'
   N_ALL=N_Type1+N_Type2+N_Type3+N_Type4+N_Type5
   %Result= 162336 if width =24
   %Result= : 704004 if width =36                                     face interface   v.1_2c                                       20
Exercise 2?

   Still keeping the 5 basic features types
(1,2,3,4,5)
   Find the number of features for a resolution of 36
x36 windows
   Answer: 704004, explain your answer.

face interface v.1_2c          21
The detection challenge
(x,y) 24x24Sub-window
   Use 24x24 base window                     
X-axis 1280
(1,1)
   For y=1;y<=1024;y++
For x=1;x<=1280;x++{
   Set (x,y) = the left top corner
of the 24x24 sub-window                        Y-
axis
   For the 24x24 sub-window,
extract 162,336 features and
see they combine to form a                     1024
face or not.
   }
   Conclusion : too slow

face interface v.1_2c                                   22
Solution to make it efficient

   The whole 162,336 feature set is too large
   Solution: select good features to make it more
efficient
   Use: “Boosting”
   Boosting
   Combine many small weak classifiers to become
a strong classifier.
   Training is needed

face interface v.1_2c          23
Boosting for face detection
   Define weak learners based on rectangle
features
value of rectangle feature

1 if pt f t ( x )  ptt
ht ( x )  
window      0 otherwise                             threshold
Pt=
polarity{+1,-1}

face interface v.1_2c                      24
Face detection using Adaboost
   AdaBoost training
   E.g. Collect 5000 faces, and 9400 non-faces.
Different scales.
   Use AdaBoost for training to build a strong
classifier.
   Pick suitable features of different scales and
positions, pick the best few. (Take months to do ,
details is in [Viola 2004] paper)
   Testing
   Scan through the image, pick a window and
rescale it to 24x24,
   Pass it to the strong classifier for detection.
   Report face, if the output is positive
face interface v.1_2c              25
Boosting for face detection [viola2004]
   In the paper it shows that the following two features (obtained
after training) in cascaded picked by AdaBoost have 100%
detection rate and 50% false positive rate
   But 50% false positive rate is not good enough       I.e.
H(face)=
   Approach [viola2004] :Attentional cascade            Sign{α h (image)
1 1
+α2h2(image)}

Pick a window in the                                                  H(face)=+1 face
image and rescale it to                                               H(face)=-1non-face
24x24 as “image”                                          h2(image)
h1(image)
type3
type2

face interface v.1_2c                               26
Boosting for face detection
    A 200-feature classifier can yield 95% detection rate and a false positive rate
of 1 in 14084 (Still not god enough)
    Recall: False positive rate
 The detector output is positive but it is false (there is actually no face).

Definition of False positive: A result that is erroneously positive when a
situation is normal. An example of a false positive: a particular test
designed to detect cancer of the toenail is positive but the person does not
have toenail cancer.
(http://www.medterms.com/script/main/art.asp?articlekey=3377)

Correct
Detection       Still not
rate
good
enough!
X10-3
False positive rate
face interface v.1_2c                       27
To improve false positive rate:
Attentional cascade
   Cascade of many AdaBoost strong classifiers
   Begin with with simple classifiers to reject many
negative sub-windows
   Many non-faces are rejected at the first few
stages.
   Hence the system is efficient enough for real time
processing.

Input image
True                 True
Adaboost             Adaboost                 Adaboost True    Face
Classifier1          Classifier2              Classifier3      found
False               False                        False
Non-face           Non-face                     Non-face
face interface v.1_2c                   28
An example
   More features for later stages in the cascade
[viola2004]

type2    type3

2 features        10 features                25 features        50 features…

Input image
True                 True
Adaboost             Adaboost                 Adaboost True              Face
Classifier1          Classifier2              Classifier3                found
False               False                        False
Non-face           Non-face                     Non-face
face interface v.1_2c                               29
Attentional cascade
   Chain classifiers that are                                                    Receiver operating
progressively more complex                                                      characteristic

and have lower false positive                                                      0
% False Pos
50
rates:                                                                        vs false neg determined by

100
% Detection

0
False positive rate
Input image
True                 True
Adaboost             Adaboost                 Adaboost True                                      Face
Classifier1          Classifier2              Classifier3                                        found
False               False                        False
Non-face           Non-face                     Non-face
face interface v.1_2c                                                     30
Attentional cascade [viola2004]
   Detection rate for each stage is 0.99 , for 10
stages,
     overall detection rate is 0.9910 ≈ 0.9

   False positive rate at each stage is 0.3, for 10
stages
     false positive rate =0.310 ≈ 6×10-6)

Input image
True                 True
Adaboost             Adaboost                 Adaboost True    Face
Classifier1          Classifier2              Classifier3      found
False               False                        False
Non-face           Non-face                     Non-face
face interface v.1_2c                   31
Detection process in practice [smyth2007]

   Use 24x24 sub-window
   Scaling
   scale the detection (not the input image)
   Features evaluated at scales by factors of 1.25 at
each level
   Location : move detector around the image (1
pixel increments)
   Final detections
   A real face may result in multiple nearby
detections (merge them to become the final result)

face interface v.1_2c             32
References
   [viola2004] Paul A. Viola, Michael J. Jones: Robust Real-Time Face Detection.
International Journal of Computer Vision 57(2): 137-154 (2004) (PDF:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.137.4879&rep=rep1&type=pd
f)
   [viola2001] Paul A. Viola, Michael J. Jones, Rapid object detection using a boosted
cascade of simple features CVPR 2001 (PDF: http://research.microsoft.com/en-
us/um/people/viola/Pubs/Detect/violaJones_CVPR2001.pdf)
   [Lazebnik09 ] www.cs.unc.edu/~lazebnik/spring09/lec23_face_detection.ppt
   [stackoverflow] http://stackoverflow.com/questions/1707620/viola-jones-face-detection-
claims-180k-features
   [Jensen 2008 ] Ole Helvig Jensen," Implementing the Viola-Jones Face Detection
Algorithm "Kongens Lyngby 2008 “, IMM-M.Sc.-2008-93,Technical University of
Denmark Informatics and Mathematical Modeling
   [smyth2007] Face detection using the viola Jones method ppt, UL Irvine (lecture notes
of CS175 Fall 2007)
   [yu tm ]http://aimm02.cse.ttu.edu.tw/class_2009_1/PR/Lecture%207/Adaboost.ppt
   [stackoverflow] http://stackoverflow.com/questions/1707620/viola-jones-face-detection-
claims-180k-features

face interface v.1_2c                               33
Appendix

face interface v.1_2c   34
Training

The face Adaboost detection system

face interface v.1_2c     35
Given : (x1,y1 ),..(xn ,yn ),where xi  X , Y  {1,1} for negative and positive examples
• Initialize weights:
– w1,i  1/ 2 M, M  number of positive example
– w1,i  1/ 2 L, L  number of positive example
Adaboost face detection
For t  1,2 ,...,T                                                          Training algorithm [Jensen

{ Step1: Normalize weights wt ,i 
wt , j
j n
2008 ]
w
j 1
t, j

Step2 : Select the weak classfier with smallest weighted error
n
find  t  min f , p ,  wi h( xi , f , p,  )  yi
i

Step3 : Define ht ( x )  h ( x, f t , pt ,  t ), where f t , pt ,  t are minimizer of εt at stage t
Step4 : update weigths
update the weights: wt 1,i  wt ,i  1ei
here ei  0 if example xi is classfied correctlyand ei  1 otherwise,
w
εt
and β 
1  εt
}
        T
1 T
1 if   t ht ( x )    t                     1
The final strong classifier is : C ( x )                          2 t 1 , where t  log
t 1
0 face interface v.1_2c                         t
                     otherwise                                   36
Inside the main loop for training
For t=1,…T
   Step1
   Init all weights
   Same weights all for samples
at t=1

face interface v.1_2c   37
Inside the main loop for training
For t=1,…T
-assume at stage t
   Step2: select the best weak classifier (weak
n
learner)       find  t  min f , p ,  wi h( xi , f , p,  )  yi
i
   For all f (1,2,3,… 162,336 feature set)
     For all p (p=+1 or -1)
   For different , ( as low as possible to produce good
result)                      Mistakenly classified
   {                    n
 f , p ,   wi h( xi , f , p,  )  yi
i
   }

{ f , p,  }best _ weak _ classifier  arguments (min{  f , p , })

face interface v.1_2c                 38
Step2 : more explanation
-assume at stage t
   Test every feature in the feature set {1,2,3,…
162,336 feature set}
   Test different polairty{+1,-1}: dark/white reversed.
   Try different  (for simplicity start from 0.4), make it
lower to see if performance (recognition, false-
positive rates are improved.
   Output= {ft (type of feature), pt (polarity), t
(threshold)} which give the minimum error ɛt
   {ft , pt , t}= (minimizer of ɛt) at stage t

face interface v.1_2c               39
Inside the main loop for training
For t=1,…T
-assume at stage t

   Step3

ht ( x)  h( x, f t , pt ,t )
f t , pt ,t are minimizer of εt at stage t

face interface v.1_2c         40
Inside the main loop for training
For t=1,…T
-assume at stage t
   step4
update the weights:
wt 1,i  wt ,i  1ei
where ei  0 if example xi is classfied correctlyand ei  1 ortherwsie,
εt
and β 
1  εt

face interface v.1_2c                 41
Inside the main loop for training
For t=1,…T
-assume at stage t
   step5
           T
1 T
1 if
C ( x)           t ht ( x)  2  t
t 1             t 1
0
                    otherwise
1
where t  log
t

face interface v.1_2c     42

DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 36 posted: 1/31/2012 language: Latin pages: 42
How are you planning on using Docstoc?