Get documents about "Face detection and face recognition
Document Sample
Face detection
KH Wong
face interface v.1_2c 1
Introduction
Face interface
Face database
Face detection
Face recognition
Output:
Mr.Chan
Face detection Face recognition
Prof..Cheng
face interface v.1_2c 2
Face detection [1]
To detect faces in an image (Not recognize it yet)
Challenges
A picture has 0,1 or many faces
Faces are not the same: with spectacles, mustache etc
Sizes of faces vary
Available in most digital cameras nowadays
The simple method
Slide a window across the window and detect faces
Too slow, pictures have too many pixels
(1280x1024=1.3M pixels)
face interface v.1_2c 3
Evaluation of face detection
Detection rate
Give positive results in locations where faces exist
Should be high > 95%
False positive rate
The detector output is positive but it is false (there is actually no
face).Definition of False positive: A result that is erroneously positive when a situation is normal.
An example of a false positive: a particular test designed to detect cancer of the toenail is positive
but the person does not have toenail cancer.
(http://www.medterms.com/script/main/art.asp?articlekey=3377)
Should be low <10-6
A good system has
High detection rate
Low false positive rate
face interface v.1_2c False positive result 4
Exercise
What are the detection rate and false
detection rate here?
9 faces in the picture, 8
are correctly detected.
Answer
detection rate=(8/9)*100%
false detection rate=(1/9)*100%
1 window reported
to have face is in
fact not a face False positive result
face interface v.1_2c 5
The Viola and Jones method [1]
The most famous method
Training may need weeks
Recognition is very fast, e.g. real-time for
digital cameras.
Techniques
Integral image for feature extraction
Ada-Boost for feature selection
Attentional cascade for fast rejection of non-face
sub-windows
face interface v.1_2c 6
Image Features ref[3]
“Rectangle filters”
Rectangle_Feature_value f=
∑ (pixels in white area) –
∑ (pixels in shaded area)
face interface v.1_2c 7
Exercise
Find the 1 2 3 3
Rectangle_Feature_val
ue (f) of the box
3 0 1 3
enclosed by the dotted
line
Rectangle_Feature_value f=
5 8 7 1
∑ (pixels in white area) –
∑ (pixels in shaded area)
f=(8+7)-(0+1)
=15-1= 14 0 2 3 6
face interface v.1_2c 8
Example: A simple face detection method
using one feature
Rectangle_Feature_value f
f= ∑(pixels in white area) – ∑ (pixels in shaded area)
If (f) is large it is face ,i.e.
if (f)>threshold, then
face Result
Else
non-face
This is a face:T he eye-part is dark,
the nose-part is bright
So f is large, hence it is face
This is not a face. face interface v.1_2c 9
Because f is small
How to find features faster
Integral images fast calculation method [Lazebnik09 ]
The integral image = sum
of all pixel values above
(x,y)
and to the left of (x,y)
Can be found very quickly
face interface v.1_2c 10
Computing the integral image [Lazebnik09 ]
face interface v.1_2c 11
Computing the integral image [Lazebnik09 ]
ii(x, y-1)
s(x-1, y)
i(x, y)
Cumulative row sum: s(x, y) = s(x–1, y) + i(x, y)
Integral image: ii(x, y) = ii(x, y−1) + s(x, y)
MATLAB: ii = cumsum(cumsum(double(i)), 2);
face interface v.1_2c 12
Calculate sum within a rectangle
A,B,C,D are the values of the
integral images at the corners of
the rectangle R. D B
The sum of image values inside R
is: Rectangle
Area_R = A – B – C + D R
Area= Area_R
If A,B,C,D are found , only 3 A
additions are needed to find C
Area_R
face interface v.1_2c 13
Why do we need to find pixel sum of rectangles?
Answer: We want to get face features
You may consider these
features as face features
Two eyes= (Area_A-Area_B)
A
B
Nose =(Area_C+Area_E-Area_D) C
D
Mouth =(Area_F+Area_H-
Area_G)
E
F
G
They can be different sizes, H
polarity and aspect ratios
face interface v.1_2c 14
Face feature and example Pixel values inside
The areas
10 20 4
Shaded area
7 45 7
-1
Integral 216 102 78
White area
+2
Image 129 210 111
F=Feat_val =
pixel sum in shared area - pixel sum in white
area
Example
•Pixel sum in white area=
216+102+78+129+210+111=846
•Pixel sum in shared area=
10+20+4+7+45+7=93
Feat_val=F=846-93
A face If F>threshold,
feature=+1
Else
feature=-1 End if;
We can choose threshold =768 , so feature is
15
face
+1. interface v.1_2c
Exercise 1?
Definition: Area at X =pixel sum of the area from top-left corner to X=
Area_X Top-left corner
Find the feature output of this
image. 1 2 3 3
Area_D=1
Area_B=1+2+3=6
Area_C =1+3=4 D B
Area_A=1+2+3+3+4+6=19 3 4 6 3
Area_E=? 1+3+5=9
Area_F=? 1+2+3+3+4+6+5+2+4=30
Pixel sum of the area inside the box C A
ABDC=
Area_A - Area_B - Area_C +Area_D 5 2 4 1
=? 19-6-4+1=10
Pixel sum of the area inside the box
EFAC= E F
?Area_F-Area_A-area_E+Area_C=
30-19-9+4=6 0 2 3 6
Feature of EFBD=
(white area-shaded area)=?
face interface v.1_2c 16
4 basic types of features
for white_area-gray_area
Type) Rows x columns Each basic type can
Type 1) 1x2 have difference sizes
and aspect ratios.
Type 2) 2x1
Type 3) 1x3
Type 4) 3x1
Type 5) 2x2
face interface v.1_2c 17
Feature selection [Lazebnik09 ]
Some examples and their types
For a 24x24 detection Fill in the types for the 2nd, 3rd rows
2 3 5 1 4
region, the number of
possible rectangle
features is ~160,000!
Exercise
Name the types of the
feature (type 1,2,3,4,5)
Type
of the features in the 1)
left figure 2)
3)
4)
face interface v.1_2c 5) 18
Features in a 24x24 window [stackoverflow ]
#include <stdio.h>
int main()
{
int i, x, y, sizeX, sizeY, width, height,
Why there are count, c;
~160,000 features in a /* All five shape types */
const int features = 5;
24x24 window? const int feature[][2] = {{2,1}, {1,2},
{3,1}, {1,3}, {2,2}};
features=162,336 const int frameSize = 24;
count = 0;
/* Each shape */
for (i = 0; i < features; i++) {
sizeX = feature[i][0];
sizeY = feature[i][1];
printf("%dx%d shapes:\n", sizeX, sizeY);
/* each size (multiples of basic shapes)
*/
for (width = sizeX; width <= frameSize;
width+=sizeX) {
for (height = sizeY; height <=
frameSize; height+=sizeY) {
printf("\tsize: %dx%d => ",
width, height);
face interface v.1_2c
http://stackoverflow.com/questions/1707620/viola-jones-face-detection-claims-180k-features 19
Matlab [AdaBoost & Its Applications by [Yu tm]
temp=0; %------matlab program to find number of features in Viola-Jones face detection cascaded Adaboost algorithm------------
%%%% 2x1 shape : (2 rows one column) , 2 types
width=36
%width=24
for w=1:width/2
for h=1:width
temp=(width-2*w+1)*(width-h+1)+temp;
end
end
N_Type1=temp
N_Type2=N_Type1 %same because the window is a square
%3x1 shape (3 rows one column)
temp=0; %--------------------------------------------------------------------
for w=1:width/3
for h=1:width
temp=(width-3*w+1)*(width-h+1)+temp;
end
end
N_Type3=temp
N_Type4=N_type3 %same because the window is a square
%2x2 shape (2 rows and 2 columns)
temp=0; %--------------------------------------------------------------------
for w=1:width/2
for h=1:width/2
temp=(width-2*w+1)*(width-2*h+1)+temp;
end
end
N_Type5=temp
'total'
N_ALL=N_Type1+N_Type2+N_Type3+N_Type4+N_Type5
%Result= 162336 if width =24
%Result= : 704004 if width =36 face interface v.1_2c 20
Exercise 2?
Still keeping the 5 basic features types
(1,2,3,4,5)
Find the number of features for a resolution of 36
x36 windows
Answer: 704004, explain your answer.
face interface v.1_2c 21
The detection challenge
(x,y) 24x24Sub-window
Use 24x24 base window
X-axis 1280
(1,1)
For y=1;y<=1024;y++
For x=1;x<=1280;x++{
Set (x,y) = the left top corner
of the 24x24 sub-window Y-
axis
For the 24x24 sub-window,
extract 162,336 features and
see they combine to form a 1024
face or not.
}
Conclusion : too slow
face interface v.1_2c 22
Solution to make it efficient
The whole 162,336 feature set is too large
Solution: select good features to make it more
efficient
Use: “Boosting”
Boosting
Combine many small weak classifiers to become
a strong classifier.
Training is needed
face interface v.1_2c 23
Boosting for face detection
Define weak learners based on rectangle
features
value of rectangle feature
1 if pt f t ( x ) ptt
ht ( x )
window 0 otherwise threshold
Pt=
polarity{+1,-1}
face interface v.1_2c 24
Face detection using Adaboost
AdaBoost training
E.g. Collect 5000 faces, and 9400 non-faces.
Different scales.
Use AdaBoost for training to build a strong
classifier.
Pick suitable features of different scales and
positions, pick the best few. (Take months to do ,
details is in [Viola 2004] paper)
Testing
Scan through the image, pick a window and
rescale it to 24x24,
Pass it to the strong classifier for detection.
Report face, if the output is positive
face interface v.1_2c 25
Boosting for face detection [viola2004]
In the paper it shows that the following two features (obtained
after training) in cascaded picked by AdaBoost have 100%
detection rate and 50% false positive rate
But 50% false positive rate is not good enough I.e.
H(face)=
Approach [viola2004] :Attentional cascade Sign{α h (image)
1 1
+α2h2(image)}
Pick a window in the H(face)=+1 face
image and rescale it to H(face)=-1non-face
24x24 as “image” h2(image)
h1(image)
type3
type2
face interface v.1_2c 26
Boosting for face detection
A 200-feature classifier can yield 95% detection rate and a false positive rate
of 1 in 14084 (Still not god enough)
Recall: False positive rate
The detector output is positive but it is false (there is actually no face).
Definition of False positive: A result that is erroneously positive when a
situation is normal. An example of a false positive: a particular test
designed to detect cancer of the toenail is positive but the person does not
have toenail cancer.
(http://www.medterms.com/script/main/art.asp?articlekey=3377)
Correct
Detection Still not
rate
good
enough!
X10-3
False positive rate
face interface v.1_2c 27
To improve false positive rate:
Attentional cascade
Cascade of many AdaBoost strong classifiers
Begin with with simple classifiers to reject many
negative sub-windows
Many non-faces are rejected at the first few
stages.
Hence the system is efficient enough for real time
processing.
Input image
True True
Adaboost Adaboost Adaboost True Face
Classifier1 Classifier2 Classifier3 found
False False False
Non-face Non-face Non-face
face interface v.1_2c 28
An example
More features for later stages in the cascade
[viola2004]
type2 type3
2 features 10 features 25 features 50 features…
Input image
True True
Adaboost Adaboost Adaboost True Face
Classifier1 Classifier2 Classifier3 found
False False False
Non-face Non-face Non-face
face interface v.1_2c 29
Attentional cascade
Chain classifiers that are Receiver operating
progressively more complex characteristic
and have lower false positive 0
% False Pos
50
rates: vs false neg determined by
100
% Detection
0
False positive rate
Input image
True True
Adaboost Adaboost Adaboost True Face
Classifier1 Classifier2 Classifier3 found
False False False
Non-face Non-face Non-face
face interface v.1_2c 30
Attentional cascade [viola2004]
Detection rate for each stage is 0.99 , for 10
stages,
overall detection rate is 0.9910 ≈ 0.9
False positive rate at each stage is 0.3, for 10
stages
false positive rate =0.310 ≈ 6×10-6)
Input image
True True
Adaboost Adaboost Adaboost True Face
Classifier1 Classifier2 Classifier3 found
False False False
Non-face Non-face Non-face
face interface v.1_2c 31
Detection process in practice [smyth2007]
Use 24x24 sub-window
Scaling
scale the detection (not the input image)
Features evaluated at scales by factors of 1.25 at
each level
Location : move detector around the image (1
pixel increments)
Final detections
A real face may result in multiple nearby
detections (merge them to become the final result)
face interface v.1_2c 32
References
[viola2004] Paul A. Viola, Michael J. Jones: Robust Real-Time Face Detection.
International Journal of Computer Vision 57(2): 137-154 (2004) (PDF:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.137.4879&rep=rep1&type=pd
f)
[viola2001] Paul A. Viola, Michael J. Jones, Rapid object detection using a boosted
cascade of simple features CVPR 2001 (PDF: http://research.microsoft.com/en-
us/um/people/viola/Pubs/Detect/violaJones_CVPR2001.pdf)
[Lazebnik09 ] www.cs.unc.edu/~lazebnik/spring09/lec23_face_detection.ppt
[stackoverflow] http://stackoverflow.com/questions/1707620/viola-jones-face-detection-
claims-180k-features
[Jensen 2008 ] Ole Helvig Jensen," Implementing the Viola-Jones Face Detection
Algorithm "Kongens Lyngby 2008 “, IMM-M.Sc.-2008-93,Technical University of
Denmark Informatics and Mathematical Modeling
[smyth2007] Face detection using the viola Jones method ppt, UL Irvine (lecture notes
of CS175 Fall 2007)
[yu tm ]http://aimm02.cse.ttu.edu.tw/class_2009_1/PR/Lecture%207/Adaboost.ppt
[stackoverflow] http://stackoverflow.com/questions/1707620/viola-jones-face-detection-
claims-180k-features
face interface v.1_2c 33
Appendix
face interface v.1_2c 34
Training
The face Adaboost detection system
face interface v.1_2c 35
Given : (x1,y1 ),..(xn ,yn ),where xi X , Y {1,1} for negative and positive examples
• Initialize weights:
– w1,i 1/ 2 M, M number of positive example
– w1,i 1/ 2 L, L number of positive example
Adaboost face detection
For t 1,2 ,...,T Training algorithm [Jensen
{ Step1: Normalize weights wt ,i
wt , j
j n
2008 ]
w
j 1
t, j
Step2 : Select the weak classfier with smallest weighted error
n
find t min f , p , wi h( xi , f , p, ) yi
i
Step3 : Define ht ( x ) h ( x, f t , pt , t ), where f t , pt , t are minimizer of εt at stage t
Step4 : update weigths
update the weights: wt 1,i wt ,i 1ei
here ei 0 if example xi is classfied correctlyand ei 1 otherwise,
w
εt
and β
1 εt
}
T
1 T
1 if t ht ( x ) t 1
The final strong classifier is : C ( x ) 2 t 1 , where t log
t 1
0 face interface v.1_2c t
otherwise 36
Inside the main loop for training
For t=1,…T
Step1
Init all weights
Same weights all for samples
at t=1
face interface v.1_2c 37
Inside the main loop for training
For t=1,…T
-assume at stage t
Step2: select the best weak classifier (weak
n
learner) find t min f , p , wi h( xi , f , p, ) yi
i
For all f (1,2,3,… 162,336 feature set)
For all p (p=+1 or -1)
For different , ( as low as possible to produce good
result) Mistakenly classified
{ n
f , p , wi h( xi , f , p, ) yi
i
}
{ f , p, }best _ weak _ classifier arguments (min{ f , p , })
face interface v.1_2c 38
Step2 : more explanation
-assume at stage t
Test every feature in the feature set {1,2,3,…
162,336 feature set}
Test different polairty{+1,-1}: dark/white reversed.
Try different (for simplicity start from 0.4), make it
lower to see if performance (recognition, false-
positive rates are improved.
Output= {ft (type of feature), pt (polarity), t
(threshold)} which give the minimum error ɛt
{ft , pt , t}= (minimizer of ɛt) at stage t
face interface v.1_2c 39
Inside the main loop for training
For t=1,…T
-assume at stage t
Step3
ht ( x) h( x, f t , pt ,t )
f t , pt ,t are minimizer of εt at stage t
face interface v.1_2c 40
Inside the main loop for training
For t=1,…T
-assume at stage t
step4
update the weights:
wt 1,i wt ,i 1ei
where ei 0 if example xi is classfied correctlyand ei 1 ortherwsie,
εt
and β
1 εt
face interface v.1_2c 41
Inside the main loop for training
For t=1,…T
-assume at stage t
step5
T
1 T
1 if
C ( x) t ht ( x) 2 t
t 1 t 1
0
otherwise
1
where t log
t
face interface v.1_2c 42
