Facial Fraud Detection using AdaBoost and Neural Network
Inho Choi and Daijin Kim
Computer Science and Engineering, POSTECH
This paper proposes the facial fraud detection using
facial feature detection and classification based on the
AdaBoost and Neural network. The proposed method
detects the face, the two eyes, and the mouth by the
AdaBoost detector. To classify detection results
whether normal eyes (mouth) or abnormal eyes (mouth),
we use the neural network. FRR and FAR of eye
discrimination of our algorithm is 0.0486 and 0.0152,
respectively. FRR and FAR of mouth discrimination of
our algorithm is 0.0702 and 0.0299, respectively.
Because of the increasing number of financial
automation machinery; Automated Teller Machine
(ATM) and Cash Dispenser (CD), not only the
conveniences of accessing the financial transactions but
also the illegal financial transactions are increased.
Where an illegal financial transactions are attempting
financial transactions by an user who has a stolen cash
card or a stolen credit card. Because biometric
information has not possibility of robbery, using
biometric information is a solution to restrict the illegal Figure 1. An overview of the proposed method.
financial transaction. According to "Financial success
for biometrics?" , many approaches which are the 2. FACIAL FEATURE DETECTION
fingerprint verification, iris verification, vein The Modified Census Transform (MCT) is a non-
verification, signature verification, keystroke dynamics parametric local transform which modifies the census
are used to verify customer's identity at the ATM. transform by Froba and Ernst . It is an ordered set of
These methods have not only the advantage of the comparisons of pixel intensities in a local neighborhood
substitution of the identification card such as a cash representing which pixels have lesser intensity than the
card or a credit card but also restricting the illegal mean of pixel intensities. We present the detection
financial transaction when cards lost. But many people method for the face, the eyes and the mouth by the
does not use these methods, because of refusal feeling AdaBoost training with MCT-based features using
about collecting biometric information. For this reason, positive and negative samples .
we study the prevention method for the illegal financial
transaction. In general, CCTV captures face images of 3. FACIAL FRAUD DETECTION
the ATM users. If an illegal financial transaction occurs,
captured images are used in order to grasp the suspect. Normal/Abnormal Face Classification using Neural
But captured images are not reliable if the customer Network
wears a mask or a sun-glass on his face. If we know the To classify detected eyes and mouth whether normal
wearing mask or sun-glass on his face, we can restrict eyes (mouth) or abnormal eyes (mouth), we use the
his financial transaction. Dong and Soh presented deep belief network by Hinton . This neural network
image-based fraud detection which based on moving has fast running speed and the good classification
objects detection, skin color and face template . This performance. To train normal/abnormal eye
approach is not a simple and very heuristic for face, classification, we use 4,000 eye image and 4,000 non-
eyes and mouth. So, we proposed a simple and a eye images. To train normal/abnormal mouth
intuitive algorithm which based on detection and classification, we use 4,500 mouth images and 4,500
classification methods using machine learning and non-mouth images. Histogram equalization is used for
pattern recognition approaches. Fig. 1 shows an reduce illumination variations in training image. Fig. 2
overview of the facial fraud detection method shows a flowchart of normal/abnormal classification
process. To normalize, we use center of eyes and
corners of lip. Fig. 3 shows some examples of the
normal/abnormal classification training data. Non-eye
and non-mouth data include sun-glass images and mask
Figure 4. Some results of facial fraud detection.
Table 1. Some results of facial fraud detection.
Category Using detector Using detector and
FRR of eye 0.93(5/535) 4.86(26/535)
FAR of eye 82.68(441/527) 1.52(8/527)
FRR of mouth 0.02(1/527) 7.02(37/527)
Figure 2. Normal/Abnormal Classification using Neural FAR of mouth 70.09(375/535) 2.99(16/535)
In this paper, we presented the facial fraud detection
algorithm that helps taking the reliable images with the
automated teller machine. It was based on the detection
(a) Eye and non-eye data (20x20) and classification algorithm which based on AdaBoost
with MCT based facial features and neural network. It
discriminated the fraud eyes (fraud mouth) by the
confidence interval of Binomial distribution for the
(b) Mouth and non-mouth data (40x20) facial feature detector and fraud classifier. The
Figure 3. Training data for normal/abnormal proposed algorithm is helpful to reduce of the illegal
classification to facial components. financial transaction on the ATM, and it helpful to
increase the reliability of the face recognition system
4. EXPERIMENTAL RESULTS and its applications.
To evaluate facial fraud detection, we make a subset
of the AR face database called AR-FDD that has the REFERENCE
purpose of testing facial fraud detection. The AR-140  “Financial success for biometrics?”, Biometric
FDD face database contains 1086 images (96 people x Technology Today, 2005.
2 different conditions x 3 different illuminations x 1 or  W. Dong and Y. Soh, “Image-based fraud detection
2 sessions, some people has one session). It has in automatic teller machine”, International Journal of
different conditions which are the wearing sun-glass or Computer Science and Network Security, 6(11), pp.
mask, and it consists of 3 different illuminations such 13–18, 2006.
as the normal illumination, the right illumination, and  B. Froba and A. Ernst, “Face detection with the
the left illumination. modified census transform”, In Proceedings of IEEE
Fig. 4 shows some examples of results in the AR- International Conference on Automatic Face and
FDD face database by the detection and classification Gesture Recognition, pp. 91–96, 2004.
algorithm in the AR-FDD face database.  I. Choi and D. Kim, “Eye correction using
Table 1 shows the performance of the facial feature correlation information”, In Proceedings of 8th Asian
detection and fraud detection algorithm in the AR-FDD Conference on Computer Vision, pp. 698–707, 2007.
face database. To compute accuracy, we use only  G. Hinton and R. Salakhutdinov, “Reducing the
correctly detected face images. The false acceptance of dimensionality of data with neural networks”, Science,
the eye detection and classification is calculated in the 313, pp. 504–507, 2006.
wearing mask group, because the wearing sun-glass
group has not detectable eyes.