Automatic Detection of Human Fall in Video by yaofenjin

VIEWS: 10 PAGES: 8

									    Automatic Detection of Human Fall in Video

         Vinay Vishwakarma, Chittaranjan Mandal and Shamik Sural

                         School of Information Technology,
                  Indian Institute of Technology, Kharagpur, India
                    vvinay, chitta, shamik@sit.iitkgp.ernet.in


      Abstract. In this paper, we present an approach for human fall de-
      tection, which has an important application in the field of safety and
      security. The proposed approach consists of two part: object detection
      and fall model. We use an adaptive background subtraction method to
      detect moving object and mark it with minimum-bounding box. Fall
      model uses a set of extracted features to analyze, detect and confirm
      the fall. We implement a two-state finite state machine (FSM) to con-
      tinuously monitor people and their activities. Experimental results show
      that our method can detect all possible types of human fall accurately
      and successfully.


1   Introduction

Falling is one of the major health problem for elderly people. Falls are danger-
ous and often cause serious injuries that may lead even to death. Fall related
injuries have been among the five most common cause of death amongst elderly
population. It is shown in [1] that the number of reported human falls per year
is around 60,000 with an associated cost of at least $400 millions.
     It has been found that the early detection of fall is an important step to avoid
any serious injuries. An automatic fall detection system can help to overcome
these problems by reducing the time between the fall and arrival of required
assistance. Here, we present an approach for human fall detection using a single
camera video sequence. Our approach consist of two step: object detection and
fall model. We use an adaptive background subtraction method to detect moving
object and mark it with minimum-bounding box. Fall model consists of two part:
fall detection and fall confirmation. It uses a set of extracted features to analyze,
detect and confirm the fall. We also implement a two-state finite state machine
to continuously monitor people and their activities.
     The organization of the paper is as follows. Section 2 explains the related
work on fall detection. Section 3 describes the object detection method. Section
4 describes the fall model. Section 5 presents experimental results followed by
our conclusion in section 6.


2   Related Work

There are three ways of fall detection, classified in the following categories:
2    Automatic Detection of Human Fall in Video

1. Acoustic Based Fall Detection
2. Wearable Sensor Based Fall Detection
3. Video Based Fall Detection

    In video based fall detection, human activity is captured in a video that
is further analyzed using image processing and computer vision techniques to
detect fall and generate an alarm. It can also provide the exact cause of human
falling. Since video cameras have been widely used for surveillance, home and
health care applications, therefore we use this approach for our fall detection
method.
    Due to the advancements in vision technologies, many individuals, researchers
and organizations are concentrating on fall detection using vision or video based
approaches. In [2], they have used background modeling and subtraction of video
frames in HSV color space. An on-line hypothesis-testing algorithm is employed
in conjunction with finite state machine to infer fall incident detection. They
have only extracted aspect ratio of person as an observation feature based on
which fall incident is detected. In [3], they have presented a fall detection al-
gorithm using dynamic motion pattern analysis. They have assumed that a fall
can only start when the subject is in an upright position and characterize a big
change in either X or Y direction when a fall starts. In [5], they have used back-
ground estimation method to detect moving regions. Using connected component
analysis, they obtain the minimum bounding rectangles (blob) and calculated
the aspect ratio. They have used audio channel data based decision and fused
with video data based decision to reach final decision of human fall using Hidden
Markov Modeling (HMM). In [6], they have subtracted the current image from
the background image to extract the foreground of interest. To obtain the associ-
ated threshold, they have used subject’s personal information such as weight and
height. Each extracted aspect ratio is validated with user’s personal information
to detect the human fall. In [7], they have used a particle filtering method to
track the person and extract its trajectories using 5-D ellipse parameters space
in each sequence. An associated threshold on person’s speed is used to label the
inactivity zone and human fall. In [8], they have used 3-D velocity as a feature
parameter to detect human fall from a single camera video sequence. At first,
3-D trajectory is extracted by tracking person’s head using a particle filter as
it has a large movement during a fall. Next, 3-D velocity is computed from 3-D
trajectory of the head.
    Most of existing vision based fall detection system use motion information
or background subtraction method for object detection. An abrupt change in
the aspect ratio is analyzed by different ways such as Hidden Markov Model
(HMM), adaptive threshold, and user’s personal information to detect falls in
video. Person’s velocity is used to classify human as walking or falling in video.
There are also some existing models, but they work well only in the restricted
environment.
    In our approach, we use an adaptive background subtraction method using
Gaussian Mixture Model (GMM) in YCbCr color space for object detection. We
propose a fall model that consists of two step, first fall detection and next fall
                    Vinay Vishwakarma, Chittaranjan Mandal and Shamik Sural            3

confirmation. We extract three feature parameters from object and use the first
two feature parameters for fall detection and last parameter for fall confirmation.
We implement a simple two-state finite state machine (FSM) to continuously
monitor people and their activities.


3     Object Detection

The first and most important task of human fall detection is to detect human
objects accurately and successfully for which we apply an adaptive background
subtraction method using Gaussian mixture model (GMM), and then extract a
set of features for fall model.


3.1     Background Modeling and Subtraction

A recorded video is used as an input video. Dump and store the input video
into a sequence of frames using Berkeley MPEG Decoder. For every frame, we
convert the pixel from RGB color space to YCbCr color space and use the value
of Y as pixel intensity. We use mean value of image pixel for further processing.
    We use an adaptive background subtraction method using Gaussian mixture
model (GMM) [9]. This method is to model each background pixel as a mixture of
Gaussians. The Gaussians are evaluated using a simple heuristics to hypothesize
which are most likely to be part of the background process. The probability that
an observed pixel has intensity value Xt at time t is modeled by a mixture of K
Gaussians as
                                         k
                           P (Xt ) =          wi,t ∗ η(Xt , µi,t , Σi,t ).            (1)
                                        i=1

where
                                        1            1
          η(Xt , µi,t , Σi,t ) =         n   n   ∗ e 2 (Xt − µt )T Σ −1 (Xt − µt ).   (2)
                                   ((2π) |Σ|
                                         2   2 )



                             ωi,t = (1 − α)ωi,t−1 + α(Mk,t ).                         (3)
where m is the mean, α is the learning rate and Mk,t is 1 for the model which
matches and 0 for the remaining model.
    The background estimation problem is solved by specifying the Gaussian
distributions, which have the most supporting evidence and the least variance.
Because the moving object has larger variance than a background pixel, so in
order to represent background process, first the Gaussians are ordered by the
value of ω in decreasing order. The background process stays on top with the
          α
lowest variance by applying a threshold T, where
                                                      b
                               B = argminb (              ωk ≥ T ).                   (4)
                                                    k=1
4                              Automatic Detection of Human Fall in Video

All pixels Xt which do not match any of these components will be marked as
foreground.
    Finally, pixels are partitioned as being either in the foreground or in the
background and marked appropriately. Next, we apply connected component
analysis (size = 2) that can identify and analyze each connected set of pixel to
mark the rectangular bounding box over object.


3.2                            Feature Extraction

We extract a set of features from object and its bounding box such as aspect
ratio, horizontal (Gx ) and vertical (Gy ) gradient distribution of object in XY
plane and fall angle, which we use further in fall model.


Aspect Ratio. The aspect ratio of a person is probably the simplest but effec-
tive feature for differentiating normal standing pose from other abnormal pose.
In Table 1(a), we compare the aspect ratio of object in different human pose.


Horizontal and Vertical Gradients of an Object. When falls start, a major
change occurs in either X or Y direction. For each pixel, we calculate its gradient
value both horizontally (Gx ) and vertically (Gy ). In Table 1(b), we compare
horizontal and vertical gradient distribution of object’s pixel in different human
pose.


Fall Angle. Fall angle (θ) is the angle of centroid of object with respect to
horizontal axis of the bounding box. Centroid (Cx , Cy ) is the center of mass co-
ordinates of an object. In Table 1(c), we compare fall angle of object in different
pose such as walking and falling.


                                Table 1. Comparison of features distribution of object in different pose


                   9                                                                                                        Horizontal Gradient       Vertical Gradient            90



                   8                                                                                                       70000
                                                                                                                                                                                   80
                                                                                                                                                                                           Walking
                   7                                                                                                                                                               70

                                                                                                                           60000
                                                                                                      G rad ien t Valu e




                                                                                                                                                                          Fall Angle
    Aspect Ratio




                   6                                                                                                                                                               60

                                                                                                                           50000                   Falling                                                   Falling
                   5                                                                                                                                                               50

                                   Walking                                                                                 40000
                   4                                                                                                                                                               40



                   3                                                    Falling                                            30000                                                   30




                   2                                                                                                       20000
                                                                                                                                                                                   20




                   1                                                                                                       10000
                                                                                                                                                                                   10



                                                                                                                                                                                       0
                   0                                                                                                           0
                       1

                           4

                               7

                                   10

                                        13

                                             16

                                                  19

                                                         22

                                                              25

                                                                   28

                                                                        31

                                                                             34

                                                                                  37

                                                                                       40

                                                                                            43

                                                                                                 46




                                                                                                                                                                                              Frame Number
                                                       Frame Number                                                                        Frame Number

                                                  (a)                                                                                    (b)                                                     (c)
                  Vinay Vishwakarma, Chittaranjan Mandal and Shamik Sural          5

4     Fall Model

Fall Model consists of two step: fall detection and fall confirmation. For fall
detection, we use aspect ratio and pixel’s gradient distribution (horizontally and
vertically). For fall confirmation, we use fall angle with respect to its horizontal
axis of bounding box. We use rule-based decisions to detect and confirm the fall.


4.1   Fall Detection

 1. Aspect ratio of human body changes during fall. When a person falls, it’s
    bounding box height and width changes drastically.
 2. When a person is walking or standing, their horizontal gradient is less than
    vertical gradient (Gx < Gy ) and when a person is falling, their horizontal
    gradient is always greater than vertical gradient (Gx > Gy ).
 3. For every feature, we assign a binary value. If the extracted features satisfy
    with rules, we assign binary value as 1 otherwise as 0.
 4. We apply OR operation with obtained feature values. If we get the resultant
    binary value as 1 then we detect the person as falling otherwise not.


4.2   Fall Confirmation

When a person is standing, we assume that he is in upright position and angle
of centroid of object with respect to horizontal axis of bounding box, θ should
be approximately 90 degree. When a person is walking, its θ value varies from
45 degree to 90 degree (depends on their style and speed of walking) and when
a person is falling, its θ value is always less than 45 degree.
    For every frame where we have detected fall earlier, we take those frames
and confirm by fall confirmation step. We calculate the fall angle (θ) and if θ
value is less than 45 degree, we confirm that a person is falling. Similarly, we
take next seven frames and analyze their feature parameters by fall model to
conclude that there is a confirm fall in video.


4.3   State Transition

To continuously monitor human behaviors, which could change from time to
time, we implement a simple two-state finite state machine (FSM). As shown
in Fig. 1, the two states are ’Walk’ and ’Fall’ respectively. Both states perform
rule testing (Rule 1 and Rule 2), if it satisfies the rule, the state transits to next
state.
    Rule 1: Features are satisfying by fall detection model.Feature values (0 <
Aspect ratio < 1 or Gx > Gy ) should return binary value as 1.
    Rule 2: Features are satisfying by fall confirmation model. Fall angle should
be less than 45 degree.
    When current state is ’Walk’, the system begins to perform Rule 1 testing. If
rule 1 does not satisfy, the state remains unchanged; otherwise the state transits
6    Automatic Detection of Human Fall in Video



                                            Rule 1
                                           satisfied

                                   Walk                Fall
                                          Rule 2 not
                      Rule 1 not           satisfied
                       satisfied                               Rule 2
                                                              satisfied


               Fig. 1. A finite state machine for human fall detection


to ’Fall’. When current state is ’Fall’, the system begins to perform Rule 2 testing.
If rule 2 does not satisfy, the state remains unchanged; otherwise it transits back
to ’Walk’ state, which is the case when a person has fallen for a meanwhile and
again started to walk. Alarms will be triggered once a person remains in the
state of ’Fall’ for a period longer than a present duration.


5   Experimental Results

We have implemented our proposed approach using C as the programming lan-
guage in Linux and tested it intensively in a laboratory environment. To verify
the feasibility of the proposed approach, we have used 40 video clips as our test
targets. The camera is placed on a table with horizontal viewing angle. The
person is about 3m - 10m away from the camera which is a reasonable distance
for indoor and outdoor surveillance system. Both indoor and outdoor video clips
are used containing all possible types of fall (Sideway, Forward, Backward) and
No fall. In all clips, only one moving object (human) exists in the scene. We
have also verified our proposed approach using omni-camera images and able to
detect human fall successfully.
    In general, our fall detection approach is able to achieve promising results
under most of test conditions. We use a set of criteria to evaluate our system
including accuracy, sensitivity and specificity [6].
    In Table 2, we have taken both indoor and outdoor video containing different
types of possible fall. In our experiment, a fixed threshold is set for every feature
parameter. For aspect ratio, we set the threshold between 0 and 1. For horizontal
and vertical gradient distribution, we have found that there is more gradient
distribution horizontally (Gx ) than vertically (Gy ). For a walking person, Gx is
less than Gy . For a falling person, Gx is greater than Gy . For fall angle, θ is
less than 45 degree. For a walking or standing person, θ is between 45 degree
to 90 degree. For a falling person, θ is less than 45 degree. We have finalized
these thresholds in experimental way and found true for all cases. Final system
performances (Accuracy, Sensitivity and Specificity) of our proposed approach
                Vinay Vishwakarma, Chittaranjan Mandal and Shamik Sural    7

                           Table 2. Recognition results

              Video Types Total Frames Fall Types TP FP FN TN
              Indoor        93           Forward    20    0   0   27
              Indoor        216          Backward   56    0   0   150
              Indoor        286          Sideway    76    0   0   172
              Indoor        100          No Fall    0     0   0   92
              Outdoor       87           Forward    47    0   0   29
              Outdoor       141          Backward   14    0   0   114


                        Table 3. Final system performance

                              Parameter Result(%)
                              Accuracy 100
                              Sensitivity 100
                              Specificity 100




are shown in Table 3. Some sample Image frames of successful fall detected by
our proposed approach are shown in Table 4.


        Table 4. Image frames of fall detected by our proposed approach




6   Conclusion

We have presented an automatic detection of human fall in video. The pro-
posed approach contains two main components, object detection and fall model.
For object detection, we use an adaptive background subtraction method using
Gaussian mixture model (GMM) in YCbCr color space. We extract a set of
8    Automatic Detection of Human Fall in Video

features such as aspect ratio, horizontal (Gx ) and vertical (Gy ) gradient distri-
bution and fall angle. Our proposed two-step fall model gives good results. We
implement a simple two-state finite state machine (FSM) to continuously moni-
tor people and their activities. The experimental results show that the proposed
method can accurately and successfully detect all possible types of fall in video.


References
[1] Marquis-Faulkes, F., McKenna, S. J., Newell, A.F., Gregor, P.: Gathering the re-
  quirements for a fall monitor using drama and video with older people. Technology
  and Disability, vol. 17 (2005) 227–236
[2] Tao, J., Turjo, M., Wong, M., Wang, M., Tan, Y.: Fall incident detection for intelli-
  gent video surveillance”, Fifth International Conference on Information. Fifth Inter-
  national Conference on Information, Communication and Signal Processing, (2005)
  1590–1594
[3] Luo, S., Hu, Q.: A dynamic motion pattern analysis approach to fall detection. In
  IEEE International workshop in Biomedical Circuit and Systems, (2004)
[4] Alwan, M., Rajendran, P. J.: A smart and passive floor-vibration based fall detector
  for elderly. Information and Communication Technologies, vol. 1 (2006) 1003–1007
[5] Ugur Toreyin, B., Dedeoglu, Y., Enis Cetin, A.: HMM based falling person de-
  tection using both audio and video. In Proc. IEEE Conf. Signal Processing and
  Communication Application (14th), (2006)
[6] Miaou, S., Sung, P., Huang, C.: A Customized Human Fall Detection System using
  omni-camera images and personal information. In Proc. of the 1st Distributed Diag-
  nosis and Home Healthcare (D2H2) Conference, Arlington, Virginia, USA, (2006)
[7] McKenna, S. J., Nait-Charif, H.: Summarising contextual activity and detecting
  unusual inactivity in a supportive home environment. Pattern Analysis Application
  (14th), vol. 7 (2005) 386-401
[8] Rougier, C., Meunier, J.: Demo: Fall detection using 3D head trajectory extracted
  from a single camera video sequence. In Journal of Telemedicine and Telecare, vol.
  11 no. 4 (2005)
[9] Stauffer, C., Grimson, W. E. L.: Adaptive background mixture model for real time
  tracking. In Proc. IEEE Conf. Computer Vision and Pattern Recognition CVPR’99,
  (1999) 246-252

								
To top