Docstoc

Performance Evaluation of Object Tracking Algorithms

Document Sample
Performance Evaluation of Object Tracking Algorithms Powered By Docstoc
					                Performance Evaluation of Object Tracking Algorithms

                                Fei Yin, Dimitrios Makris, Sergio Velastin
                  Digital Imaging Research Centre, Kingston University London, UK
                           {fei.yin, d.makris, sergio.velastin}@kingston.ac.uk


                      Abstract                              detection, based on ROC-like curves and the F-
    This paper deals with the non-trivial problem of        measure. The latter allows straight-forward comparison
performance evaluation of motion tracking. We               using a single value that takes into account the
propose a rich set of metrics to assess different aspects   application domain.
of performance of motion tracking. We use six different        Needham and Boyle [9] proposed a set of metrics
video sequences that represent a variety of challenges      and statistics for comparing trajectories and evaluating
to illustrate the practical value of the proposed metrics   tracking motion systems.
by evaluating and comparing two motion tracking                Brown et al [10] suggest a motion tracking
algorithms. The contribution of our framework is that       evaluation framework that estimates the number of
allows the identification of specific weaknesses of         True Positive (TP), False Positive (FP) and False
motion trackers, such as the performance of specific        Negative (FN), Merged and Split trajectories. However
modules or failures under specific conditions.              their definition, based on the comparison of the system
                                                            track centroid and an enlarged ground truth bounding
1. Introduction                                             box) favours tracks of large objects.
                                                               Bashir and Porikli [11] gave definitions of the above
    Significant research effort has focused on video-       metrics based on the spatial overlap of ground truth and
based motion tracking [1] [2] [3] [4] and attract the       system bounding boxes that are not biased towards
interest of industry. Performance evaluation of motion      large objects. However they are counted in terms of
tracking is important not only for the comparison and       frame samples. Such an approach is justified when the
further development of algorithms from researchers,         objective of performance evaluation is object detection
but also for the commercialisation and standardisation      [7] [8]. In object tracking, measuring TP, FP and FN in
of the technology as typified by the i-LIDS Programme       terms of tracks rather than frames is a natural choice
in the UK [5].                                              that is consistent to the expectations of the end-users.
    In this paper, we have selected a set of motion            This paper is organized as follows: Section 2
tracking metrics that are used to highlight different       defines provides the definitions for motion tracking and
aspects of motion tracking performance. We illustrate       track. Section 3 describes the performance evaluation
the purpose of the proposed metrics in the evaluation of    methodology and gives definition of different metrics.
two motion tracking algorithms, using a variety of          Results are presented and discussed in section 4.
datasets.                                                   Section 5 concludes the paper.
    Ellis [6] investigated the main requirements for
effective performance analysis for surveillance systems     2. Motion Tracking
and proposed some methods for characterising video
datasets.                                                      We define motion tracking as the problem of
    Nascimento and Marques [7] proposed a framework         estimating the position and the spatial extent of the
which compares the output of different motion               non-background objects for each frame of a video
detection algorithms against given ground truth and         sequence. The result of motion tracking is a set of
estimates objective metrics such as Correct Detections,     tracks Tj, j=1..M, for all M moving objects of the
False alarms, Detection failures, Merges and Splits.        scene. A track Tj is defined as: Tj={xij, Bij}, i =1..N,
They also proposed ROC-like curves that can                 where xij and Bij are the centre and the spatial extent
characterize algorithms over a range of parameters.         (usually represented by a bounding box) respectively of
    Lazarevic-McManus et al [8] developed an object-        the object j for the frame i and N is the number of
based approach to enable evaluation of motion               frames.
                                                              where Length is the number of frames and TRov an
3 Performance Evaluation                                      arbitrary threshold. If Eq.4 is true, then, we associate
                                                              the system track with the GT track and start evaluating
3.1 Preparation                                               the performance of the system track.

    We propose a set of metrics that compare the output
 of motion tracking systems to a Ground Truth in order
 to evaluate the performance of the systems.
    Before the evaluation metrics are introduced, it is
 important to define the concepts of spatial and
 temporal overlap between tracks, which are required
 to quantify the level of matching between Ground
 Truth (GT) tracks and System (ST) tracks, both in
                                                                     Figure 2 Example of track overlapping
 space and time.
    The spatial overlap is defined as the overlapping
level A(GTi, STj) between GTi and STj tracks in a
specific frame k (Fig. 1).                                    3.2 Metrics
                    Area (GTik I ST jk )
 A(GTik , ST jk ) =                                 (1)           In this section, we give definitions of high level
                    Area (GTik U ST jk )                      metrics such as True Positive (TP), False Positive (FP)
                                                              and False Negative (FN) tracks. Such metrics are useful
                                                              because they are the base for estimating metrics such as
          GTik                      GTik                      Specificity and Accuracy [11] and allow the
                                                              construction of ROC-like curves [8]. Metrics such as
                 STjk                      STjk
                                                              Track Fragmentation and ID Change assess the
                                                              integrity of tracks. Finally, we define metrics that
                                                              measure the accuracy of motion tracking in estimating
Figure 1: Area (GTik I ST jk ) and Area (GTik U ST jk )       the position (Track Matching Error), the spatial extent
                                                              (Closeness), the completeness and the temporal
   We also define the binary variable O(GTi, STj),            latency.
based on a threshold Tov which in our examples is
arbitrarily set to 20%.                                       Correct detected track (CDT) or True Positive
                                                              (TP):
                   1   if A(GTik , ST jk ) > Tov                 A GT track is considered to have been detected
                                                       (2)
O(GTik , ST jk ) =                                           correctly if it satisfies both of the following conditions:
                   0
                       if A(GTik , ST jk ) ≤ Tov
                                                                  Condition 1: The temporal overlap between GT
                                                              Track i and system track j is larger than a predefined
    Temporal overlap TO(GTi,         STj) is a number that    track overlap threshold TROV which in our examples is
indicates overlap of frame span      between system track j   arbitrarily set to 15%.
and GT track i:                                                Length (GTi I ST j )
                                                                                      >= TRov                         (5)
                   TO − TO S ,      TO E > TO S                  Length(GTi )
TO (GTi , ST j ) =  E                             (3)
                       0,          TO E ≤ TO S                   Condition 2: The system track j has sufficient
where TOS is the maximum of the first frame indexes           spatial overlap with GT track i.
of TOE is the minimum of the last frame indexes of the         N

two tracks. The temporal and spatial overlap between          ∑ A(GT
                                                              k =1
                                                                        ik   , ST jk )
tracks is illustrated graphically in Fig. 2:                                             >= Tov                      (6)
    We use a temporal-overlap criterion to associate                  N
systems tracks to GT tracks according to the following            Each GT track is compared to all system tracks
condition:                                                    according to the conditions above. Even if there is
 Length (GTi I ST j )                                         more than one system track meets the conditions for
                       > TRov                      (4)        one GT track (which is probably due to
 Length (GTi U ST j )                                         fragmentation), we still consider the GT track to have
                                                              been correctly detected. Fragmentation errors are
counted separately (see below). So, if each of the GT         Condition 2: A GT track i does not have any sufficient
tracks is detected correctly (by one or more system           spatial overlap with any system track, although it has
tracks), the number of CDT equals the number of GT            enough temporal overlap with system track j.
tracks.                                                        N

                                                              ∑ A(GT    ik   , ST jk )
False alarm track (FAT) of False Positive (FP):
                                                              k =1
                                                                                         < Tov                     (10)
                                                                      N
    Although it is easy for human operators to realise
what is a false alarm track (event) even in complex
                                                                  Similar definitions of the above metrics have been
situation, it is hard for an automated system to do so.
                                                              given in [10] and [11]. In [10], spatial overlap is
Here, we give a practical definition of false alarm track
                                                              defined by checking whether the centroid of the system
(Fig. 3). We consider a system track as false alarm if
                                                              track is within the area of the GT track, enlarged by
the system track meets any of the following conditions:
                                                              20%. However, such a definition favours tracks of
    Condition 1: A system track j does not have
                                                              large objects. In [11], spatial overlap is defined in three
temporal overlap larger than TROV with any GT track i.
                                                              different ways: a) Euclidean distance between
Length(GTi I ST j )
                                < TRov                 (7)    centroids, b) system track centroid within GT area and
       Length( ST j )                                         c) area ratio overlap. Our approach is similar to the
                                                              latter definition. However, [11] estimates TP, FP, and
    Condition 2: A system track j does not have               FN in terms of frame samples, while we think that they
sufficient spatial overlap with any GT track although it      must be measured in terms of tracks (using the concept
has enough temporal overlap with GT track i.                  of temporal overlap) to be more informative to the end-
 N                                                            user.
∑ A(GT      ik   , ST jk )
k =1
                             < Tov                     (8)    Track Fragmentation (TF):
           N                                                      Fragmentation indicates the lack of continuity of
                                                              system track for a single GT track. Fig. 4 shows an
                                                              example of track fragmentation error:




                                                              Figure 4: The number of track fragmentations is
                                                              TF=2 (the system track fragmented two times).
Figure 3 Example of two false alarm tracks. The
                                                                  In an optimal condition, track fragmentation error
left ST fails condition (Eq.7), while the right ST
                                                              should be zero which means the tracking system is able
fails condition (Eq.8)
                                                              to produce continuous and stable tracking for the
                                                              ground truth object.
    FAT is an important metric because it is
                                                                  As mentioned before, we allow multiple
consistently indicated by operators that a system which
                                                              associations between GT track and system track
does not have a false alarm rate close to zero is likely
                                                              therefore fragmentation is measured from the track
to be switched off, not matter its TP performance.
                                                              correspondence results.
Track detection failure (TDF):
 A GT track is considered to have not been detected           ID Change (IDC):
(i.e. it is classified as a track detection failure), if it       We introduce the metric IDCj to count the number
satisfies any of the following conditions.                    of ID changes for each STj track. Note that such a
Condition 1: A GT track i does not have temporal              metric provides more elementary information than an
overlap larger than TROV with any system track j.             ID swap metric.
                                                                  For each frame k, the bounding box Dj,k of the
Length (GTi I ST j )
                                < TRov                 (9)    system track STj may be overlapped with N Dj ,k GT
       Length(GTi )
                                                              areas, where       N Dj ,k is given by:
                                                              N Dj ,k = ∑ O(Gik , D jk )                           (11)
                                                                             i
   We take into account only the frames for which              enough to trigger the tracking in time or indicates that
N Dj ,k =1 (which means that the track STj is associated       the detection is not good enough to trigger the tracking.
                                                                   It is estimated by the difference in frames between
(spatially overlapped) with only one GT Track for each         the first frame of system track and the first frame of GT
of these frames). We use these frames to estimate the          track.
ID changes of STj as the number of changes of                                                                       (13)
                                                                LT = start frame of ST j − start frame of GTi
associated GT tracks.
    We can estimate the total number of IDC changes
in a video sequences as:
IDC = ∑ IDC j                                           (12)
          j
The procedure for counting ID change is shown in Fig.
5. Some examples of estimating ID changes are shown
in Fig. 6:
                                                                  Figure 7: Example of Latency of system track
 For each System track j;
   IDCj=-1                                                     Closeness of Track (CT):
      For every frame k;                                          For a pair of associated GT track and system track,
         If there is no occlusion, and   N Dj ,k =1;           a sequence of spatial overlaps (Fig. 2) is estimated by
           If IDCj=-1 or M(IDCj) ≠i                            Eq.2 for the period of temporal overlap:
          ( i is the ID of GT track whose area is              CT (GTi , ST j ) = { A(GTi1 , ST j1 ),... A(GTiN ES , ST jN ES )} (14)
 overlapped with the GT track in this frame.)                      From Eq.14, we can estimate the average closeness
          Then,
               IDCj=IDCj+1; M(IDCj) = i;
                                                               for the specific pair of GT and system tracks. To
           End                                                 compare all M pairs in one video sequence, we define
           IDC=sum(IDCj)                                       the closeness of this video as the weighted average of
      End                                                      track closeness of all M pairs:
                                                                                    M
 End
Figure 5: Pseudo-code for estimated ID Changes
                                                                                   ∑ CT
                                                                                   t =1
                                                                                          t
                                                                                                                               (15)
                                                               CTM =
                    (IDC)                                                    M

                                                                          ∑ Length (CT )
                                                                           t =1
                                                                                                  t




                                                               and the weighted standard deviation of track closeness
                                                               for the whole sequence:
                                                                         M

                                                                        ∑ Length(CT ) × std (CT )
                                                                         t =1
                                                                                              t           t
                                                                                                                               (16)
                                                               CTD =              M
                                                                                                              ,
     ID swap = 2 ID changes              No ID change
                                                                                  ∑ Length(CT )
                                                                                  t =1
                                                                                                      t


                                                               where std(CTt) is the standard deviation of CTt

                                                               Track Matching Error (TME):
    ID change                              No ID change            This metric measures the positional error of system
                                                               tracks. Fig. 8 shows positions of a pair of tracks.
  Figure 6: Examples of ID swap and ID change

Latency of the system track (LT):
    Latency (time delay) of the system track is the time
gap between the time that an object starts to be tracked             Figure 8 Example of a pair of trajectories
by the system and the first appearance of the object
(Fig.7). The optimal latency should be zero. A very                TME is the average distance error between a system
large latency means the system may not be sensitive            track and the GT track. The smaller the TME, the
                                                               better the accuracy of the system track will be.
         N                                                                                           I

      ∑ Dist (GTC , STC )                ik               jk                                        ∑ {max(TC
                                                                                                    i =1
                                                                                                                GTi   ) − TCT }
                                                                                                                                              (22)
TME =   k =1                                                                         (17)   TCD =
       Length(GT I ST )                  i            j
                                                                                                            N −1


where Dist() is the Euclidean distance between the                                          4. Results
centroids of GT and the system track:
                                                                                                We demonstrate the practical value of the proposed
   TMED is the standard deviation of distance errors,                                       metrics by evaluating two motion tracking systems (an
which is defined as:                                                                        experimental industrial tracker from BARCO and the
                                                                                 2
                                                                                            OpenCV1.0 blobtracker [12]). We run the trackers on
                N

                ∑ ( Dist (GTCik , STC jk ) − TMEM )                                         six video sequences (shown in Fig.9-Fig.14) that
                                                                                            represent a variety of challenges, such as illumination
TMED =          k =1                                                                 (18)
                                  Length(GTi I ST j ) − 1                                   changes, shadows, snow storm, quick moving objects,
                                                                                            blurring of FOV, slow moving objects, mirror image of
    Similarly, track matching error (TMEMT) for the                                         objects and multiple object intersections. The ground
whole video sequence is defined as the weighted                                             truth for all videos was manually generated using Viper
average over the duration of overlapping of each pair                                       GT [13].
of tracks as the weight coefficient.
                M

                ∑ Length(GT
                t =1
                                                 i   I ST j )t × TMEt
                                                                                     (19)
TMEMT =                    M

                           ∑ Length(GT
                           t =1
                                                          i    I ST j )t

and the standard deviation of track matching errors for
the whole sequence:                                                                         Figure 9: PETS2001 PetsD1TeC1.avi sequence is
                       M                                                                    2686 frames (00:01:29) long and depicts 4
                    ∑ Length(GT I ST ) × TMED
                    t =1
                                                      i              j t         t
                                                                                     (20)
                                                                                            persons, 2 groups of persons and 3 vehicles. Its
TMEMTD =                          M
                                                                                            main challenge is the multiple object intersections.
                                  ∑ Length(GT I ST )
                                  t =1
                                                                 i         j t




Track Completeness (TC):
    This is defined as the time span that the system
track overlapped with GT track divided by the total
time span of GT track. A fully complete track is where
this value is 100%.                                                                         Figure 10: i-LIDS SZTRA103b15.mov sequence is
       N ES
                                                                                            5821 frames (00:03:52) long and depicts 1
       ∑ O(GT              ik   , ST jk )
                                                                                            person. Its main challenges are the illuminations
TC =   k =1                                                                          (21)
        number of GTi                                                                       changes and a quick moving object.


   If there is more than one system track associated
with the GT track, then we choose the maximum
completeness for each GT track
   Also, we define the average track completeness of a
video sequence as:
          I

         ∑ max(TC                  GTi       )                                              Figure 11: i-LIDS SZTRA104a02.mov sequence is
TCM =    i =1                                                                        (22)   4299 frames (00:02:52) long and depicts one
                N                                                                           person.
where N is the number of GT tracks and the standard
deviation of track completeness for the whole sequence
is:
                                                             Table 1 Evaluation results for PETS2001 Seq.
                                                                    PetsD1TeC1.avi         BARCO            OpenCV
                                                                                           tracker          tracker
                                                                    GT Tracks              9                9
                                                                    System Tracks          12               17
                                                                    CDT                    9                9
Figure 12: i-LIDS PVTRA301b04.mov sequence is                       FAT                    3                6
7309 frames (00:04:52) long and depicts 12                          TDF                    0                0
persons and 90 vehicles. Its main challenges are                    TF                     3                3
shadows, moving object in the beginning of                          IDC                    5                7
sequence and multiple object intersections.                         LT                     46               66
                                                                    CTM                    0.47             0.44
                                                                    CTD                    0.24             0.14
                                                                    TMEMT                  15.75            5.79
                                                                    TMEMTD                 23.64            5.27
                                                                    TCM                    0.67             0.58
                                                                    TCD                    0.24             0.89

Figure 13: BARCO 060306_04_Parkingstab.avi is                   Table 2 BARCO track association results for
7001 frames long and depicts 3 pedestrians and 1                               Pets2001
vehicle. Its main challenge is the quick illumination
                                                                PetsD1TeC1.avi: BARCO tracker
changes.
                                                                GT-ID   ST-ID   LT     CTM                CTD         TC
                                                                0              1     198          0.571   3.19        0.61
                                                                1              2     27           0.677   8.34        0.91

                                                                2              3     49           0.512   11.64       0.29
                                                                2              5     341          0.410   28.90       0.55
                                                                3              1     26           0.529   14.64       0.15
                                                                3              5     252          0.256   54.46       0.08
Figure 14: BARCO 060306_02_Snowdivx.avi is                      4              4     35           0.639   5.91        0.84
8001 frames long and depicts 3 pedestrians. Its                 5              5     0            0.391   10.80       0.62
main challenges are snow storm, blurring of FOV,                6              5     0            0.049   184.60      0.14
slow moving objects and mirror image of objects.                6              4     95           0.528   7.21        0.77
                                                                7              1     32           0.460   5.83        0.88
   The results of performance evaluation are presented          8              5     0            0.420   18.80       0.87
in Tables 1-8. Generally, high level metrics such CDT,
FAT, TDF show that the BARCO tracker outperforms                Table 3 OpenCV track association results for
the OpenCV tracker in almost all cases. The only                              PETS2001
exception is the i-Lids sequence PVTRA301b04.mov            PetsD1TeC1.avi: OpenCV tracker
(Fig. 12, Table 6). On the other hand, the OpenCV           GTID          cvSTID     LT            CTM     CTD        TC
tracker is generally more accurate in estimating the        0             1          16            0.54    2.13       0.97
position of the objects (lower TMEM).                       1             2          56            0.35    11.83      0.82
   Also, the OpenCV tracker dealt better with the snow
                                                            2             7          24            0.46    8.95       0.27
scene (Fig.14, Table8) in estimating the position and
the spatial and temporal extent of the objects (lower       2             12         0             0.49    5.01       0.13
TMEM, higher CTM and TCM), which implies a better           3             3          226           0.36    9.89       0.09
object segmentation module for this scene. However,         3             5          346           0.56    4.72       0.12
the BARCO tracker has better high level metrics (lower      4             4          120           0.52    6.24       0.56
FAT, lower TF), which implies a better tracking policy.     5             6          166           0.50    7.37       0.33
   Note that without the rich set of metrics as used here   6             11         12            0.40    3.60       0.98
it is very difficult to identify possible causes of         7             14         26            0.37    2.87       0.81
poor/good performance in different trackers.                8             13         14            0.45    10.16      0.40
                                                            8             15         300           0.34    15.59      0.28
                                                     LT                57             78
     Table 4 Tracking PE results for i-LIDS          CTM               0.30           0.12
                SZTRA103b15                          CTD               0.21           0.16
SZTRA103b15              BARCO         OpenCV        TMEMT             49.70          24.65
.mov                     tracker       tracker       TMEMTD            60.31          22.85
GT Tracks                1             1             TCM               0.34           0.26
                         8                           TCD               0.57           0.65
System Tracks                          15
CDT                      1             1
                                                      Table 7 Tracking PE results for BARCO
FAT                      3             12                          Parkingstab
TDF                      0             1             Parkingstab.avi   BARCO          OpenCV
TF                       0             0                               tracker        tracker
IDC                      0             0             GT Tracks         4              4
LT                       50            9             System Tracks     9              17
CTM                      0.65          0.23          CDT               4              4
CTD                      0.21          0.10          FAT               1              11
TMEMT                    9.10          15.05         TDF               0              0
TMEMTD                   12.48         3.04          TF                0              0
TCM                      0.68          0.42          IDC               0              0
TCD                      0.00          0.00          LT                72             35
                                                     CTM               0.50           0.39
     Table 5 Tracking PE results for i-LIDS          CTD               0.20           0.14
                SZTRA104a02                          TMEMT             13.32          11.82
SZTRA104a02         BARCO          OpenCV            TMEMTD            11.55          8.16
.mov                tracker        tracker           TCM               0.82           0.77
                                                     TCD               0.11           0.96
GT Tracks           1              1
System Tracks       4              9
CDT                 1              1             Table 8 Tracking PE results for BARCO Snowdivx
FAT                 0              5                Snowdivx.avi      BARCO         OpenCV
TDF                 0              0                                  tracker       tracker
TF                  0              2                 GT Tracks         3              3
IDC                 0              0                 System Tracks     28             29
LT                  74             32                CDT               3              3
CTM                 0.79           0.34              FAT               19             20
CTD                 0.21           0.17              TDF               0              0
TMEMT               7.02           16.69             TF                2              5
TMEMTD              15.67          7.55              IDC               0              0
TCM                 0.73           0.44              LT                590            222
TCD                 0.00           0.00              CTM               0.14           0.42
                                                     CTD               0.23           0.12
     Table 6 Tracking PE results for i-LIDS          TMEMT             28.50          16.69
                PVTRA301b04                          TMEMTD            35.44          11.62
PVTRA301b04         BARCO          OpenCV            TCM               0.33           0.35
.mov                tracker        tracker           TCD               0.47           0.71
GT Tracks           102            102
System Tracks       225            362           5. Conclusions
CDT                 90             95
FAT                 67             112               We presented a new set of metrics to assess
TDF                 12             7             different aspects of performance of motion tracking.
TF                  62             98            We proposed statistical metrics, such as Track
IDC                 95             102           matching Error (TME), Closeness of Tracks (CT) and
Track Completeness (TC) that indicate the accuracy of            Imagery library for intelligent detection systems (i-LIDS),
estimating the position, the spatial and temporal extent         http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-
of the objects respectively and they are closely related         imaging-technology/video-based-detection-systems/i-lids/,
                                                                 [Last accessed: August 2007]
to the motion segmentation module of the tracker.
    Metrics, such as Correct Detection Track (CDT),              [6] T. Ellis, “Performance Metrics and Methods for
False Alarm Track (FAT) and Track Detection Failure              Tracking in Surveillance”, Third IEEE International
(TDF) provide a general overview of the algorithm                Workshop on Performance Evaluation of Tracking and
performance. Track Fragmentation (TF) shows the                  Surveillance, June, Copenhagen, Denmark, 2002, pp26-31.
temporal coherence of tracks. ID Change (IDC) is                 [7] J. Nascimento, J. Marques, “Performance evaluation of
useful to test the data association module of multi-             object detection algorithms for video surveillance”, IEEE
target trackers.                                                 Transactions on Multimedia, 2005, pp761-774.
    We tested two trackers using six video sequences             [8] N.Lazarevic-McManus, J.R.Renno, D. Makris,
that provide a variety of challenges, such as                    G.A.Jones, “An Object-based Comparative Methodology for
illumination changes, shadows, snow storm, quick                 Motion Detection based on the F-Measure”, in 'Computer
moving objects, blurring of FOV, slow moving objects,            Vision and Image Understanding', Special Issue on
mirror image of objects and multiple object                      Intelligent Visual Surveillance TO APPEAR, 2007.
intersections.                                                   [9] C.J. Needham, R.D. Boyle. “Performance Evaluation
    The variety of metrics and datasets allows us to             Metrics and Statistics for Positional Tracker Evaluation”
reason about the weaknesses of particular modules of             International Conference on Computer Vision Systems
the trackers against specific challenges, assuming               (ICVS'03), Graz, Austria, April 2003, pp. 278 - 289.
orthogonality of modules and challenges. This
                                                                 [10] L. M. Brown, A. W. Senior, Ying-li Tian, Jonathan
approach is a realistic way to understand the drawbacks          Connell, Arun Hampapur, Chiao-Fe Shu, Hans Merkl, Max
of motion trackers, which is important for improving             Lu, “Performance Evaluation of Surveillance Systems Under
them.                                                            Varying Conditions”, IEEE Int'l Workshop on Performance
    In future work, we will use this framework for               Evaluation of Tracking and Surveillance, Colorado, Jan
evaluating more trackers. We will also extend the                2005,.
framework to allow evaluation of high level tasks such           [11] F. Bashir, F. Porikli. “Performance evaluation of object
as event detection and action recognition.                       detection and tracking systems”, IEEE International
                                                                 Workshop on Performance Evaluation of Tracking and
6. Acknowledgments                                               Surveillance (PETS), June 2006
                                                                 [12] OpenCV Computer Vision Library
The authors would like to acknowledge financial                  http://www.intel.com/technology/computing/opencv/index.ht
support from BARCO View, Belgium and the                         m, [Last accessed: August 2007]
Engineering and Physical Sciences Research Council
                                                                 [13] Guide to Authoring Media Ground Truth with ViPER-
(EPSRC) REASON project under grant number                        GT,     http://viper-toolkit.sourceforge.net/docs/gt/, [Last
EP/C533410.                                                      accessed: July 2007]
                                                                 [14] Pets Metrics, http://www.petsmetrics.net/         [Last
7. References                                                    accessed: August 2007]
[1] I. Haritaoglu, D. Harwood, and L. S. Davis, “W : Real-
time surveillance of people and their activities,” IEEE Trans.
Pattern Anal. Machine Intell., Aug. 2000, pp. 809 - 830
[2] M. Isard and A. Blake, “Contour tracking by stochastic
propagation of conditional density,” in Proc. European Conf.
Computer Vision, 1996, pp. 343 - 356
[3] M. Xu, T.J. Ellis, “Partial observation vs. blind
tracking through occlusion”, British Machine Vision
Conference, BMVA, September, Cardiff, 2002, pp. 777 -786
[4] Comaniciu D., Ramesh V. and Meer P., \Kernel-Based
Object Tracking", IEEE Trans. Pattern Anal. Machine Intell.,
2003. pp. 564 - 575
[5]   Home Office Scientific Development Branch

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:134
posted:3/12/2010
language:English
pages:8
Description: Performance Evaluation of Object Tracking Algorithms