Docstoc

iv07_camshift

Document Sample
iv07_camshift Powered By Docstoc
					  Image processing
and computer vision

Mean-shift and Cam-shift
Ref
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust
Approach Toward Feature Space Analysis"Volume 24 , Issue 5
(May 2002),IEEE Transactions on Pattern Analysis and Machine
Intelligence
[2]
web.missouri.edu/~hantx/ECE8001/notes/Lect7_mean_shift.pdf



                         Camshift v3a                          1
What is Mean-shift?
   Find the peak of a probability function by the
    change of the mean of the data
   Applications:
       Segmentation
       Non-rigid object tracking




                            Camshift v3a             2
Applications: segmentation of
regions of images in a movie
   Use color to segment the image into logical
    regions for analysis.
   If the regions are moving , mean-shift is
    useful.




                                Camshift v3a                     3

    http://www.youtube.com/watch?v=n1W0IwipRBQ&feature=related
Application: tracking non-rigid
object
   Human tracking




                              Camshift v3a         4

      http://www.youtube.com/watch?v=zLtjPfPP9HY
Intuition: find the mode by
mean-shift
   Target : Find the modes (peaks) in a set
    of sample data.
       The mode of a continuous probability
        distribution is the peak.
       There may be multiple peaks.
   The method used is called mean-shift.
       By finding the shift of the mean, we can find
        the mode (peak)
   It can be used to segment an image into
    logical regions. (e.g. within each region,
    the color is the same.)
                                Camshift v3a            5
      First we need to
       understand the
   Probability Density
         Function PDF
                        We use
Kernel density estimation to find
                            PDF



                Camshift v3a        6
Motivation for Kernel density
estimation to find PDF
   The formula (parametric form) of the PDF
    (probability density function) is difficult to find.
   Use sampling method to estimate the P.D.F.
   That means: Gaussian ( a parametric form with
    mean , standard deviation etc., is easy to use),
    but it is too simple to model real life problems.
                                                                            PDF(x)


                         Too simple
                         to model
                         real life problems
                         1
                         ||x||2                                              0                   x
     K N ( x)  c  e    2
                                                             An irregular shape PDF, the distribution
    Gaussian distribution                     Camshift v3a   Is difficult to model using parameters
                                                                                                7

                                                             --use non-parametric methods instead
 Example
     Outbreak of flu in a year
     How do you model this PDF?
       CUHK Clinic
       Patients
       Number
100    Per day




                     3    6             9   12 month

                         Camshift v3a                  8
Kernel density estimation KDE Demo








http://parallel.vub.ac.be/research/causalModels/tutorial/kde.html

                                   Camshift v3a                     9
kernel density distribution
function
   The general form of a                            1 n  x  xi 
    kernel density distribution            fh ( x)  d  K 
                                            ˆ                     
                                                    nh i 1  h 
    function                               n  number of samples
   The Kernel (K) has many                h  window radius
    choices
                                           d  dimension
       Epanechnikov
                                           x  target position
       Uniform
                                           xi  samples
       Normal (Gaussian)



                            Camshift v3a                          10
Non-parametric methods
--Histogram
--Kernel density estimation (discuss here)
                                                General form of a Kernel density function
                                                                 1     n
                                                                            x  xi 

                                                       ˆ
                                                       fh ( x)  d
                                                                nh
                                                                       h 
                                                                     i 1
                                                                          K
                                                                                    
                                                     Add kernel
                                                                              Each one
                                                     functions
                                                                              is a kernel
                                                     to become
                                                                              function
                                                     the final P.D.F
                                                                              K[(x-xi)/h)]




              Histogram
                                                              Kernel density
                                     Camshift v3a                                       11
http://en.wikipedia.org/wiki/Kernel_density_estimation
The concept:
General form of a Kernel density function
The PDF= summation of many small kernel functions




                                                        ˆ ( x)  1
                                                                        n
                                                                             x  xi 
                                             The PDF    fh
                                                                nh d
                                                                        K h 
                                                                       i 1         
                                                          Kernel functions at
                                                          each sample point




    http://www.cs.cornell.edu/courses/cs664/2005fa/Lectures/lecture3.pdf
                                    Camshift v3a                                12
Kernel density in 2D (x1,x2 as axes)
example
                                                1        n
                                                               x  xi 
   E.g PDF of mosquitoes in
    CU.                         x2
                                      ˆ
                                      fh ( x)  d
                                               nh
                                                          h 
                                                        i 1
                                                             K
                                                                       
   At time t, if you can find
    one mosiquito at a
    location xi =(x1,x2)i, mark                     Each green dot has such a circle
    it green in the diagram.
   Each point is a density
    kernel function :                         h
   K[(x-xi)/h)]                             xi
   Say h=1 meter
   You can build up the
    final PDF of mosquitoes
    by summing all kernel
    functions as in the
    above equation fh(x)




                                                                                        x1
                                     Camshift v3a
                                                     PDF of mosquitoes in CU       13
Illustration of the cam-shift (mean-shift) idea
You want to find the location of a circle (fixed radius) to enclose
the biggest number of points.  The more points, the bigger
the PDF.


        Guess the peak is at xt=0
    1.    Draw a circle of radius h                x2
                                                      Mean-shift-vector
    2.    Find mean of points inside                  m(xt)         mean_loc(xt)
          window=mean_loc(xt)                                       of the green
    3.    Move the circle for xt+1 at mean_loc(xt)                  dots in the circle
    4.    m(t)=mean_loc(xt)-xt=mean-shift vector
    5.    t=t+1
                                                            h
        Repeat 1-5 , till m(t) is
         small enough                                          x

        X is the result. (peak)

                                     Camshift v3a                          x1   14
Example 1
Show graphically of mean-shit
                                ˆ ( x)  1
                                                 n
                                                      x  xi 
   Starting from x
                         x2
                                fh
                                        nh d
                                                 K h 
                                                i 1         
    and the circle
    shown                                                         x’
   Find the peak of                   h
    the PDF.                            x

   Repeat the task
    if your first
    selection is at x’




                              Camshift v3a
                                             PDF of mosquitoes in CU x1   15
How about the kernels
         “K” (內核 )?
            Building blocks




           Camshift v3a       16
                                                          

                                                           K ( w)dw  1
                                                          


 Kernel choices (內核)K (w)  K (w) for all values of w.
    Different radial symmetric kernel profiles
Epanechnikov

                        c1 || w ||2      || w || 1
             K E ( w)  
                         0              otherwise

Uniform
                        c            || w || 1
             KU ( w)  
                       0              otherwise


                                           1
Normal                                     ||w||2
(Gaussain)     K N ( w)  c  e            2

                                         Camshift v3a                      17
                       Epanechnikov

                      c1 || w ||2 
Exercise 1 K E ( w)                      || w || 1
                       0              otherwise
 Draw KE(w), c=1

                                            KE(w)




                                                               w
               -1 -0.75 -0.5 -0.25 0         0.25 0.5 0.75 1

                             Camshift v3a                          18
                                                     x  xi
                                                                2
                                                                            x  xi
                                     x  xi  c1                               1
                                 KE                 h                    h
                                       h                        
Exercise 2                                           0                 otherwise

   Replace w in KE(w), by (||([x-xi]/h||)
   Draw KE(||[x-xi]/h||), c=1 when
       Xi=0 and h=2 when ||([x-xi]/h||<1
       Xi=3 and h=3 when ||([x-xi]/h||<1


                 1    KE([x-xi]/h),




                                                     x
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

                                      Camshift v3a                                       19
Exercise 3
   Each sample has
    an Epanechnikov
    distribution
   Estimate h=?
                        ˆ
                        f h ( x)
   n=?
   Draw the final                                    Each sample xi is an
    PDF fˆh ( x)                                      Epanechnikov
                                                      distribution

                                                      Only show 1-dimension
                                                  x
                                   xi      Xi+1
                      0 25 50 75…
                            Camshift v3a                              20
  Go back to the PDF
Find the peak of PDF by finding x
                when PDF(x)=0




                 Camshift v3a       21
Motivation:
Kernel Density (gradient) Estimation
   The peak of a Probability distribution (PDF)
    P(x) is difficult to find,
   so we find the gradient dP(x)/dx=P(x)
   And when P(x)=0 , it is at the peak of P(x)




                       Camshift v3a                22
If you have the PDF, you can
find the peak (mode), how?
   If you know the formula of the PDF, you can
    find the mode (peak). I.e. peak=max (pdf)
            peak




   IF max (pdf) is difficult to find,
   find gradient (pdf)=0 : so (PDF)=0
               peak
                                      (PDF)=0


                       Camshift v3a               23
Step1: Model the PDF
 The Kernel density if                          h
               ck,d n  x  xi 
  Pn ,k ( x )  d  K         
               nh i 1  h 
  n  number of samples
  h  size of window
  ck,d is the normalization constant
  we need to find x when
                       n
               ck,d
 Pn ,k ( x )  d
               nh
                       K  x  x   0
                      i 1
                                            i

                             Camshift v3a           24
Exercise 4
Derivation exercise
                             ck,d n  x  xi 
     Staring from Pn ,k ( x)  d  K             (i )
                              nh i 1  h 
     Choose Radial SymmetricKernel profile, the form is
       x  xi      x  xi           2
                                                                                  n    x  xi   2
                                                                                                      
     K          k                      , so (i) becomes Ph , K ( x)  ck,d    k h              
       h           h                                                   nh d                      
                                                                                i 1               
              x  xi 2         x  xi 2 
     Given g             k           ,
              h                h        
                                        
     The exerciseis to show the following relation is true.
                                                                 
                                     xi  x g  x  xi
                                 n                            2
                     2ck ,d
     Ph ,k ( x) 
                     nh d  2
                                                h
                                                                  
                                                                  
                                i 1                             


                                               Camshift v3a                                               25
Derivation
                             c           n
                                                  x  xi 
    Staring from Pn ,k ( x)  k,d
                             nh d
                                        K
                                          
                                        i 1           h 
                                                                (i )

                      ck,d n  x  xi   ck,d
   Ph ,k ( x)   d  K 
                      nh                       ,
                                                             is _ constant
                                     h   nh
                                                           d
                            i 1


                x  xi         x  xi 2 
    since K              k   h
                                             ,
                                             
                  h                       
                               x  x 2  
                             d  k           i    
    dPh ,k ( x) ck,d n    h                      
                  d                            
                                                         , apply chain rule
       dx          nh i 1            dx               
                                                       
                                                       
                                                       
                              n  x  x 2                           2 
                                                                            
                             d  k
                                                    i       d  x  xi  
    dPh ,k ( x) ck,d n    i 1  h
                                         
                                                             h
                                                            
                                                                            
                                                                                 (ii)
                  d       
       dx          nh i 1           x  xi     2
                                                                    dx       
                                   d                                     
                                     h                                   
                                                                         



                                                              Camshift v3a                   26
Continue
                              x  x 2 
                           d k       i    
             x  xi 2       h           
replace  g                            
                                                    (ii),
             h                x  xi 2
                           d            
                                h         
                                          
   x  xi 2                       x  xi 2 
g             -ve gradient of k            ,
   h                              h         
                                            
put in (ii)
                       n  x  x 2                              
                                                  xx
                                                                   2
                      d  k                                         
                             h i   d h i
             ck,d n    i 1 
                      
                                             
                                             
                                                                       
                                                                            (iii)
Ph ,k ( x)  d 
             nh i 1         x  xi 2         dx                    
                         d                                          
                             h                                      
                                                                    
                       n  x  x 2                     
                      d  k                            
                             h     i

             ck,d n    i 1 
                                           2
                                                             
Ph ,k ( x)  d                                    x  xi , put (ii) into (iii)
             nh i 1         x  xi 
                                       2
                                               h 2
                                                             
                         d                              
                             h                          
                                                        
                             xx                                                              
                                                                          x  x g  x  xi
                      n                2                               n                        2
             2ck,d                          x  xi   2ck,d
Ph ,k ( x)  d  2
             nh
                         g h i
                                                    nh d  2    i               h
                                                                                                           (iv)
                                                                                                    
                      i 1
                                                               i 1
                                                                                                  
                                                          Camshift v3a                                                    27
Continue
                                                                     
                                   x  x g  x  xi
                                 n                                 2
             2ck,d                                                            (iv)
Ph ,k ( x)  d  2
             nh
                              i              h                      
                             i 1
                                                                     
                    n             x  xi                                                                            
                                                                    , make it easier to read g  x  xi
                                                              2                                                      2
 2ck,d
 d  2  xi  x    xi  x g                                                                                         g  , so
 nh                               h                                                           h                     
                  i 1
                                                                                                                    
                                               2ck,d n
                                              
                                   x  x g  x  xi
                                 n                                 2
             2ck,d
Ph ,k ( x)  d  2           i              
                                                nh d  2  i
                                               h
                                                                x  x g  ,
             nh              i 1
                                             
                                                          i 1

             2ck,d  n                      2ck,d  n
                                                                                                             
                                                                          n
Ph ,k ( x)  d  2   xi g    xg    d  2    xi g      xg                                 
             nh  i 1                      nh  i 1                  i 1                                     
do some math tricks
                n                                                                      n
                                                                                                                                               
                g                                                                  g                                                    
                                                   xg   
                                 n                    n                                                      n                           n
    2ck,d                                                                  2ck,d             
                          xi g                                                                            xi g    n          xg  ,
                                                                                                        1                        1
              i 1
                                                                                     i 1
    nh d  2     n
                                                                           nh d  2               n                                            
                g    i 1                                          
                                                                                                      g   i 1              g   i 1
                                                     i 1                                    1
                                                                                                                                                
               i 1                                                                               i 1                     i 1                
                                          n                  n
x is independent of i,  xg                          x g  
                                         i 1               i 1

                                                n
                                                           
                                     n
                                           xi g   
                      2ck,d
Ph ,kX ( x) 
                      nh d  2
                                  g   i 1 n
                                        
                                                        x        (v )
                                                           
                                 i 1
                                               g       
                                         i 1                     Camshift v3a                                                                     28
Exercise 5
Kernel Density (gradient) Estimation
    Exercise 5a :
                                                          n                
                                                   n
                                                            xi g   
                                       2ck,d
    Recall (v)        Ph ,kX ( x) 
                                       nh d  2
                                                   g   i1 n
                                                         
                                                                         x       (v )
                                                                            
                                                           g 
                                                  i 1
                                                                                               Mean-shift-vector
                                                          i 1             
                       x  xi 2 
    where g    g i           
                       h        
                                
    Show the graident of Ph ,kX ( x) is
                                                       n         x  xi 2  
                                                        xi g i 
                                                                  h
                                                                             
                                                                             
                   2ck ,d  n            x  xi   i 1
                                                 2

     Ph ,kX ( x)  d  2  g i                                          x    (vi)
                   nh  i 1               h        n  x  x 2             
                                                                            
                                                        i h
                                                            g         i  
                                                                                
                                                       i 1                   
    Exercise 5b :
                   n  x  x 2 
    Show the term  g i     i  is a positive constant w using the Epanechnikov kernel .
                                                             hen
                          h
                   i 1       
                               
    Given the Epanechnikov kernel as follows, c is a normalization positive constant,assumec  1
                                                
                       c1  x  xi                      x  xi
                                             2                      2
        x  xi   2
                                                                       1
    kE                      h                          h
        h                                                                Camshift v3a                           29
                      
                              0                      otherwise
  Finding Mean-Shift-vector

                                               n        x  xi 2      
                                                xi gi 
                                                         h
                                                                   
                                                                        
        2ck ,d      n          x  xi   i 1
                                         2
                                                                    x   (iii)
P( x )  d 2        gi 
                                           
                                             n  x x 2               
         nh          
                                  h       
                                            
                                                  h  
                       i 1
                                                    gi       i
                                               i 1                   
                                                                       
          Always Positive

                                   Mean-shift-vector
Simple Mean Shift procedure:
Compute mean-shift-vector
                 n        x  xi 2     
                  xi gi 
                                    
                                         
                 i 1     h          x
       m( x )                           
                     n   x  xi 2 
                  gi                  
                 i 1  h              
                                        
•Translate the Kernel window by m(x)          Camshift v3a                     30
The gradient of PDF  P(x) is proportional
to mean-shift vector m(x)


     P( x)  m( x)
    because
                                              n        x  xi 2     
                                               xi gi 
                                                        h
                                                                  
                                                                      
             2ck ,d   n        x  xi 
                                        2
                                             i 1                x
    P( x )  d 2      gi 
                             
                                          
                                            n  x x 2              
             nh        i 1 
                                  h       
                                          
                                                  gi  h i  
                                              i 1                  
                                                                    
                             +ve
                                                       mean shift
                                                       vector m(x)
                                                            n        x  xi 2     
                                                             xi gi 
                                                                               
                                                                                    
                                                            i 1       h        x
                                                  m( x )                           
                                                                n   x  xi 2 
                                                             gi                  
                                       Camshift v3a         i 1  h
                                                                  
                                                                              
                                                                                    
                                                                                        31
P(a) points to the increasing
side of P(x)
   Taylor series, definition
   P(x)=P(a)+ P(a)(x-a)+…
   At the left hand side of the peak
       At P(a), bringing (a) to (x) will increase
        p(x).                                                           Peak
       (x-a) is +ve , and  P(a) is +ve and is          P(x)
        pointing to the direction of increase of
        P(x)
   At the right hand side of the peak
       At P(a’) , bringing (a’) to (x’) will
        increase p(x).
       (x’-a’) is -ve , and  P(a) is -ve and is
        pointing to the direction of increase of
        P(x)                                                                                      x
       So  P(a) is pointing towards the                           a     x        x’   a’
        increase of P(x)
       Since  P(a) proportional to m(x)                   left side of peak
                                                                                right side of peak
       so m(x) points to the direction of
        increase of P(x).




                                                    Camshift v3a                             32
       Mean-shift
Algorithm and procedure




        Camshift v3a      33
Procedures
    To find the peak of PDF
                                           Gradient of PDF                                      n          x  xi 2     
    For gradient of PDF = P                                                                      xi gi 
                                                                                                            h
                                                                                                                      
                                                                                                                          
                                                                2c       n       x  xi   i 1
                                                                                          2
                                                                                                                       x
        The first term is a PDF , it is positive. P( x )  d ,d2  gi 
                                                              k                             
                                                            nh  i 1                        n  x x 2                
                                                                                   h       
                                                                                             
        The second tern m(xt) always points to
         the maximum PDF direction (proof in                                                      gi 
                                                                                                i 1  h
                                                                                                                 i  
                                                                                                                    
                                                                                                                           
                                                                                                                           
         previous slides [1])                                                                                           
    So the peak can be found by                                          +ve                          or
                                                                                         mean shift vect
        First guess at x(t=0)                                                                     n       x  xi 2     
        iterate followings until Xt’+1=xt’ (meaning                                                xi g 
                                                                                                            h
                                                                                                                      
                                                                                                                          
         m(xt) too small)                                                                          i 1               x
                                                                                         m( x )                          
                                                                                                                    
                                                                                                    g  x  xi 
                                                                                                       n           2
         1.  Find m(xt)                                                                                                    
                                                                                                   i 1  h
                                                                                                         
                                                                                                                     
                                                                                                                          
         2. Xt+1=xt + m(xt)                                  P(x)                                                         
         3. Increment t                                                       Peak
        Xpeak = xt’
    Same as
    x(t+1)=x(t) +  P(x)
    Since  P(x) m(xt)
                                                                     m(xt) m(xt+1)
                                                                                                Xpeak(t=t’)               x
                                              Camshift v3a           xt        xt+1 xt +2                          34
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust Approach Toward Feature Space Analysis"
Volume 24 , Issue 5 (May 2002),IEEE Transactions on Pattern Analysis and Machine Intelligence
A target tracking tool using
mean-shift: Camshaft
   Demo opencv video




    http://www.youtube.com/watch?v=iBOlbs8i7Og
                                 Camshift v3a    35
                Cam shift
An algorithm for object tracking




               Camshift v3a        36
Motivation
   For target tracking , mean shift can be used.
   Implementing the full mean shift algorithm is too
    complex
   It is found that the implementation can be found by
    zero-moment and first-moment
   We will show how
   Ref: Gary R. Bradski, “Computer Vision Face
    Tracking For Use in a Perceptual User”,
    Microcomputer Research Lab, Santa Clara, CA,
    Intel Corporation
                          Camshift v3a                    37
Overview
1.        Color features for tracking
2.        Object tracking by cam-shift
     1.     Step1: find object color model (histogram)
     2.     Step2: use the model to track using mean-shift, use
            histogram-back-project to identify the object for tracking
3.        Illustration of the cam-shift idea
4.        Demonstration
5.        Applications
     1.     Face tracking
     2.     Object tracking
                                   Camshift v3a                          38
(1) Color features for tracking

   Color processing
   Recall, hue =色調
   From 0 to 360 degrees
   Encode values:01




0            0.5        1




                            Camshift v3a   39
2) Object tracking
Step1
Procedure to find object color model histogram, example

   Learn object color distribution
    e.g. face color, hue is enough to
    identify face
   Circle face by hand, keep hue
    image only: Hue =07 levels
   Find model face histogram of
    Hue color inside
   E.g. 8 bins
                                          I(x,y)=
                 counts
                                          probility_of_face_color(Hue of the pixel)


                           Camshift v3a         Hue                         40

              Histogram
Example to find model
                                                e.g.
and histogram back project                      Pixel           x264x64
                                                I(12,34)
   Color histogram Model building              Hue is 3
   Image size M=64,N=64. Total=(MxN)                                  x1
   For the hue image, each pixel value rk =07. (In
    practice, use 0255)                               I(rk)
   The number of pixels having hue_value rk is nk
   Use 8 bins (buckets). (In practice, use 32 bins)
   Histogram back project Phist[I(x,y)]
   You have a pixel rk =I(x1,x2) (with hue color rk.)
   Phist[I(x1,x2)] is the probability this pixel                         Hue
    belonging to a face
                                                                    nk
   E.g. rk =I(12,34)=3 (see next slide for the        Phist(rk ) 
    histogram)                                                      MN
   Read up Phist(rk =3) from histogram ,              histogram backproject
   Phist[3] =0.160
                                                            Phist I(x1,x2 )
                                 Camshift v3a                                    41
                                                 nk
                                    Phist(rk ) 
                                                 MN
Example                             histogram backproject
   A numerical exercise             Phist I(x1,x2 )
    rk     Ik      P_hist(rk)
                                                         Histogram(8-bins)

                                               0.3
    r0=0    790      0.192871094
                                               0.2
    r1=1   1023      0.249755859
    r2=2    850      0.207519531               0.1
    r3=3    656       0.16015625                0
    r4=4    329      0.080322266                     1   2   3   4   5   6   7   8
    r5=5    245      0.059814453
    r6=6    122      0.029785156
    r7=7      87     0.021240234
    m         64
    n         64
    L          8
    sum     4102




                                Camshift v3a                                         42
                                                                                     42
2) Object tracking
step2:
Recall: mean-shift procedures to find the peak of PDF
                                           Gradient of PDF                                      n          x  xi 2     
    For gradient of PDF = P                                                                      xi gi 
                                                                                                            h
                                                                                                                      
                                                                                                                          
                                                                2c       n       x  xi   i 1
                                                                                          2
                                                                                                                       x
        The first term is a PDF , it is positive. P( x )  d ,d2  gi 
                                                              k                             
                                                            nh  i 1                        n  x x 2                
                                                                                   h       
                                                                                             
        The second tern m(xt) always points to
         the maximum PDF direction (proof in                                                      gi 
                                                                                                i 1  h
                                                                                                                 i  
                                                                                                                    
                                                                                                                           
                                                                                                                           
         previous slides [1])                                                                                           
    So the peak can be found by                                          +ve
                                                                                         mean shift vector
        First guess at x(t=0)                                                                     n        x  xi 2     
        iterate followings until Xt’+1=xt’ (meaning                                                xi gi 
                                                                                                             h
                                                                                                                       
                                                                                                                           
         m(xt) too small)                                                                          i 1                x
                                                                                         m( x )                           
                                                                                                                     
                                                                                                    gi  x  xi 
                                                                                                       n            2
         1.  Find m(xt)                                                                                                     
                                                                                                   i 1  h              
                                                                                                                          
         2. Xt+1=xt + m(xt)                                  P(x)
         3. Increment t                                                       Peak
        Xpeak = xt’
    Same as
    x(t+1)=x(t) +  P(x)
    Since  P(x) m(xt)
                                                                     m(xt) m(xt+1)
                                                                                                Xpeak(t=t’)                x
                                              Camshift v3a           xt        xt+1 xt +2                           43
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust Approach Toward Feature Space Analysis"
Volume 24 , Issue 5 (May 2002),IEEE Transactions on Pattern Analysis and Machine Intelligence
Weighted kernel density for
image tracking
   we showed previously that
      x  xi 2 
    g            is a positive constant
      h        
               
    For computer v   ision tracking
                         x  xi
                                       2

    add a weight wi to g
                           h
          xx                          x  xi
     n                 2       n                2

     xi g h i
    i 1
                             xi wi g
                             i 1        h
          xx                                    
                             wi g  x  xi
     n                2         n               2

     g h i
         
    i 1 
                                     h
                                                    
                                                    
                             i 1                 
      x  xi
             2

    g        is a constant(for simplicity, make it 1)
        h
    The weight wi is actually the probabilit y Phist [I(x1,x2 )] that
    this pixel I(x1,x2 ) has the color belonging to a face, so
         x  xi
                  2

    wi g               Phist [I(x1,x2 )]
           h
                                                        Camshift v3a    44
What is moment in computer
vision?
   In mechanics, it is moment=length * mass.



            3Kg                                   2Kg

                  2m                3m
                         Position ‘o’
        •M1=First Moment                        •M1=First Moment
        on this side about                      on this side about
        position ‘o’                            position ‘o’is 2x3
        is 3x2 Kgm                              Kgm
        •M0=Zero moment                         •M0=Zero moment
        is 3Kg, no length                       is 2Kg, no length
        involved                                involved

                                 Camshift v3a                        45
In computer vision it                    I(x,y),                 x2
                                         the probability of this
is the pixel intensity *                 pixel has
length                                   the color of skin
                                         is Phist[I(x1,I2)]                           x1
   If only intensity is considered     The zeroth moment
                                        M 00   I ( x1, x 2)
                                                 x1 x 2

                                        The first moment for the horizontal direction
                                        M 10   x1  I ( x1, x 2)
                                                 x1 x 2

                                        The first moment for the vertical direction
                                        M 01   x 2  I ( x1, x 2)
                                                 x1 x 2


   In our application, we want to      The zeroth moment
    know the probability of a           M 00   Phist I ( x1, x 2) 
    pixel that has the color of                  x1 x 2

    skin, use histogram back            The first moment for the horizontal direction
    project Phist[I(x1,I2)] instead     M 10   x1  Phist I ( x1, x 2) 
                                                 x1 x 2
    of I(x1,I2)
                                        The first moment for the vertical direction
                                        M 01   x 2  Phist I ( x1, x 2) 
                                                 x1 x 2
                                      Camshift v3a                                         46
Find first, zero moment in
mean shift vector m(x)
                                   Histogram_back_project :Phist[I(x1,x2)]
  Find the zeroth moment                                                                      First moment
                                          x  xi 2 
  M 00   Phist [ I ( x1, x 2)]  wi g            (in mean - shift)
                                          h                                       n                      
                                                                                               x  xi
                                                                                                      2
                                                   
                                                                                     xi wi g
         x y
                                                                                                            
  First momentsfor horizontal (x1 ) and vertical(x2 ) directions                                h          
  M 10   x1  Phist [ I ( x1, x 2)];
                                                                          m( x )   i 1                 x
                                                                                       n     x  xi 2 
           x   y                                                                     wi g               
  M 01   x 2  Phist [ I ( x1, x 2)];                                            i 1    h            
                                                                                                         
           x   y

  Find the mean search window , centered at ( x1c , x 2c )
                                                                                       Zero moment
       M           M
  x1c  10 ; x 2c  01 ;
       M 00        M 00
  The mean shift vector m(x)
              M        M        
  m(x1,x2 )   10  x1, 01  x 2
               M 00    M 00     
    m(x)=Mean-shift-vector
    in the mean-shift algorithm
                                                Camshift v3a                                             47
The Cam-shift algo.
[xt=(x1,x2)t]
        Recall : m(x1,x2)=[(M10/M00), (M01/M00)]-[x1,x2]
        ---The camshift algorithm --
        First guess at x(t=0) =[x1,x2]t=0 .
        Form a search window centered at xt=0
    1.     Calculate M10,M01,M00 of the search window
    2.     Find mean_loc(xt)=[(M10/M00),(M01/M00)]t
    3.     Find mean-shift-vector m(xt)=mean_loc(xt)-[x1,x2]t
    4.     New search window will center at mean_loc(xt)
    5.     t=t+1;                                          Recall the mean-shift algo.
    6.     Loop to 1 until m(xt) < threshold               The peak can be found by
        Xpeak = [x1,x2]t                                       First guess at x(t=0)
                                                                iterate followings until
                                                                 Xt’+1=xt’ (meaning m(xt)
                                                                 too small)
                                                                 1. Find m(xt)
                                                                 2. Xt+1=xt + m(xt)
                                                                 3. Increment t
                                        Camshift v3a            Xpeak = xt’          48
(3) Illustration of the cam-shift (mean-shift) idea
You want to find the location of a circle (fixed radius) to enclose
the biggest number of points.  The more points, the bigger
the PDF.


        Guess the peak is at xt=0
    1.    Draw a circle of radius h                x2
                                                      Mean-shift-vector
    2.    Find mean of points inside                  m(xt)         mean_loc(xt)
          window=mean_loc(xt)                                       of the green
    3.    Move the circle for xt+1 at mean_loc(xt)                  dots in the circle
    4.    m(t)=mean_loc(xt)-xt=mean-shift vector
    5.    t=t+1
                                                            h
        Repeat 1-5 , till m(t) is
         small enough                                          x

        X is the result. (peak)

                                     Camshift v3a                          x1   49
(4) Demonstration of result
Animation of tracking
   Demo




                                  Camshift v3a                  50

    http://il.youtube.com/watch?v=Mfo_wE17bog&feature=related
Appendix




Camshift v3a   51
(5) Rotation tracking





               Camshift v3a   52
(7) Applications: Head motion
tracking





               Camshift v3a     53
Compare mean shift vector and zero,
first moments
   The general Mean shift                     The new Cam shift (mean shift using
                                                zero-first-moment)
    algorithm                                  Recall in the previous slide
                                               M(x,y)=[(M10/M00), (M01/M00)]-[x,y]
   The peak is found by:                      ---The new camshift algorithm --
   First guess at x(t=0)                      First guess at x(t=0)
       iterate until m(xt)=0                  Iterate until m(xt)=0
                                                   Find m(xt)=[(M10/M00),(M10/M00)]t-[x,y]t
         Find m(xt)                               [x,y]t+1=[x,y]t + m(xt)
         Xt+1=xt + m(xt)                          [x,y]t+1=[x,y]t + [(M10/M00),(M10/M00)]t-
                                                    [x,y]t
       Xpeak = xt                                 [x,y]t+1=[(M10/M00),(M10/M00)]t
                                               Xpeak = [x,y]t
                                                The centriod window is shifted by
                                                [xc,yc]t= =[(M10/M00),(M10/M00)]t

                            For Camshift
                            X[x,y]

                                 Camshift v3a                                         54
Use excel to plot an Epanechnikov distribution
 (cut and paste to excel to view plot)
each cell is
=(1/(2*3.1416*$A$25*$A$25))*(1-(1*( ($A2-$B$25)^2 + (B$1-$C$25)^2)                          / (1*($A$25)*($A$25) )))


                         0.01   0.02     0.03     0.04        0.05         0.06       0.07      0.08     0.09        0.1     0.11
              0.01
               0.02
                       0.0396
                       0.0396
                              0.0396
                              0.0397
                                       0.0397
                                       0.0397
                                                0.0397
                                                0.0397
                                                            0.0397
                                                            0.0397
                                                                         0.0397
                                                                         0.0397
                                                                                    0.0397
                                                                                    0.0397
                                                                                              0.0397
                                                                                              0.0397
                                                                                                       0.0397
                                                                                                       0.0397
                                                                                                                 0.0397
                                                                                                                 0.0397
                                                                                                                           0.0397
                                                                                                                           0.0397
               0.03    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
               0.04    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0398    0.0398    0.0398
               0.05    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0398    0.0398   0.0398    0.0398    0.0398
               0.06    0.0397 0.0397   0.0397     0.0398
                                                0.0397      0.0397       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
               0.07    0.0397 0.0397   0.0397     0.0398
                                                0.0397      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398
                                                                                                           0.0398-0.0398   0.0398
               0.08    0.0397 0.0397   0.0397   0.0397
                                                   0.0397   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398
                                                                                                           0.0397-0.0398   0.0398
               0.09    0.0397 0.0397   0.0397   0.0398
                                                   0.0397   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
                                                                                                           0.0397-0.0397
                0.1    0.0397 0.0397   0.0397   0.0398
                                                   0.0396   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
                                                                                                           0.0396-0.0397
               0.11    0.0397 0.0397   0.0397   0.0398      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
                                                   0.0396                                                  0.0396-0.0396
               0.12    0.0397 0.0397   0.0397   0.0397      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
                                                   0.0395                                                  0.0395-0.0396
               0.13    0.0397 0.0397   0.0397   0.0397      0.0398       0.0398     0.0398 S120.0398   0.0398    0.0398    0.0398
               0.14    0.0397 0.0397   0.0397      0.0395
                                                0.0397      0.0397       0.0398     0.0398    0.0398       0.0395-0.0395
                                                                                                       0.0398    0.0398    0.0398


                                                            1
                                                                5
                                                                     9
                                                                                       S1




                                                                          13
               0.15    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0398    0.0398   0.0398    0.0398    0.0398




                                                                               17
               0.16    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0398    0.0398    0.0398
               0.17    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
               0.18    0.0396 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
               0.19    0.0396 0.0396   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
                0.2    0.0396 0.0396   0.0396   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
           Epanechnikov
           std      mean_x mean_y
           std      mean_x mean_y
                  2       0.1    0.1
                  2



                                                      Camshift v3a                                                                  55
Use excel to plot a Gaussian (or normal)
distribution
(cut and paste to excel to view plot)                                       each cell is
=(1/(2*3.1416*$A$25*$A$25))*EXP(-1*( ($A3-$B$25)^2 + (B$2-$C$25)^2)                           / (2*($A$25)*($A$25) ))


                        1          2          3          4          5            6        7          8          9         10         11
            1    6.3E-05   0.000107   0.000171 0.000257     0.000362      0.00048 0.000597 0.000698     0.000767   0.000791   0.000767
             2   0.000107   0.000182   0.000291 0.000437     0.000616     0.000817 0.001016 0.001188     0.001305   0.001346   0.001305
             3   0.000171   0.000291   0.000465 0.000698     0.000985     0.001305 0.001624 0.001898     0.002085   0.002151   0.002085
             4   0.000257   0.000437   0.000698 0.001048     0.001479     0.001959 0.002438 0.00285       0.00313   0.003229    0.00313
             5   0.000362   0.000616   0.000985 0.001479     0.002085     0.002762 0.003438 0.004019     0.004414   0.004554   0.004414
             6    0.00048   0.000817   0.001305 0.001959     0.002762     0.003659 0.004554 0.005324     0.005848   0.006033   0.005848
             7   0.000597   0.001016   0.001624 0.002438     0.003438     0.004554 0.005668 0.006626     0.007277   0.007509   0.007277
                                                 0.01
             8   0.000698   0.001188   0.001898 0.00285      0.004019     0.005324 0.006626 0.007747     0.008508   0.008778   0.008508
             9   0.000767   0.001305               0.00313
                                       0.0020850.008         0.004414     0.005848 0.007277 0.008508     0.009344   0.009641   0.009344
                                                                                          0.008-0.01
            10   0.000791   0.001346              0.003229
                                       0.0021510.006         0.004554     0.006033 0.007509 0.008778     0.009641   0.009947   0.009641
            11   0.000767   0.001305   0.002085 0.00313      0.004414                     0.006-0.008
                                                                          0.005848 0.007277 0.008508     0.009344   0.009641   0.009344
            12   0.000698   0.001188   0.001898 0.004
                                                   0.00285   0.004019                     0.004-0.006
                                                                          0.005324 0.006626 0.007747     0.008508   0.008778   0.008508
            13   0.000597   0.001016   0.001624 0.002
                                                  0.002438   0.003438                     0.002-0.004
                                                                          0.004554 0.005668 0.006626     0.007277   0.007509   0.007277
            14    0.00048   0.000817   0.001305 0.001959     0.002762             S13
                                                                          0.003659 0.004554 0.005324
                                                                                          0-0.002        0.005848   0.006033   0.005848
                                                     0
                                                     1


            15   0.000362   0.000616   0.000985 0.001479     0.002085     0.002762 0.003438 0.004019     0.004414   0.004554   0.004414
                                                         6

                                                                11             S1

                                                                     16
            16   0.000257   0.000437   0.000698 0.001048     0.001479     0.001959 0.002438 0.00285       0.00313   0.003229    0.00313
            17   0.000171   0.000291   0.000465 0.000698     0.000985     0.001305 0.001624 0.001898     0.002085   0.002151   0.002085
            18   0.000107   0.000182   0.000291 0.000437     0.000616     0.000817 0.001016 0.001188     0.001305   0.001346   0.001305
            19    6.3E-05   0.000107   0.000171 0.000257     0.000362      0.00048 0.000597 0.000698     0.000767   0.000791   0.000767
            20   3.48E-05   5.91E-05   9.45E-05 0.000142       0.0002     0.000265 0.00033 0.000386      0.000424   0.000437   0.000424

      std     mean_x mean_y
            4       10     10



                                                             Camshift v3a                                                            56
Answer: Exercise 1
   Draw KE(w), c=1
                        KE(w)              c1 || w ||2      || w || 1
                                K E ( w)  
                                            0              otherwise


                                                                      x              KE
                                                                                -1             0
                                                                             -0.75        0.4375        1.2
                                                                                                          1
                                                                              -0.5          0.75
                                                                                                        0.8
                                                                             -0.25        0.9375
                                                                                                        0.6           Series1
                                                                                 0             1
                                                                                                        0.4
                                                                              0.25        0.9375
                                                                                                        0.2
                                                                               0.5          0.75
                                                                                                          0
                                                                              0.75        0.4375
                                                                                                   -2         0   2
                                                                                 1             0




                                                                     x
-1 -0.75 -0.5 -0.25 0    0.25 0.5 0.75 1

                                            Camshift v3a                                                              57
Answer: Exercise 2
                                                                              x  xi       x  xi
   Draw KE([x-xi]/h), h=2,c=1                                x  xi  c1                       1
                                                          KE                 h            h
                                                              h  
    when                                                                     0          otherwise
                                                          xi=0,h=2
       Xi=0 when ||([x-xi]/h||<1                         x
                                                                 -5
                                                                     KE
                                                                        -5.25
                                                                              abs[(x-xi)/h]<1
                                                                                           2.5
                                                                 -4        -3                2
       Xi=3 when ||([x-xi]/h||<1                                -3     -1.25              1.5
                                                                                                               1.2
                                                                                                                 1
                                                                 -2         0                1                 0.8
                                                                 -1      0.75              0.5                 0.6

                 1     KE([x-xi]/h),                               0
                                                                   1
                                                                            1
                                                                         0.75
                                                                                             0
                                                                                           0.5
                                                                                                               0.4
                                                                                                               0.2
                                                                   2        0                1                   0
                                                                   3    -1.25              1.5      -5               0        5

                                                          xi=2,h=3
                                                          x            KE        abs[(x-xi)/h]<1
                                                                 -5     -4.44444 2.333333333
                                                                 -4           -3                2        1.2
                                                                 -3     -1.77778 1.666666667               1
                                                      x          -2
                                                                 -1
                                                                        -0.77778 1.333333333
                                                                               0                1
                                                                                                         0.8
                                                                                                         0.6
                                                                   0   0.555556 0.666666667              0.4
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6                                    1
                                                                   2
                                                                       0.888889 0.333333333
                                                                               1                0
                                                                                                         0.2
                                                                                                           0
                                                                   3   0.888889 0.333333333         -5         0         5    10
                                       Camshift v3a                                                                      58
Answer: Exercise 4a
Derivation                                                                     xx                         2
                                                                                                                 
                                                                              k     i                         
                     ck,d n  x  xi                 Ph ,K ( x ) ck,d n        h                            
                                                                                                                  , use chain rule
 From (i), P( x )  nh d  K  h , so Ph ,K ( x )  x  nh d  
                                                                      i 1       x
                                                                                                                  
                          i 1                                                                                    
                                                                                                                 
                                                                                                                 
                           x  x 2    x  x 2                                                x  x 2
                          k     i            i    x  xi                               k        i    
                 ck,d n      h         h                                             n       h           x  x 1
   Ph ,K ( x )  d                                                            ck,d                          2    i  
                                                                                               
                                                                h
                                                                                                                           
                 nh i 1   x  xi 2        x  xi  x                        nh d     i 1       x  xi    h  h 
                                                                                                                  2
                                                                                                               
                           h                h      
                                                                                   
                                                                                                    h             
                                                                                                             
                         x  x 2                                                 x  x 2
                        k     i                                                k       i    
                           h                                   x  xi 
                                                                            2
                                                                                       h         
                                                                                                   
              n
    2ck,d
    d 2    x  xi              
                                                      Given g 
                                                               
                                                                                
                                                                                       x  xi  
                                                                                                      
                         x  xi 
                                   2                                                             2
    nh      i 1                                                      h                        
                         h                                                      h           
                                                                                              
                                                            2ck,d                                  
                                x  xi g  x  xi                        xi  x g  x  xi
                           n                            2               n                         2
                 2ck,d
   Ph ,K ( x )  d  2
                 nh
                                            h
                                                              d 2
                                                             nh                       h
                                                                                                      
                                                                                                      
                                                                                                          done!
                          i 1                                       i 1                          

                                                             Camshift v3a                                                      59
Answer:Exercise 3
   Each sample has
    an Epanechnikov
    distribution
   Estimate h=?
                        ˆ
                        f h ( x)
   n=?
   Draw the final                                    Each sample xi is an
    PDF fˆh ( x)                                      Epanechnikov
                                                      distribution

                                                      Only show 1-dimension
                                                  x
                                   xi      Xi+1
                      0 25 50 75…
                            Camshift v3a                              60
Answer5a: Exercise 5
Kernel Density (gradient) Estimation
                                     2ck            n            x  xi           2
                                                                                        
  Starting from (ii ), Ph ,K ( x )  d ,d2       xi  x gi  h
                                                  i 1 
                                                                 
                                                                                        
                                                                                        
                                    nh                                               
    2ck ,d  n          
              xi gi  x  xi
                                          n                              
                                             x gi  x  xi
                                     2                                 2

    d 2                                                                  
    nh  i 1            h               i 1         h                 
                                                                       
               n    x  xi    2
                                
               gi  h
                   
                                
                                 n                                                              
                                   x g  x  xi                              
                                                                       x  gi  x  xi
                                                                2                             2
                   
                                                                               n
    2ck
                               2 
    d ,d2
              i 1
                                            
                                           i
                                                                                                  
    nh              x  xi       i 1
                                         i                                        h             
                                             h                                                 
                n

               gi  h
                                                                             i 1
                                
                               
              i 1             
                                       n         x  xi 2     
                                        xi gi                
              n        x  xi  i 1          h        
                                                              x
                                2
    2ck                                          
    d ,d2    gi 
                                  
                                                                              (done!)
    nh         i 1 
                          h      
                                    
                                          n      x  xi 2      
                                         hgi           
                                                                
                                       i 1                   
                                                 Camshift v3a                                          61
Answer5b: Exercise 5
Kernel Density (gradient) Estimation
                      n  x  x 2 
     Show theterm  gi       i   is positive.
                             h    
                      i 1 
                                  
      By definition Epanechnikov kernel isshown below,
       c is a normalization positive constant
                                               
                          c1  x  xi                  x  xi
                                             2                     2
           x  xi   2
                                                                      1
       kE                      h                      h
           h                                  
                         
                                 0                  otherwise
                                                                     x  x 2 
                                                                   k       i   
                     x  xi     2
                                                   x  xi 
                                                            2        h          
       Given that g                   k E                              
                                                                                     ,
                     h                              h              x  xi 2 
                                                                           
                                                                       h        
                                                                                
           x  xi 2 
       gi              2c   2c
           h        
                    
       so it is positive constant(done!)

                                             Camshift v3a                                62

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:2/1/2013
language:English
pages:62
xuxianglp xuxianglp http://
About