# iv07_camshift by xuxianglp

VIEWS: 0 PAGES: 62

• pg 1
```									  Image processing
and computer vision

Mean-shift and Cam-shift
Ref
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust
Approach Toward Feature Space Analysis"Volume 24 , Issue 5
(May 2002),IEEE Transactions on Pattern Analysis and Machine
Intelligence
[2]
web.missouri.edu/~hantx/ECE8001/notes/Lect7_mean_shift.pdf

Camshift v3a                          1
What is Mean-shift?
   Find the peak of a probability function by the
change of the mean of the data
   Applications:
   Segmentation
   Non-rigid object tracking

Camshift v3a             2
Applications: segmentation of
regions of images in a movie
   Use color to segment the image into logical
regions for analysis.
   If the regions are moving , mean-shift is
useful.

Camshift v3a                     3

http://www.youtube.com/watch?v=n1W0IwipRBQ&feature=related
Application: tracking non-rigid
object
   Human tracking

Camshift v3a         4

http://www.youtube.com/watch?v=zLtjPfPP9HY
Intuition: find the mode by
mean-shift
   Target : Find the modes (peaks) in a set
of sample data.
   The mode of a continuous probability
distribution is the peak.
   There may be multiple peaks.
   The method used is called mean-shift.
   By finding the shift of the mean, we can find
the mode (peak)
   It can be used to segment an image into
logical regions. (e.g. within each region,
the color is the same.)
Camshift v3a            5
First we need to
understand the
Probability Density
Function PDF
We use
Kernel density estimation to find
PDF

Camshift v3a        6
Motivation for Kernel density
estimation to find PDF
   The formula (parametric form) of the PDF
(probability density function) is difficult to find.
   Use sampling method to estimate the P.D.F.
   That means: Gaussian ( a parametric form with
mean , standard deviation etc., is easy to use),
but it is too simple to model real life problems.
PDF(x)

Too simple
to model
real life problems
1
 ||x||2                                              0                   x
K N ( x)  c  e    2
An irregular shape PDF, the distribution
Gaussian distribution                     Camshift v3a   Is difficult to model using parameters
7

--use non-parametric methods instead
Example
    Outbreak of flu in a year
    How do you model this PDF?
CUHK Clinic
Patients
Number
100    Per day

3    6             9   12 month

Camshift v3a                  8
Kernel density estimation KDE Demo



http://parallel.vub.ac.be/research/causalModels/tutorial/kde.html

Camshift v3a                     9
kernel density distribution
function
   The general form of a                            1 n  x  xi 
kernel density distribution            fh ( x)  d  K 
ˆ                     
nh i 1  h 
function                               n  number of samples
   The Kernel (K) has many                h  window radius
choices
d  dimension
   Epanechnikov
x  target position
   Uniform
xi  samples
   Normal (Gaussian)

Camshift v3a                          10
Non-parametric methods
--Histogram
--Kernel density estimation (discuss here)
General form of a Kernel density function
1     n
 x  xi 

ˆ
fh ( x)  d
nh
  h 
i 1
K

Add kernel
Each one
functions
is a kernel
to become
function
the final P.D.F
K[(x-xi)/h)]

Histogram
Kernel density
Camshift v3a                                       11
http://en.wikipedia.org/wiki/Kernel_density_estimation
The concept:
General form of a Kernel density function
The PDF= summation of many small kernel functions



ˆ ( x)  1
n
 x  xi 
The PDF    fh
nh d
 K h 
i 1         
Kernel functions at
each sample point

http://www.cs.cornell.edu/courses/cs664/2005fa/Lectures/lecture3.pdf
Camshift v3a                                12
Kernel density in 2D (x1,x2 as axes)
example
1        n
 x  xi 
   E.g PDF of mosquitoes in
CU.                         x2
ˆ
fh ( x)  d
nh
  h 
i 1
K

   At time t, if you can find
one mosiquito at a
location xi =(x1,x2)i, mark                     Each green dot has such a circle
it green in the diagram.
   Each point is a density
kernel function :                         h
   K[(x-xi)/h)]                             xi
   Say h=1 meter
   You can build up the
final PDF of mosquitoes
by summing all kernel
functions as in the
above equation fh(x)

x1
Camshift v3a
PDF of mosquitoes in CU       13
Illustration of the cam-shift (mean-shift) idea
You want to find the location of a circle (fixed radius) to enclose
the biggest number of points.  The more points, the bigger
the PDF.

        Guess the peak is at xt=0
1.    Draw a circle of radius h                x2
Mean-shift-vector
2.    Find mean of points inside                  m(xt)         mean_loc(xt)
window=mean_loc(xt)                                       of the green
3.    Move the circle for xt+1 at mean_loc(xt)                  dots in the circle
4.    m(t)=mean_loc(xt)-xt=mean-shift vector
5.    t=t+1
h
        Repeat 1-5 , till m(t) is
small enough                                          x

        X is the result. (peak)

Camshift v3a                          x1   14
Example 1
Show graphically of mean-shit
ˆ ( x)  1
n
 x  xi 
   Starting from x
x2
fh
nh d
 K h 
i 1         
and the circle
shown                                                         x’
   Find the peak of                   h
the PDF.                            x

   Repeat the task
if your first
selection is at x’

Camshift v3a
PDF of mosquitoes in CU x1   15
How about the kernels
“K” (內核 )?
Building blocks

Camshift v3a       16


 K ( w)dw  1


Kernel choices (內核)K (w)  K (w) for all values of w.
   Different radial symmetric kernel profiles
Epanechnikov

c1 || w ||2      || w || 1
K E ( w)  
 0              otherwise

Uniform
 c            || w || 1
KU ( w)  
0              otherwise

1
Normal                                     ||w||2
(Gaussain)     K N ( w)  c  e            2

Camshift v3a                      17
Epanechnikov

c1 || w ||2 
Exercise 1 K E ( w)                      || w || 1
 0              otherwise
 Draw KE(w), c=1

KE(w)

w
-1 -0.75 -0.5 -0.25 0         0.25 0.5 0.75 1

Camshift v3a                          18
      x  xi
2
        x  xi
 x  xi  c1                               1
KE                 h                    h
   h                        
Exercise 2                                           0                 otherwise

   Replace w in KE(w), by (||([x-xi]/h||)
   Draw KE(||[x-xi]/h||), c=1 when
   Xi=0 and h=2 when ||([x-xi]/h||<1
   Xi=3 and h=3 when ||([x-xi]/h||<1

1    KE([x-xi]/h),

x
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

Camshift v3a                                       19
Exercise 3
   Each sample has
an Epanechnikov
distribution
   Estimate h=?
ˆ
f h ( x)
   n=?
   Draw the final                                    Each sample xi is an
PDF fˆh ( x)                                      Epanechnikov
distribution

Only show 1-dimension
x
xi      Xi+1
0 25 50 75…
Camshift v3a                              20
Go back to the PDF
Find the peak of PDF by finding x
when PDF(x)=0

Camshift v3a       21
Motivation:
Kernel Density (gradient) Estimation
   The peak of a Probability distribution (PDF)
P(x) is difficult to find,
   so we find the gradient dP(x)/dx=P(x)
   And when P(x)=0 , it is at the peak of P(x)

Camshift v3a                22
If you have the PDF, you can
find the peak (mode), how?
   If you know the formula of the PDF, you can
find the mode (peak). I.e. peak=max (pdf)
peak

   IF max (pdf) is difficult to find,
   find gradient (pdf)=0 : so (PDF)=0
peak
(PDF)=0

Camshift v3a               23
Step1: Model the PDF
The Kernel density if                          h
ck,d n  x  xi 
Pn ,k ( x )  d  K         
nh i 1  h 
n  number of samples
h  size of window
ck,d is the normalization constant
we need to find x when
n
ck,d
Pn ,k ( x )  d
nh
 K  x  x   0
i 1
i

Camshift v3a           24
Exercise 4
Derivation exercise
                             ck,d n  x  xi 
Staring from Pn ,k ( x)  d  K             (i )
nh i 1  h 
Choose Radial SymmetricKernel profile, the form is
 x  xi      x  xi           2
                                       n    x  xi   2

K          k                      , so (i) becomes Ph , K ( x)  ck,d    k h              
 h           h                                                   nh d                      
                                                            i 1               
 x  xi 2         x  xi 2 
Given g             k           ,
 h                h        
                           
The exerciseis to show the following relation is true.
                 
xi  x g  x  xi
n                            2
2ck ,d
Ph ,k ( x) 
nh d  2
                h


i 1                             

Camshift v3a                                               25
Derivation
c           n
 x  xi 
Staring from Pn ,k ( x)  k,d
nh d
 K

i 1           h 
    (i )

 ck,d n  x  xi   ck,d
   Ph ,k ( x)   d  K 
 nh                       ,
            is _ constant
 h   nh
d
       i 1

 x  xi         x  xi 2 
since K              k   h
,

   h                       
   x  x 2  
 d  k           i    
dPh ,k ( x) ck,d n    h                      
 d                            
, apply chain rule
dx          nh i 1            dx               
                           
                           
                           
  n  x  x 2                           2 

 d  k
                i       d  x  xi  
dPh ,k ( x) ck,d n    i 1  h
       
   h
  

     (ii)
 d       
dx          nh i 1           x  xi     2
      dx       
       d                                     
         h                                   
                                             

Camshift v3a                   26
Continue
  x  x 2 
d k       i    
 x  xi 2       h           
replace  g                            
     (ii),
 h                x  xi 2
               d            
 h         
           
 x  xi 2                       x  xi 2 
g             -ve gradient of k            ,
 h                              h         
                                          
put in (ii)
  n  x  x 2                              
xx
2
 d  k                                         
  h i   d h i
ck,d n    i 1 
 
  
  

     (iii)
Ph ,k ( x)  d 
nh i 1         x  xi 2         dx                    
    d                                          
        h                                      
                                               
  n  x  x 2                     
 d  k                            
  h     i

ck,d n    i 1 
                      2

Ph ,k ( x)  d                                    x  xi , put (ii) into (iii)
nh i 1         x  xi 
2
h 2

    d                              
        h                          
                                   
  xx                                                              
  x  x g  x  xi
n                2                               n                        2
2ck,d                          x  xi   2ck,d
Ph ,k ( x)  d  2
nh
   g h i
                         nh d  2    i               h
       (iv)

i 1
                                    i 1
                          
Camshift v3a                                                    27
Continue
                                   
 x  x g  x  xi
n                                 2
2ck,d                                                            (iv)
Ph ,k ( x)  d  2
nh
 i              h                      
i 1
                                   
n             x  xi                                                                            
 , make it easier to read g  x  xi
2                                                      2
2ck,d
 d  2  xi  x    xi  x g                                                                                         g  , so
nh                               h                                                           h                     
i 1
                                                                                             
             2ck,d n

 x  x g  x  xi
n                                 2
2ck,d
Ph ,k ( x)  d  2           i              
  nh d  2  i
 h
x  x g  ,
nh              i 1
           
            i 1

2ck,d  n                      2ck,d  n

n
Ph ,k ( x)  d  2   xi g    xg    d  2    xi g      xg                                 
nh  i 1                      nh  i 1                  i 1                                     
do some math tricks
n                                                                      n
                                                  
 g                                                                  g                                                    
   xg   
n                    n                                                      n                           n
2ck,d                                                                  2ck,d             
   xi g                                                                            xi g    n          xg  ,
1                        1
              i 1
                      i 1
nh d  2     n
nh d  2               n                                            
 g    i 1                                          
     g   i 1              g   i 1
i 1                                    1

i 1                                                                               i 1                     i 1                
n                  n
x is independent of i,  xg                          x g  
i 1               i 1

        n

n
   xi g   
2ck,d
Ph ,kX ( x) 
nh d  2
 g   i 1 n

 x        (v )

i 1
       g       
 i 1                     Camshift v3a                                                                     28
Exercise 5
Kernel Density (gradient) Estimation
Exercise 5a :
 n                
                                                   n
   xi g   
2ck,d
Recall (v)        Ph ,kX ( x) 
nh d  2
 g   i1 n

 x       (v )

  g 
i 1
                   Mean-shift-vector
 i 1             
 x  xi 2 
where g    g i           
 h        
          
Show the graident of Ph ,kX ( x) is
 n         x  xi 2  
  xi g i 
 h
 
 
2ck ,d  n            x  xi   i 1
2

Ph ,kX ( x)  d  2  g i                                          x    (vi)
nh  i 1               h        n  x  x 2             
                                                  
  i h
g         i  
      
 i 1                   
Exercise 5b :
 n  x  x 2 
Show the term  g i     i  is a positive constant w using the Epanechnikov kernel .
hen
 h
 i 1       
             
Given the Epanechnikov kernel as follows, c is a normalization positive constant,assumec  1
                        
 c1  x  xi                      x  xi
2                      2
 x  xi   2
                     1
kE                      h                          h
 h                                                                Camshift v3a                           29
               
      0                      otherwise
Finding Mean-Shift-vector

 n        x  xi 2      
  xi gi 
 h

     
      2ck ,d      n          x  xi   i 1
2
            x   (iii)
P( x )  d 2        gi 

  n  x x 2               
nh          
             h       

  h  
i 1
gi       i
 i 1                   
                         
Always Positive

Mean-shift-vector
Simple Mean Shift procedure:
Compute mean-shift-vector
 n        x  xi 2     
  xi gi 
          
    
 i 1     h          x
m( x )                           
n   x  xi 2 
  gi                  
 i 1  h              
                        
•Translate the Kernel window by m(x)          Camshift v3a                     30
The gradient of PDF  P(x) is proportional
to mean-shift vector m(x)


P( x)  m( x)
because
 n        x  xi 2     
  xi gi 
 h

    
2ck ,d   n        x  xi 
2
  i 1                x
P( x )  d 2      gi 


  n  x x 2              
nh        i 1 
            h       

 gi  h i  
 i 1                  
                       
+ve
mean shift
vector m(x)
 n        x  xi 2     
  xi gi 
          
    
 i 1       h        x
m( x )                           
n   x  xi 2 
  gi                  
Camshift v3a         i 1  h
       

      
   31
P(a) points to the increasing
side of P(x)
   Taylor series, definition
   P(x)=P(a)+ P(a)(x-a)+…
   At the left hand side of the peak
   At P(a), bringing (a) to (x) will increase
p(x).                                                           Peak
   (x-a) is +ve , and  P(a) is +ve and is          P(x)
pointing to the direction of increase of
P(x)
   At the right hand side of the peak
   At P(a’) , bringing (a’) to (x’) will
increase p(x).
   (x’-a’) is -ve , and  P(a) is -ve and is
pointing to the direction of increase of
P(x)                                                                                      x
   So  P(a) is pointing towards the                           a     x        x’   a’
increase of P(x)
   Since  P(a) proportional to m(x)                   left side of peak
right side of peak
   so m(x) points to the direction of
increase of P(x).

Camshift v3a                             32
Mean-shift
Algorithm and procedure

Camshift v3a      33
Procedures
To find the peak of PDF
Gradient of PDF                                      n          x  xi 2     
    For gradient of PDF = P                                                                      xi gi 
 h

    
2c       n       x  xi   i 1
2
            x
   The first term is a PDF , it is positive. P( x )  d ,d2  gi 
k                             
nh  i 1                        n  x x 2                
               h       

   The second tern m(xt) always points to
the maximum PDF direction (proof in                                                      gi 
 i 1  h
i  



previous slides [1])                                                                                           
    So the peak can be found by                                          +ve                          or
mean shift vect
   First guess at x(t=0)                                                                     n       x  xi 2     
   iterate followings until Xt’+1=xt’ (meaning                                                xi g 
 h

    
m(xt) too small)                                                                          i 1               x
m( x )                          
           
  g  x  xi 
n           2
1.  Find m(xt)                                                                                                    
 i 1  h


     
2. Xt+1=xt + m(xt)                                  P(x)                                                         
3. Increment t                                                       Peak
   Xpeak = xt’
    Same as
    x(t+1)=x(t) +  P(x)
    Since  P(x) m(xt)
m(xt) m(xt+1)
Xpeak(t=t’)               x
Camshift v3a           xt        xt+1 xt +2                          34
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust Approach Toward Feature Space Analysis"
Volume 24 , Issue 5 (May 2002),IEEE Transactions on Pattern Analysis and Machine Intelligence
A target tracking tool using
mean-shift: Camshaft
   Demo opencv video

http://www.youtube.com/watch?v=iBOlbs8i7Og
Camshift v3a    35
Cam shift
An algorithm for object tracking

Camshift v3a        36
Motivation
   For target tracking , mean shift can be used.
   Implementing the full mean shift algorithm is too
complex
   It is found that the implementation can be found by
zero-moment and first-moment
   We will show how
   Ref: Gary R. Bradski, “Computer Vision Face
Tracking For Use in a Perceptual User”,
Microcomputer Research Lab, Santa Clara, CA,
Intel Corporation
Camshift v3a                    37
Overview
1.        Color features for tracking
2.        Object tracking by cam-shift
1.     Step1: find object color model (histogram)
2.     Step2: use the model to track using mean-shift, use
histogram-back-project to identify the object for tracking
3.        Illustration of the cam-shift idea
4.        Demonstration
5.        Applications
1.     Face tracking
2.     Object tracking
Camshift v3a                          38
(1) Color features for tracking

   Color processing
   Recall, hue =色調
   From 0 to 360 degrees
   Encode values:01

0            0.5        1

Camshift v3a   39
2) Object tracking
Step1
Procedure to find object color model histogram, example

   Learn object color distribution
e.g. face color, hue is enough to
identify face
   Circle face by hand, keep hue
image only: Hue =07 levels
   Find model face histogram of
Hue color inside
   E.g. 8 bins
I(x,y)=
counts
probility_of_face_color(Hue of the pixel)

Camshift v3a         Hue                         40

Histogram
Example to find model
e.g.
and histogram back project                      Pixel           x264x64
I(12,34)
   Color histogram Model building              Hue is 3
   Image size M=64,N=64. Total=(MxN)                                  x1
   For the hue image, each pixel value rk =07. (In
practice, use 0255)                               I(rk)
   The number of pixels having hue_value rk is nk
   Use 8 bins (buckets). (In practice, use 32 bins)
   Histogram back project Phist[I(x,y)]
   You have a pixel rk =I(x1,x2) (with hue color rk.)
   Phist[I(x1,x2)] is the probability this pixel                         Hue
belonging to a face
nk
   E.g. rk =I(12,34)=3 (see next slide for the        Phist(rk ) 
histogram)                                                      MN
   Read up Phist(rk =3) from histogram ,              histogram backproject
   Phist[3] =0.160
 Phist I(x1,x2 )
Camshift v3a                                    41
nk
Phist(rk ) 
MN
Example                             histogram backproject
   A numerical exercise             Phist I(x1,x2 )
rk     Ik      P_hist(rk)
Histogram(8-bins)

0.3
r0=0    790      0.192871094
0.2
r1=1   1023      0.249755859
r2=2    850      0.207519531               0.1
r3=3    656       0.16015625                0
r4=4    329      0.080322266                     1   2   3   4   5   6   7   8
r5=5    245      0.059814453
r6=6    122      0.029785156
r7=7      87     0.021240234
m         64
n         64
L          8
sum     4102

Camshift v3a                                         42
42
2) Object tracking
step2:
Recall: mean-shift procedures to find the peak of PDF
Gradient of PDF                                      n          x  xi 2     
    For gradient of PDF = P                                                                      xi gi 
 h

    
2c       n       x  xi   i 1
2
            x
   The first term is a PDF , it is positive. P( x )  d ,d2  gi 
k                             
nh  i 1                        n  x x 2                
               h       

   The second tern m(xt) always points to
the maximum PDF direction (proof in                                                      gi 
 i 1  h
i  



previous slides [1])                                                                                           
    So the peak can be found by                                          +ve
mean shift vector
   First guess at x(t=0)                                                                     n        x  xi 2     
   iterate followings until Xt’+1=xt’ (meaning                                                xi gi 
 h

    
m(xt) too small)                                                                          i 1                x
m( x )                           
           
  gi  x  xi 
n            2
1.  Find m(xt)                                                                                                     
 i 1  h              
                        
2. Xt+1=xt + m(xt)                                  P(x)
3. Increment t                                                       Peak
   Xpeak = xt’
    Same as
    x(t+1)=x(t) +  P(x)
    Since  P(x) m(xt)
m(xt) m(xt+1)
Xpeak(t=t’)                x
Camshift v3a           xt        xt+1 xt +2                           43
[1] Dorin Comaniciu, Peter Meer,"Mean Shift: A Robust Approach Toward Feature Space Analysis"
Volume 24 , Issue 5 (May 2002),IEEE Transactions on Pattern Analysis and Machine Intelligence
Weighted kernel density for
image tracking
   we showed previously that
 x  xi 2 
g            is a positive constant
 h        
          
For computer v   ision tracking
x  xi
2

add a weight wi to g
h
xx                          x  xi
n                 2       n                2

 xi g h i
i 1
  xi wi g
i 1        h
 xx                                    
   wi g  x  xi
n                2         n               2

 g h i

i 1 
           h


   i 1                 
x  xi
2

g        is a constant(for simplicity, make it 1)
h
The weight wi is actually the probabilit y Phist [I(x1,x2 )] that
this pixel I(x1,x2 ) has the color belonging to a face, so
x  xi
2

wi g               Phist [I(x1,x2 )]
h
Camshift v3a    44
What is moment in computer
vision?
   In mechanics, it is moment=length * mass.

3Kg                                   2Kg

2m                3m
Position ‘o’
•M1=First Moment                        •M1=First Moment
on this side about                      on this side about
position ‘o’                            position ‘o’is 2x3
is 3x2 Kgm                              Kgm
•M0=Zero moment                         •M0=Zero moment
is 3Kg, no length                       is 2Kg, no length
involved                                involved

Camshift v3a                        45
In computer vision it                    I(x,y),                 x2
the probability of this
is the pixel intensity *                 pixel has
length                                   the color of skin
is Phist[I(x1,I2)]                           x1
   If only intensity is considered     The zeroth moment
M 00   I ( x1, x 2)
x1 x 2

The first moment for the horizontal direction
M 10   x1  I ( x1, x 2)
x1 x 2

The first moment for the vertical direction
M 01   x 2  I ( x1, x 2)
x1 x 2

   In our application, we want to      The zeroth moment
know the probability of a           M 00   Phist I ( x1, x 2) 
pixel that has the color of                  x1 x 2

skin, use histogram back            The first moment for the horizontal direction
project Phist[I(x1,I2)] instead     M 10   x1  Phist I ( x1, x 2) 
x1 x 2
of I(x1,I2)
The first moment for the vertical direction
M 01   x 2  Phist I ( x1, x 2) 
x1 x 2
Camshift v3a                                         46
Find first, zero moment in
mean shift vector m(x)
Histogram_back_project :Phist[I(x1,x2)]
Find the zeroth moment                                                                      First moment
 x  xi 2 
M 00   Phist [ I ( x1, x 2)]  wi g            (in mean - shift)
 h                                       n                      
x  xi
2
          
  xi wi g
x y

First momentsfor horizontal (x1 ) and vertical(x2 ) directions                                h          
M 10   x1  Phist [ I ( x1, x 2)];
m( x )   i 1                 x
n     x  xi 2 
x   y                                                                     wi g               
M 01   x 2  Phist [ I ( x1, x 2)];                                            i 1    h            
                      
x   y

Find the mean search window , centered at ( x1c , x 2c )
Zero moment
M           M
x1c  10 ; x 2c  01 ;
M 00        M 00
The mean shift vector m(x)
M        M        
m(x1,x2 )   10  x1, 01  x 2
 M 00    M 00     
m(x)=Mean-shift-vector
in the mean-shift algorithm
Camshift v3a                                             47
The Cam-shift algo.
[xt=(x1,x2)t]
        Recall : m(x1,x2)=[(M10/M00), (M01/M00)]-[x1,x2]
        ---The camshift algorithm --
        First guess at x(t=0) =[x1,x2]t=0 .
        Form a search window centered at xt=0
1.     Calculate M10,M01,M00 of the search window
2.     Find mean_loc(xt)=[(M10/M00),(M01/M00)]t
3.     Find mean-shift-vector m(xt)=mean_loc(xt)-[x1,x2]t
4.     New search window will center at mean_loc(xt)
5.     t=t+1;                                          Recall the mean-shift algo.
6.     Loop to 1 until m(xt) < threshold               The peak can be found by
        Xpeak = [x1,x2]t                                       First guess at x(t=0)
   iterate followings until
Xt’+1=xt’ (meaning m(xt)
too small)
1. Find m(xt)
2. Xt+1=xt + m(xt)
3. Increment t
Camshift v3a            Xpeak = xt’          48
(3) Illustration of the cam-shift (mean-shift) idea
You want to find the location of a circle (fixed radius) to enclose
the biggest number of points.  The more points, the bigger
the PDF.

        Guess the peak is at xt=0
1.    Draw a circle of radius h                x2
Mean-shift-vector
2.    Find mean of points inside                  m(xt)         mean_loc(xt)
window=mean_loc(xt)                                       of the green
3.    Move the circle for xt+1 at mean_loc(xt)                  dots in the circle
4.    m(t)=mean_loc(xt)-xt=mean-shift vector
5.    t=t+1
h
        Repeat 1-5 , till m(t) is
small enough                                          x

        X is the result. (peak)

Camshift v3a                          x1   49
(4) Demonstration of result
Animation of tracking
   Demo

Camshift v3a                  50

http://il.youtube.com/watch?v=Mfo_wE17bog&feature=related
Appendix

Camshift v3a   51
(5) Rotation tracking


Camshift v3a   52
(7) Applications: Head motion
tracking


Camshift v3a     53
Compare mean shift vector and zero,
first moments
   The general Mean shift                     The new Cam shift (mean shift using
zero-first-moment)
algorithm                                  Recall in the previous slide
   M(x,y)=[(M10/M00), (M01/M00)]-[x,y]
   The peak is found by:                      ---The new camshift algorithm --
   First guess at x(t=0)                      First guess at x(t=0)
   iterate until m(xt)=0                  Iterate until m(xt)=0
   Find m(xt)=[(M10/M00),(M10/M00)]t-[x,y]t
 Find m(xt)                               [x,y]t+1=[x,y]t + m(xt)
 Xt+1=xt + m(xt)                          [x,y]t+1=[x,y]t + [(M10/M00),(M10/M00)]t-
[x,y]t
   Xpeak = xt                                 [x,y]t+1=[(M10/M00),(M10/M00)]t
   Xpeak = [x,y]t
    The centriod window is shifted by
[xc,yc]t= =[(M10/M00),(M10/M00)]t

For Camshift
X[x,y]

Camshift v3a                                         54
Use excel to plot an Epanechnikov distribution
(cut and paste to excel to view plot)
each cell is
=(1/(2*3.1416*\$A\$25*\$A\$25))*(1-(1*( (\$A2-\$B\$25)^2 + (B\$1-\$C\$25)^2)                          / (1*(\$A\$25)*(\$A\$25) )))

0.01   0.02     0.03     0.04        0.05         0.06       0.07      0.08     0.09        0.1     0.11
              0.01
0.02
0.0396
0.0396
0.0396
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.0397
0.03    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
0.04    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0398    0.0398    0.0398
0.05    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0398    0.0398   0.0398    0.0398    0.0398
0.06    0.0397 0.0397   0.0397     0.0398
0.0397      0.0397       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
0.07    0.0397 0.0397   0.0397     0.0398
0.0397      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398
0.0398-0.0398   0.0398
0.08    0.0397 0.0397   0.0397   0.0397
0.0397   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398
0.0397-0.0398   0.0398
0.09    0.0397 0.0397   0.0397   0.0398
0.0397   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
0.0397-0.0397
0.1    0.0397 0.0397   0.0397   0.0398
0.0396   0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
0.0396-0.0397
0.11    0.0397 0.0397   0.0397   0.0398      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
0.0396                                                  0.0396-0.0396
0.12    0.0397 0.0397   0.0397   0.0397      0.0398       0.0398     0.0398    0.0398   0.0398    0.0398    0.0398
0.0395                                                  0.0395-0.0396
0.13    0.0397 0.0397   0.0397   0.0397      0.0398       0.0398     0.0398 S120.0398   0.0398    0.0398    0.0398
0.14    0.0397 0.0397   0.0397      0.0395
0.0397      0.0397       0.0398     0.0398    0.0398       0.0395-0.0395
0.0398    0.0398    0.0398

1
5
9
S1

13
0.15    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0398    0.0398   0.0398    0.0398    0.0398

17
0.16    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0398    0.0398    0.0398
0.17    0.0397 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
0.18    0.0396 0.0397   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
0.19    0.0396 0.0396   0.0397   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
0.2    0.0396 0.0396   0.0396   0.0397      0.0397       0.0397     0.0397    0.0397   0.0397    0.0397    0.0397
Epanechnikov
std      mean_x mean_y
std      mean_x mean_y
2       0.1    0.1
2

Camshift v3a                                                                  55
Use excel to plot a Gaussian (or normal)
distribution
(cut and paste to excel to view plot)                                       each cell is
=(1/(2*3.1416*\$A\$25*\$A\$25))*EXP(-1*( (\$A3-\$B\$25)^2 + (B\$2-\$C\$25)^2)                           / (2*(\$A\$25)*(\$A\$25) ))

1          2          3          4          5            6        7          8          9         10         11
            1    6.3E-05   0.000107   0.000171 0.000257     0.000362      0.00048 0.000597 0.000698     0.000767   0.000791   0.000767
2   0.000107   0.000182   0.000291 0.000437     0.000616     0.000817 0.001016 0.001188     0.001305   0.001346   0.001305
3   0.000171   0.000291   0.000465 0.000698     0.000985     0.001305 0.001624 0.001898     0.002085   0.002151   0.002085
4   0.000257   0.000437   0.000698 0.001048     0.001479     0.001959 0.002438 0.00285       0.00313   0.003229    0.00313
5   0.000362   0.000616   0.000985 0.001479     0.002085     0.002762 0.003438 0.004019     0.004414   0.004554   0.004414
6    0.00048   0.000817   0.001305 0.001959     0.002762     0.003659 0.004554 0.005324     0.005848   0.006033   0.005848
7   0.000597   0.001016   0.001624 0.002438     0.003438     0.004554 0.005668 0.006626     0.007277   0.007509   0.007277
0.01
8   0.000698   0.001188   0.001898 0.00285      0.004019     0.005324 0.006626 0.007747     0.008508   0.008778   0.008508
9   0.000767   0.001305               0.00313
0.0020850.008         0.004414     0.005848 0.007277 0.008508     0.009344   0.009641   0.009344
0.008-0.01
10   0.000791   0.001346              0.003229
0.0021510.006         0.004554     0.006033 0.007509 0.008778     0.009641   0.009947   0.009641
11   0.000767   0.001305   0.002085 0.00313      0.004414                     0.006-0.008
0.005848 0.007277 0.008508     0.009344   0.009641   0.009344
12   0.000698   0.001188   0.001898 0.004
0.00285   0.004019                     0.004-0.006
0.005324 0.006626 0.007747     0.008508   0.008778   0.008508
13   0.000597   0.001016   0.001624 0.002
0.002438   0.003438                     0.002-0.004
0.004554 0.005668 0.006626     0.007277   0.007509   0.007277
14    0.00048   0.000817   0.001305 0.001959     0.002762             S13
0.003659 0.004554 0.005324
0-0.002        0.005848   0.006033   0.005848
0
1

15   0.000362   0.000616   0.000985 0.001479     0.002085     0.002762 0.003438 0.004019     0.004414   0.004554   0.004414
6

11             S1

16
16   0.000257   0.000437   0.000698 0.001048     0.001479     0.001959 0.002438 0.00285       0.00313   0.003229    0.00313
17   0.000171   0.000291   0.000465 0.000698     0.000985     0.001305 0.001624 0.001898     0.002085   0.002151   0.002085
18   0.000107   0.000182   0.000291 0.000437     0.000616     0.000817 0.001016 0.001188     0.001305   0.001346   0.001305
19    6.3E-05   0.000107   0.000171 0.000257     0.000362      0.00048 0.000597 0.000698     0.000767   0.000791   0.000767
20   3.48E-05   5.91E-05   9.45E-05 0.000142       0.0002     0.000265 0.00033 0.000386      0.000424   0.000437   0.000424

std     mean_x mean_y
4       10     10

Camshift v3a                                                            56
Answer: Exercise 1
   Draw KE(w), c=1
KE(w)              c1 || w ||2      || w || 1
K E ( w)  
 0              otherwise

x              KE
-1             0
-0.75        0.4375        1.2
1
-0.5          0.75
0.8
-0.25        0.9375
0.6           Series1
0             1
0.4
0.25        0.9375
0.2
0.5          0.75
0
0.75        0.4375
-2         0   2
1             0

x
-1 -0.75 -0.5 -0.25 0    0.25 0.5 0.75 1

Camshift v3a                                                              57
Answer: Exercise 2
      x  xi       x  xi
   Draw KE([x-xi]/h), h=2,c=1                                x  xi  c1                       1
KE                 h            h
 h  
when                                                                     0          otherwise
xi=0,h=2
   Xi=0 when ||([x-xi]/h||<1                         x
-5
KE
-5.25
abs[(x-xi)/h]<1
2.5
-4        -3                2
   Xi=3 when ||([x-xi]/h||<1                                -3     -1.25              1.5
1.2
1
-2         0                1                 0.8
-1      0.75              0.5                 0.6

1     KE([x-xi]/h),                               0
1
1
0.75
0
0.5
0.4
0.2
2        0                1                   0
3    -1.25              1.5      -5               0        5

xi=2,h=3
x            KE        abs[(x-xi)/h]<1
-5     -4.44444 2.333333333
-4           -3                2        1.2
-3     -1.77778 1.666666667               1
x          -2
-1
-0.77778 1.333333333
0                1
0.8
0.6
0   0.555556 0.666666667              0.4
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6                                    1
2
0.888889 0.333333333
1                0
0.2
0
3   0.888889 0.333333333         -5         0         5    10
Camshift v3a                                                                      58
Answer: Exercise 4a
Derivation                                                                     xx                         2

 k     i                         
ck,d n  x  xi                 Ph ,K ( x ) ck,d n        h                            
 , use chain rule
 From (i), P( x )  nh d  K  h , so Ph ,K ( x )  x  nh d  
                                       i 1       x

i 1                                                                                    
                                    
                                    
  x  x 2    x  x 2                                                x  x 2
 k     i            i    x  xi                               k        i    
ck,d n      h         h                                             n       h           x  x 1
Ph ,K ( x )  d                                                            ck,d                          2    i  
 
h
                                                                                  
nh i 1   x  xi 2        x  xi  x                        nh d     i 1       x  xi    h  h 
2
                                                                                  
  h                h      

                 h             
                                                                                    
  x  x 2                                                 x  x 2
 k     i                                                k       i    
    h                                   x  xi 
2
    h         
                                                             
n
2ck,d
 d 2    x  xi              
              Given g 

  
         x  xi  

  x  xi 
2                                                             2
nh      i 1                                                      h                        
  h                                                      h           
                                                                       
                2ck,d                                  
 x  xi g  x  xi                        xi  x g  x  xi
n                            2               n                         2
2ck,d
Ph ,K ( x )  d  2
nh
                  h
  d 2
 nh                       h


done!
i 1                                       i 1                          

Camshift v3a                                                      59
Answer:Exercise 3
   Each sample has
an Epanechnikov
distribution
   Estimate h=?
ˆ
f h ( x)
   n=?
   Draw the final                                    Each sample xi is an
PDF fˆh ( x)                                      Epanechnikov
distribution

Only show 1-dimension
x
xi      Xi+1
0 25 50 75…
Camshift v3a                              60
Answer5a: Exercise 5
Kernel Density (gradient) Estimation
2ck            n            x  xi           2

Starting from (ii ), Ph ,K ( x )  d ,d2       xi  x gi  h
i 1 



                                    nh                                               
2ck ,d  n          
   xi gi  x  xi
 n                              
    x gi  x  xi
2                                 2

 d 2                                                                  
nh  i 1            h               i 1         h                 
                                                            
n    x  xi    2

 gi  h


 n                                                              
   x g  x  xi                              
   x  gi  x  xi
2                             2

n
2ck
2 
 d ,d2
i 1

i

nh              x  xi       i 1
i                                        h             
 h                                                 
n

 gi  h
i 1

            
i 1             
 n         x  xi 2     
  xi gi                
n        x  xi  i 1          h        
  x
2
2ck                                          
 d ,d2    gi 

                                            (done!)
nh         i 1 
            h      

n      x  xi 2      
   hgi           
     
 i 1                   
Camshift v3a                                          61
Answer5b: Exercise 5
Kernel Density (gradient) Estimation
 n  x  x 2 
     Show theterm  gi       i   is positive.
 h    
 i 1 
             
By definition Epanechnikov kernel isshown below,
c is a normalization positive constant
                    
 c1  x  xi                  x  xi
2                     2
 x  xi   2
                     1
kE                      h                      h
 h                                  
               
      0                  otherwise
  x  x 2 
 k       i   
 x  xi     2
              x  xi 
2        h          
Given that g                   k E                              
,
 h                              h              x  xi 2 
                                                       
 h        
          
 x  xi 2 
gi              2c   2c
 h        
          
so it is positive constant(done!)

Camshift v3a                                62

```
To top