; Fuzzy Clustering
Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Fuzzy Clustering

VIEWS: 24 PAGES: 42

  • pg 1
									Fuzzy logic & systems for signal
           separation

         Presented by Ana Pérez-Neira

  Dept. of Signal Theory and Communications
        Technical University of Catalonia (UPC)
             Collaborating with the CTTC


         e-mail: anuska@gps.tsc.upc.edu
                     Jinan 2005                   1
Table of Contents

  Introduction
  Signal filtering: application to communication
   signals
  Signal classification: application to seismic
   waveform separation
  Conclusions



                      Jinan 2005
          Introduction:
Main features of fuzzy processing
    Fuzzy systems build up mathematical models from
     linguistic knowledge
    Statistical knowledge is not necessary, but can be
     incorporated: scalable design with the available
     information
    Fast acquisition and tracking
    Easy implementation & tunning of complex
     systems
    Robust design to uncertainties in the model

                          Jinan 2005
            Interference Canceller
     Sent CDMA
     Signal dk

     Narrow band
     Interference ik


     Received signal
     rk

rk=Akdk +ik + nk               Canceller


     Detected
     information



                       Jinan 2005
              Interference Canceller
        rk
                             + -
               Estimator/            -
                                     -         Estimation and
                detector     ik                subtraction solution

P1. Each type of interference requires a different filter
P2. Non-Gaussian noise is present


   The fuzzy filter will cope with
     - Problems P1 and P2
     - Low complexity and scalable solution in front of:
         Amount of information

         Computation affordable


                                  Jinan 2005
                         Interference Canceller
       Ri: IF x1 is A1i and x2 is A2i THEN y is Bi

x1  rk  rk 1              rk=dk +ik                           y  ik
        A11     A12 A13                                     B1      B2     B3
                                      Filtering
                                x1                                                      y
                  rk-1                               ymin           rk-1         ymax
    x1min r                 x1max
            k
                                                             rk      dk     ik
    x2  ik 1  rk 1
        A21     A22 A23                                     B1      B2     B3
                                     Prediction
                                x2                                                      y
               zk-1                                  ymin           rk-1         ymax
    x2min i                 x2max
           k-1

    x3  ik 2  rk 1                  Jinan 2005
                                                             ik-1           ik
                 Interference Canceller

         S
                 l  x1 , x 2 , x3          S
    i   il
    ˆ
                S
                                           il p  il / rl , i l 1 , ml  
        l 1
                l  x1 , x2 , x3 
               m 1
                                             l 1



      E il / rl , il 1
                                        Parzen


Features of the designed fuzzy filter
   * Scalable design
      Fuzzy Logic
   * No statistic modeling is required: robustness
           or           Fuzzy sets algebra       k                    i  f ( x)
      Filtering by
    * The system can be tunned if additional data is available
   decision making


                                         Jinan 2005
               Interference Canceller




Interference AR, SIR=-20dB,Gaussian
   Noise.


                              Jinan 2005
                  Fuzzy Robust Beamforming
                                                                                 Rx             s(k )
    Beamformers are spatial filters                                                   #1


    xk   s k   i k   n k                                          Linear              q

     s  k   a d  ud   i  k   n  k                    ˆ
                                                                 s (k )       Processing
                                                                                       #i




    u  sin q 
                                                                                            Δ
                               “steering vector” =
                               spatial information with                                #P       i(k )
                               uncertainty
                               w1                x1


                               w2                x2                              DOA
                                                                          q

    s k 
    ˆ


                               wN                xN
                                                              s  k   wH x  k 
                                                              ˆ

                                                 Jinan 2005
     Fuzzy Robust Beamforming
   w = R -1a(q )                                                                 Output fuzzy sets:
                                                                                 optimum beamformers
                                                                                 pointing to ui

• Triangular L fuzzy sets
• Fuzzification Singleton




                                                                                          Additive
                                                                                        Fuzzy System
       IF DOA is Ai THEN the
       desired outpus is Bi

                         L               L
                                                  Ai (ud )
                                                       ˆ
                 w F   vi w opt (ui )      L
                                                                   w opt (ui )
                        i 1            i 1
                                               
                                               j 1
                                                      Aj
                                                            ˆ
                                                           (ud )
                                      Jinan 2005
      Fuzzy Robust beamforming:
      Parameter tunning
   For L=6

          SNR             In-prior SIR    Width

           0 dB              1 dB         1


    0  dB SNR  –10 dB       1 dB         3


        whole range          < 1 dB       High




                             Jinan 2005
            Fuzzy Robust beamforming:
                     Results
      DOA Robustness


                                         ud=0, SNR = 0dB,
                                         uint={-0.5, 0.6},
                                         INR={20, 20} dB




                     DOA misadjustment


Optimized solution
for DOA = 0

                        Jinan 2005
    Seismic wave
     separation

 Unsupervised
  classifiers
 Feature space:
  spectrogram,proximity
  (distance) notion
 Objective: separate
  different time waves
  from a wavelet image
  series
                      Jinan 2005
Fuzzy A C-Means
                                                 c   N

    Minimize              ˆ (U , V , A)    u m x  v
                           Jm
                                                                              2
                                                 ik   k   i                   Ai
                                                i 1 k 1


    Variable Norm for each cluster                                                2     C                 2
                                                                                                                  
                                                                         1               1         
                                                                                  m 1                     m 1
                                                                                                                  
     d  xk  vi             xk  vi  Ai  xk  vi            ik   md               d md    
       2              2                  T
                                                                                                                
                                                                         dik 
      ik              Ai                                                                   j 1       
                                                                                         
                                                                                         
                                                                                                  jk
                                                                                                                  
                                                                                                                  
    Fuzzy Covariance Matrix Cf
            N                                T            N         m
     C fj    u jk   xk  vj  xk  vj                u jk  
                       m

             k 1                                          k 1      
                                                              1
    Update Ai                Ajopt    j det  C fj   C 1
                                                         
                                                              p
                                                              fj
                           1 j  c



    Determinant (volume) of each Ai fixed
                                             Jinan 2005
Geometric Shape
     Distance (Norm)
d  xk  vi         
  2             2
 ik

  xk  vi  A  xk  vi 
            T



  Euclidean A=I
  Ellipsoid

    1/10 0     0 
A   0 1/ 50
               0  
     0
         0   1/100 
                    
                              Jinan 2005
r 2   x    1  x   
                    t




  12 2 3
         2 2



x2 y2 z2
  2
     2  2 1
a    b   c
                           4
V  V3             r3        1 2 3
               12

                            3
  Jinan 2005
Fuzzy A C-Means
    Use of fuzzy information




    Separation
     of classes




                          Jinan 2005
                 Fuzzy A C-Means
   Evolution: a class disappears

   Use of
    background
    class
    (fixed)


   Volume
    estimation
    at each
    iteration
                          Jinan 2005
                Segmentation with FACM
   FACM
    direct
    – m=3
    – md=4, 2

Complete
tracking




                         Jinan 2005
Segmentation with watershed



                               Fuzzy  alternatives have been
                               considered
                                                                  Veinatge [Crisp]                                                                        Veinatge [Exponencial + Dist. Euclidea]
                                 1                                                                                                  1



                               0.9                                                                                                0.9



                               0.8                                                                                                0.8



                               0.7                                                                                                0.7



                               0.6                                                                                                0.6
          Grau de pertinença




                                                                                                             Grau de pertinença
                               0.5                                                                                                0.5



                               0.4                                                                                                0.4



                               0.3                                                                                                0.3



                               0.2                                                                                                0.2



                               0.1                                                                                                0.1



                                 0                                                                                                  0
                                     -25   -20   -15   -10   -5             0        5   10   15   20   25                              -25   -20   -15   -10         -5           0           5    10   15   20   25
                                                                         Eix X                                                                                                  Eix X




                                           Jinan 2005
         Segmentation with watershed
Manual
maxima
 selection


Fuzzy W
gives similar
results



                    Jinan 2005
Jinan 2005
Classifier      Gaussian               NonGaussian

   CM             0.2404                 0.3505
  FCM             0.2436                 0.3536
  ACM             0.1226                 0.1693
 FACM             0.1175                 0.1446
  MLU             0.1071                 0.1455

             Misclassification error
                     Jinan 2005
Classifier        Gaussian           NonGaussian

   CM             7.447 10-8          3.056 10-8

  FCM             3.256 10-8          1.755 10-8

  ACM             2.839 10-8          2.240 10-8

 FACM             6.707 10-9          9.397 10-9

  MLU            1.131 10-11          1.305 10-8

             Reconstruction error

                        Jinan 2005
Conclusions on seismic wave separation
    FCM can be used to separate the time waveforms
     contained in this wavelet image series.
    The Background class allows to use 3D data, improving
     the results.
    The use of fuzzy sets gives more information about the
     border lines and let classes to get overlap.
    There are some degrees of freedom that let the classifier to
     adapt to different situations in an intuitive way.
    It’s important to know how many classes there are and an
     initial estimation of their shape and volume.
    The algorithms needs to be tested with more sample
     images and compared with other methods.
                             Jinan 2005
               Conclusions:
     Main features of fuzzy processing
   Fuzzy systems build up mathematical models from
    linguistic knowledge
   Statistical knowledge is not necessary, but can be
    incorporated: scalable design with the available
    information
   Fast acquisition and tracking
   Easy implementation & tunning of complex
    systems
   Robust design to uncertainties in the model

                         Jinan 2005
Fuzzy logic & systems for signal
           separation

         Presented by Ana Pérez-Neira

  Dept. of Signal Theory and Communications
        Technical University of Catalonia (UPC)


         e-mail: anuska@gps.tsc.upc.edu


                     Jinan 2005                   27
Interference Canceller...and within
            the bounds


                                       Interference: digital



      dB
   SIRm in



  Interference: multitone
Number of CDMA users K=4,
  Ganancia de procesado                            dB
                                                SIRm in
  G=11,SNRgauss=20dB,
       SNRmai=5dB
         Pd=1, D=3        Jinan 2005
                                                                                      FACM and MLU
                                1          ˆ i  Σi1  x k  μi   P i 
                                                                             1 2
                          exp    x k  μ ˆ                  ˆ ˆ ˆ
                                                 t
                                                                    Σi
ˆ         ˆ
P i x k , θ                  2
                                   1
                                                                    
                                                                   ˆ ˆ
                              exp    x k  μ j  Σ 1  x k  μ j  P  j 
                         1 2
                j 1 ˆj                              ˆj
                C                                  t
                      Σ                         ˆ
                                   2                                                                                                                                                                                                                                   Comparacio de funcions nucli

      Equivalent                                                                                                                                                                                                                   8

                                                                                                                                                                                                                                   7
                                                                                                                                                                                                                                                                                                                  1/x
                                                                                                                                                                                                                                                                                                                  1/x
                                                                                                                                                                                                                                                                                                                     2




                                                                          x                                                                                                   
                                                                                                                                                                                                                                                                                                                  exp(-0.5·x)
                                                                                                                                                                                                                2
                 Σ ˆ
                          1 2
                                                                     f                     vi 
                                                                                                          t
                                                                                                               Σi1  x k  v i  P i  
                                                                                                               ˆ                  ˆ                                                                                                6


                  i                                                                                                                      
                                                                                                                                        
                                                                                    k
 ik 
                                                                                                                                                                                                                                   5
          2
                                                                                                                                                                                                                     2             4

               C ˆ                                                                                                 ˆ i1  x k  v i  P i  
                                                                               xk  vi                                                                                              
                                             1 2
                 Σi
                                                                                                                t
                                                                                                                    Σ                   ˆ                                                                                          3
                                                                         f                                                                      
                j 1                                                                                                                           
                                                                                                                                                                                                                                   2

                                                                                                                                                                                                                                   1


                                                                                                                                                                                                                                   0
                   1
f x 
                                                                                                                                                                                                                                    -2                             0           2                4             6                   8
                                                                              Comparacio entre FACM i ML no supervisat                                                              Comparacio entre FACM i ML no supervisat                                                            Comparacio FACM
                                                              1.2                                                                                                       1.2
                   md                                                                                      ML
                                                                                                           FACM (m = 2, md = 1)
                                                                                                                                                                                                                 ML
                                                                                                                                                                                                                 FACM (m = 2, md = 1)
                                                                                                                                                                                                                                                                  1.2                                         m = 2, md = 1
                                                                                                                                                                                                                                                                                                              m = 2, md = 2
                                                                                                                                                                                                                                                                                                              m = 3, md = 1
                   m 1                                        1                                                                                                         1

               x                                                                                                                                                                                                                                                   1
                            Grau de pertinença/probabilitat




                                                                                                                                      Grau de pertinença/probabilitat




                                                              0.8                                                                                                       0.8                                                                                       0.8




                                                                                                                                                                                                                                             Grau de pertinença
                                                              0.6                                                                                                       0.6                                                                                       0.6


                                                              0.4                                                                                                       0.4                                                                                       0.4


                                                              0.2                                                                                                       0.2                                                                                       0.2


                                                               0                                                                      Jinan 2005                         0                                                                                         0

                                                                -3       -2          -1         0          1             2        3                                       -3   -2          -1         0          1             2         3                          -3    -2       -1          0          1       2           3
                                                                                              Eix X                                                                                                 Eix X                                                                                    Eix X
C-Means II
   Generate samples using the image as a pdf
    (probability density function)




                                     Jinan 2005
Fuzzy C-Means I                                                                        M: fuzziness

     Minimize                                                 Algorithm
                        c           N
J m (U , V )    uik                     xk  vi
                                         m             2                    N                  N
                                                                   vi    uik  xk           uik 
                                                                                     m                    m
                      i 1 k 1                                 1i  c
                                                                            k 1              k 1

     Fuzzy Partition
      Matrix U                                                            I k  i 1  i  c; dik  0
                                                                         
                                                                   
                                                                1 k  N
                                                                          I k  1, 2,..., c \ I k
                                                                         
     1)       uik   0,1
           1i  c
           1 k  N
                                                                     2      c             2
                                                                                                
                       c
                                                              1   m 1
                                                                                     1  m 1 
     2)      
           1 k  N
                      u       ik   1                         
                                                              d              d   , I k  
                                                                                         
                      i 1
                                                      uik   ik            j 1  jk 
                                                                                                
                                                    1i  c                                    
                                                             
                             N
     3)  0   uik  N                           1 k  N
                                                                       0,  uik  1,             Ik  
           i i  c
                             k 1
                                                             
                                                             
                                                             
                                                                     iI k
                                                                           iI k

                                                   Jinan 2005
Geometric Shape
     Distance (Norm)
d  xk  vi         
  2             2
 ik

  xk  vi  A  xk  vi 
            T



  Euclidean A=I
  Ellipsoid

    1/10 0     0 
A   0 1/ 50
               0  
     0
         0   1/100 
                    
                              Jinan 2005
r 2   x    1  x   
                    t




  12 2 3
         2 2



x2 y2 z2
  2
     2  2 1
a    b   c
                           4
V  V3             r3        1 2 3
               12

                            3
  Jinan 2005
       FACM direct


- Form
adaptation
-Accurate
initialization
- Background

class




                      Jinan 2005
FCM with Background Class I
    Background class: classifies only in z

    Norm
     initialization
     to an
     approximate
     volume of
     the cluster
    3D data
     with mass
    Centers in
     x,y plane
                            Jinan 2005
Fuzzy C-Means (Modified Norm) III
    Evolution: continuing without one class




    Cluster
     shapes can
     rotate and
     adapt better




                           Jinan 2005
Fuzzy C-Means (Modified Norm) IV
    Use of fuzzy information




    Separation
     of classes




                          Jinan 2005
                                               Seismic Profile

                0




                5



               10



               15




               20




     Sensors
               25




               30




               35



               40




               45




                    0   50   100   150   200         250         300   350   400   450   500
                                                    Temps




Jinan 2005
Jinan 2005
Classifier      Gaussian               NonGaussian

   CM             0.2404                 0.3505
  FCM             0.2436                 0.3536
  ACM             0.1226                 0.1693
 FACM             0.1175                 0.1446
  MLU             0.1071                 0.1455

             Misclassification error
                     Jinan 2005
Classifier        Gaussian           NonGaussian

   CM             7.447 10-8          3.056 10-8

  FCM             3.256 10-8          1.755 10-8

  ACM             2.839 10-8          2.240 10-8

 FACM             6.707 10-9          9.397 10-9

  MLU            1.131 10-11          1.305 10-8

             Reconstruction error

                        Jinan 2005
         N                                  T N         m
   CFi     ik   x k  vi  x k  vi      ik  
                    m

          k 1                                 k 1      

                                                                                                                         Influence of md
   – Introduction of
                                                                                                                                                                                                                                                                            Fronteres de les 4 classes
                                                                                           9


                                                                                                                                                                                                                                         0.01
                                                                                           8

                                                                                                                                                                                                                            0.009
                                           More possib.                                                                         Less possib.
      md:                                                                                  7


                                                                                           6
                                                                                                                                                                                                                            0.008




                                                                                                                                                                                         Densitat de Probabilitat
                                                                                                                                                                                                                            0.007




                                                                      Distancia mesurada
                                                                                           5                                                                                                                                0.006

                                                                                           4                                                                                                                                0.005

                                                                                                                                                                                                                            0.004
                                                                                           3


                  2
                        C                 2
                                                                                                                                                                                                                           0.003




                          1         
                                                                                           2



       1       m 1                     m 1
                                                 
                                                                                                                                                                                                                            0.002




                           d md
                                                                                                                                                                            d




ik   md 
                                                                                                                                                                                2
                                                                                           1                                                                                d




                                      
                                                                                                                                                                                                                            0.001



                                               
                                                                                           0                                                                                                                                                0
                                                                                               0              0.5          1              1.5               2         2.5            3



       d ik 
                                                                                                                                                                                                                                                    20     40          60        80     100      120     140         160   180   200


                          j 1       
                                                                                                                               Coordenada respecte l'origen
                                                                                                                                                                                                                                                                                       Eix X


                        
                        
                                 jk
                                                 
                                                                                                                   Funcions de Pertinença a cada Classe                                                                                                          Funcions de Pertinença a cada Classe


                                                                                           1   m=5
                                                                                               md = 20                                                                                                                                      1 m=2
                                                                                                                                                                                                                                                          md = 20

                                                                                                   md = 4
                                                                                                                                                                                                                                                         md = 4
                                                                      0.8                                                                                                                                                                 0.8            md = 2
                                                                                                   md = 2
                                                                                                                                                                                                                                                         md = 1




                                                                                                                                                                                                                    Grau de Pertinença
                                                 Grau de Pertinença




                                                                      0.6                                                                                                                                                                 0.6
                                                                                                     md = 1



                                                                      0.4                                                                                                                                                                 0.4



                                                                      0.2                                                                                                                                                                 0.2



                                                                                           0                                                                                                                                                0


                                                                                               0                    50                   100                    150                 200                                                         0                 50                    100                    150               200
                                                                                                                                        Eix X                                                                                                                                          Eix X




                                                                                                              Jinan 2005

								
To top
;