; Fuzzy Clustering
Documents
User Generated
Resources
Learning Center
Your Federal Quarterly Tax Payments are due April 15th

# Fuzzy Clustering

VIEWS: 24 PAGES: 42

• pg 1
```									Fuzzy logic & systems for signal
separation

Presented by Ana Pérez-Neira

Dept. of Signal Theory and Communications
Technical University of Catalonia (UPC)
Collaborating with the CTTC

e-mail: anuska@gps.tsc.upc.edu
Jinan 2005                   1

 Introduction
 Signal filtering: application to communication
signals
 Signal classification: application to seismic
waveform separation
 Conclusions

Jinan 2005
Introduction:
Main features of fuzzy processing
   Fuzzy systems build up mathematical models from
linguistic knowledge
   Statistical knowledge is not necessary, but can be
incorporated: scalable design with the available
information
   Fast acquisition and tracking
   Easy implementation & tunning of complex
systems
   Robust design to uncertainties in the model

Jinan 2005
Interference Canceller
Sent CDMA
Signal dk

Narrow band
Interference ik

rk

rk=Akdk +ik + nk               Canceller

Detected
information

Jinan 2005
Interference Canceller
rk
+ -
Estimator/            -
-         Estimation and
detector     ik                subtraction solution

P1. Each type of interference requires a different filter
P2. Non-Gaussian noise is present

   The fuzzy filter will cope with
- Problems P1 and P2
- Low complexity and scalable solution in front of:
 Amount of information

 Computation affordable

Jinan 2005
Interference Canceller
Ri: IF x1 is A1i and x2 is A2i THEN y is Bi

x1  rk  rk 1              rk=dk +ik                           y  ik
A11     A12 A13                                     B1      B2     B3
Filtering
x1                                                      y
rk-1                               ymin           rk-1         ymax
x1min r                 x1max
k
rk      dk     ik
x2  ik 1  rk 1
A21     A22 A23                                     B1      B2     B3
Prediction
x2                                                      y
zk-1                                  ymin           rk-1         ymax
x2min i                 x2max
k-1

x3  ik 2  rk 1                  Jinan 2005
ik-1           ik
Interference Canceller

S
l  x1 , x 2 , x3          S
i   il
ˆ
S
  il p  il / rl , i l 1 , ml  
l 1
 l  x1 , x2 , x3 
m 1
l 1

 E il / rl , il 1
Parzen

Features of the designed fuzzy filter
* Scalable design
Fuzzy Logic
* No statistic modeling is required: robustness
or           Fuzzy sets algebra       k                    i  f ( x)
Filtering by
* The system can be tunned if additional data is available
decision making

Jinan 2005
Interference Canceller

Interference AR, SIR=-20dB,Gaussian
Noise.

Jinan 2005
Fuzzy Robust Beamforming
Rx             s(k )
    Beamformers are spatial filters                                                   #1

xk   s k   i k   n k                                          Linear              q

 s  k   a d  ud   i  k   n  k                    ˆ
s (k )       Processing
#i

u  sin q 
Δ
“steering vector” =
spatial information with                                #P       i(k )
uncertainty
w1                x1

w2                x2                              DOA
q

s k 
ˆ

wN                xN
s  k   wH x  k 
ˆ

Jinan 2005
Fuzzy Robust Beamforming
w = R -1a(q )                                                                 Output fuzzy sets:
optimum beamformers
pointing to ui

• Triangular L fuzzy sets
• Fuzzification Singleton

Fuzzy System
IF DOA is Ai THEN the
desired outpus is Bi

L               L
 Ai (ud )
ˆ
w F   vi w opt (ui )      L
w opt (ui )
i 1            i 1

j 1
Aj
ˆ
(ud )
Jinan 2005
Fuzzy Robust beamforming:
Parameter tunning
   For L=6

SNR             In-prior SIR    Width

 0 dB              1 dB         1

0  dB SNR  –10 dB       1 dB         3

whole range          < 1 dB       High

Jinan 2005
Fuzzy Robust beamforming:
Results
   DOA Robustness

ud=0, SNR = 0dB,
uint={-0.5, 0.6},
INR={20, 20} dB

Optimized solution
for DOA = 0

Jinan 2005
Seismic wave
separation

 Unsupervised
classifiers
 Feature space:
spectrogram,proximity
(distance) notion
 Objective: separate
different time waves
from a wavelet image
series
Jinan 2005
Fuzzy A C-Means
c   N

   Minimize              ˆ (U , V , A)    u m x  v
Jm
2
ik   k   i                   Ai
i 1 k 1

   Variable Norm for each cluster                                                2     C                 2

 1               1         
m 1                     m 1

d  xk  vi             xk  vi  Ai  xk  vi            ik   md               d md    
2              2                  T
                
 dik 
ik              Ai                                                                   j 1       


jk


   Fuzzy Covariance Matrix Cf
N                                T            N         m
C fj    u jk   xk  vj  xk  vj                u jk  
m

 k 1                                          k 1      
1
   Update Ai                Ajopt    j det  C fj   C 1
                  
p
fj
1 j  c

   Determinant (volume) of each Ai fixed
Jinan 2005
Geometric Shape
    Distance (Norm)
d  xk  vi         
2             2
ik

  xk  vi  A  xk  vi 
T

 Euclidean A=I
 Ellipsoid

1/10 0     0 
A   0 1/ 50
           0  
 0
     0   1/100 

Jinan 2005
r 2   x    1  x   
t

  12 2 3
2 2

x2 y2 z2
2
 2  2 1
a    b   c
4
V  V3             r3        1 2 3
12

3
Jinan 2005
Fuzzy A C-Means
   Use of fuzzy information

   Separation
of classes

Jinan 2005
Fuzzy A C-Means
   Evolution: a class disappears

   Use of
background
class
(fixed)

   Volume
estimation
at each
iteration
Jinan 2005
Segmentation with FACM
   FACM
direct
– m=3
– md=4, 2

Complete
tracking

Jinan 2005
Segmentation with watershed

Fuzzy  alternatives have been
considered
Veinatge [Crisp]                                                                        Veinatge [Exponencial + Dist. Euclidea]
1                                                                                                  1

0.9                                                                                                0.9

0.8                                                                                                0.8

0.7                                                                                                0.7

0.6                                                                                                0.6
Grau de pertinença

Grau de pertinença
0.5                                                                                                0.5

0.4                                                                                                0.4

0.3                                                                                                0.3

0.2                                                                                                0.2

0.1                                                                                                0.1

0                                                                                                  0
-25   -20   -15   -10   -5             0        5   10   15   20   25                              -25   -20   -15   -10         -5           0           5    10   15   20   25
Eix X                                                                                                  Eix X

Jinan 2005
Segmentation with watershed
Manual
maxima
selection

Fuzzy W
gives similar
results

Jinan 2005
Jinan 2005
Classifier      Gaussian               NonGaussian

CM             0.2404                 0.3505
FCM             0.2436                 0.3536
ACM             0.1226                 0.1693
FACM             0.1175                 0.1446
MLU             0.1071                 0.1455

Misclassification error
Jinan 2005
Classifier        Gaussian           NonGaussian

CM             7.447 10-8          3.056 10-8

FCM             3.256 10-8          1.755 10-8

ACM             2.839 10-8          2.240 10-8

FACM             6.707 10-9          9.397 10-9

MLU            1.131 10-11          1.305 10-8

Reconstruction error

Jinan 2005
Conclusions on seismic wave separation
   FCM can be used to separate the time waveforms
contained in this wavelet image series.
   The Background class allows to use 3D data, improving
the results.
border lines and let classes to get overlap.
   There are some degrees of freedom that let the classifier to
adapt to different situations in an intuitive way.
   It’s important to know how many classes there are and an
initial estimation of their shape and volume.
   The algorithms needs to be tested with more sample
images and compared with other methods.
Jinan 2005
Conclusions:
Main features of fuzzy processing
   Fuzzy systems build up mathematical models from
linguistic knowledge
   Statistical knowledge is not necessary, but can be
incorporated: scalable design with the available
information
   Fast acquisition and tracking
   Easy implementation & tunning of complex
systems
   Robust design to uncertainties in the model

Jinan 2005
Fuzzy logic & systems for signal
separation

Presented by Ana Pérez-Neira

Dept. of Signal Theory and Communications
Technical University of Catalonia (UPC)

e-mail: anuska@gps.tsc.upc.edu

Jinan 2005                   27
Interference Canceller...and within
the bounds

Interference: digital

dB
SIRm in

Interference: multitone
Number of CDMA users K=4,
SIRm in
G=11,SNRgauss=20dB,
SNRmai=5dB
Pd=1, D=3        Jinan 2005
FACM and MLU
 1          ˆ i  Σi1  x k  μi   P i 
1 2
exp    x k  μ ˆ                  ˆ ˆ ˆ
t
Σi
ˆ         ˆ
P i x k , θ                  2
 1

ˆ ˆ
exp    x k  μ j  Σ 1  x k  μ j  P  j 
1 2
 j 1 ˆj                              ˆj
C                                  t
Σ                         ˆ
 2                                                                                                                                                                                                                                   Comparacio de funcions nucli

Equivalent                                                                                                                                                                                                                   8

7
1/x
1/x
2

 x                                                                                                   
exp(-0.5·x)
2
Σ ˆ
1 2
f                     vi 
t
Σi1  x k  v i  P i  
ˆ                  ˆ                                                                                                6

 i                                                                                                                      
                                                                                                                          
k
 ik 
5
2
2             4

C ˆ                                                                                                 ˆ i1  x k  v i  P i  
 xk  vi                                                                                              
1 2
  Σi
t
Σ                   ˆ                                                                                          3
f                                                                      
 j 1                                                                                                                           
2

1

0
1
f x 
-2                             0           2                4             6                   8
Comparacio entre FACM i ML no supervisat                                                              Comparacio entre FACM i ML no supervisat                                                            Comparacio FACM
1.2                                                                                                       1.2
md                                                                                      ML
FACM (m = 2, md = 1)
ML
FACM (m = 2, md = 1)
1.2                                         m = 2, md = 1
m = 2, md = 2
m = 3, md = 1
m 1                                        1                                                                                                         1

x                                                                                                                                                                                                                                                   1
Grau de pertinença/probabilitat

Grau de pertinença/probabilitat

0.8                                                                                                       0.8                                                                                       0.8

Grau de pertinença
0.6                                                                                                       0.6                                                                                       0.6

0.4                                                                                                       0.4                                                                                       0.4

0.2                                                                                                       0.2                                                                                       0.2

0                                                                      Jinan 2005                         0                                                                                         0

-3       -2          -1         0          1             2        3                                       -3   -2          -1         0          1             2         3                          -3    -2       -1          0          1       2           3
Eix X                                                                                                 Eix X                                                                                    Eix X
C-Means II
   Generate samples using the image as a pdf
(probability density function)

Jinan 2005
Fuzzy C-Means I                                                                        M: fuzziness

   Minimize                                                 Algorithm
c           N
J m (U , V )    uik                     xk  vi
m             2                    N                  N
 vi    uik  xk           uik 
m                    m
i 1 k 1                                 1i  c
k 1              k 1

   Fuzzy Partition
Matrix U                                                            I k  i 1  i  c; dik  0

 
1 k  N
 I k  1, 2,..., c \ I k

   1)       uik   0,1
1i  c
1 k  N
        2      c             2

c
 1   m 1
         1  m 1 
   2)      
1 k  N
u       ik   1                         
 d              d   , I k  
    
i 1
 uik   ik            j 1  jk 

1i  c                                    

N
   3)  0   uik  N                           1 k  N
 0,  uik  1,             Ik  
i i  c
k 1



iI k
iI k

Jinan 2005
Geometric Shape
    Distance (Norm)
d  xk  vi         
2             2
ik

  xk  vi  A  xk  vi 
T

 Euclidean A=I
 Ellipsoid

1/10 0     0 
A   0 1/ 50
           0  
 0
     0   1/100 

Jinan 2005
r 2   x    1  x   
t

  12 2 3
2 2

x2 y2 z2
2
 2  2 1
a    b   c
4
V  V3             r3        1 2 3
12

3
Jinan 2005
   FACM direct

- Form
-Accurate
initialization
- Background

class

Jinan 2005
FCM with Background Class I
   Background class: classifies only in z

   Norm
initialization
to an
approximate
volume of
the cluster
   3D data
with mass
   Centers in
x,y plane
Jinan 2005
Fuzzy C-Means (Modified Norm) III
   Evolution: continuing without one class

   Cluster
shapes can
rotate and

Jinan 2005
Fuzzy C-Means (Modified Norm) IV
   Use of fuzzy information

   Separation
of classes

Jinan 2005
Seismic Profile

0

5

10

15

20

Sensors
25

30

35

40

45

0   50   100   150   200         250         300   350   400   450   500
Temps

Jinan 2005
Jinan 2005
Classifier      Gaussian               NonGaussian

CM             0.2404                 0.3505
FCM             0.2436                 0.3536
ACM             0.1226                 0.1693
FACM             0.1175                 0.1446
MLU             0.1071                 0.1455

Misclassification error
Jinan 2005
Classifier        Gaussian           NonGaussian

CM             7.447 10-8          3.056 10-8

FCM             3.256 10-8          1.755 10-8

ACM             2.839 10-8          2.240 10-8

FACM             6.707 10-9          9.397 10-9

MLU            1.131 10-11          1.305 10-8

Reconstruction error

Jinan 2005
N                                  T N         m
CFi     ik   x k  vi  x k  vi      ik  
m

 k 1                                 k 1      

Influence of md
– Introduction of
Fronteres de les 4 classes
9

0.01
8

0.009
More possib.                                                                         Less possib.
md:                                                                                  7

6
0.008

Densitat de Probabilitat
0.007

5                                                                                                                                0.006

4                                                                                                                                0.005

0.004
3

2
C                 2
                                                                                                                                                                          0.003

  1         
2

 1       m 1                     m 1

0.002

   d md
d

ik   md 
2
1                                                                                d


0.001

                
0                                                                                                                                                0
0              0.5          1              1.5               2         2.5            3

 d ik 
20     40          60        80     100      120     140         160   180   200

j 1       
Eix X



jk

                                                                  Funcions de Pertinença a cada Classe                                                                                                          Funcions de Pertinença a cada Classe

1   m=5
md = 20                                                                                                                                      1 m=2
md = 20

md = 4
md = 4
0.8                                                                                                                                                                 0.8            md = 2
md = 2
md = 1

Grau de Pertinença
Grau de Pertinença

0.6                                                                                                                                                                 0.6
md = 1

0.4                                                                                                                                                                 0.4

0.2                                                                                                                                                                 0.2

0                                                                                                                                                0

0                    50                   100                    150                 200                                                         0                 50                    100                    150               200
Eix X                                                                                                                                          Eix X

Jinan 2005

```
To top
;