Docstoc

Neuro-Fuzzy Data Analysis

Document Sample
Neuro-Fuzzy Data Analysis Powered By Docstoc
					Industrial Applications of
 Neuro-Fuzzy Networks

      Prof. Dr. Rudolf Kruse
          University of Magdeburg
         Faculty of Computer Science
             Magdeburg, Germany
      rudolf.kruse@cs.uni-magdeburg.de


                                         NF
                                         SEURO



                                            UZZY
Example: Continously Adapting Gear Shift Schedule in VW New Beetle



    classification of driver / driving situation         gear shift
                  by fuzzy logic                        computation

fuzzification           inference        defuzzifi-     interpolation
                        machine          cation
accelerator pedal

filtered speed of
                                                        determination
accelerator pedal
                                                        of speed limits
                         rule                           for shifting      gear
number of                                  sport
                         base                           into higher or    selection
changes in                                 factor [t]
                                                        lower gear
pedal direction
                                                        depending on
                                                        sport factor

sport factor [t-1]




    März 2001                    Rudolf Kruse
                                                                                NF
                                                                                S     EURO



                                                                                        UZZY
      Continously Adapting Gear Shift Schedule: Technical Details


 Mamdani controller with 7 rules

 Optimized program                                           AG4

    24 Byte RAM

    702 Byte ROM
                    }  on Digimat



 Runtime 80 ms
  12 times per second a new sport factor is assigned

 How to generate knowledge automatically from data?


   März 2001                  Rudolf Kruse
                                                                    NF
                                                                    SEURO



                                                                       UZZY
Learning from Examples (Observations, Databases)

 Statistics:          parameter fitting, structure
                       identification, inference method,
                       model selection
 Machine Learning:    computational learning (PAC
                       learning), inductive learning, learning
                       decision trees, concept learning, ...
 Neural Networks:     learning from data
 Cluster Analysis:    unsupervised classification



 Learning Problem is transformed into an optimization problem.
 How to use these methods in fuzzy systems?

     März 2001             Rudolf Kruse
                                                                 NF
                                                                 SEURO



                                                                    UZZY
Function Approximation with Fuzzy Rules


                                            if x is large then y is large




                                y
    output value



                                                         x
                   current input value

   März 2001                 Rudolf Kruse
                                                                            NF
                                                                            SEURO



                                                                               UZZY
How to Derive a Fuzzy Controller Automatically from Observed Process Data




                                                   ut
   • Function approximation




                                                 tp
                                               ou
                                                            input
                         current input value

   • Perform fuzzy cluster analysis of input-output data (FCM, GK, GG, ...)
   • Project clusters
   • Obtain fuzzy rules of the kind: „If x is small then y is medium“


     März 2001                         Rudolf Kruse
                                                                              NF
                                                                              SEURO



                                                                                 UZZY
                Fuzzy Cluster Analysis
 Classification of a given data set X = {x1, ..., xn}  p into c
  clusters.
 Membership degree of datum xk to class i is uik.
 Representation of cluster i by prototype vi  p.

Formal: Minimisation of functional:

                JX, U, v    uik  d2 v i, x k 
                                  c     n
                                                m

                                  i1 k 1

under constraints n
                 u     ik
                              0 for all i   , ..., c
                                              1
                 k 1
                  c

                 u     ik
                              1 for all k   , ..., n
                                              1

    März 2001
                  i1

                                 Rudolf Kruse
                                                                     NF
                                                                     SEURO



                                                                        UZZY
Simplest Algorithm: Fuzzy-c-Means (FCM)

                                d v i , x k   v i  x k
                                  2                           2




   Iterative Procedure (with random initialisation of prototypes vi)

                                                                          u 
                                                                          n
                                                                                       m
                                                                                  ik        xk
                                1
          ui k                                         and       vi    k 1


                                                                            u 
                                               1                             n

                         d2 v i , x k  
                                                                                            m
                    c                         m 1

                     d2 v , x 
                        
                   j 1 
                                                                          k 1
                                                                                       ik


                                j     k 




 FCM is searching for equally large clusters in form of (hyper-)balls.


    März 2001                            Rudolf Kruse
                                                                                                 NF
                                                                                                 SEURO



                                                                                                    UZZY
            Examples




März 2001    Rudolf Kruse
                            NF
                            SEURO



                               UZZY
               Fuzzy Cluster Analysis
 Fuzzy C-Means: simple, looks for spherical
  clusters of same size, uses Euclidean distance
 Gustafson & Kessel: looks for hyper-ellipsoidal
  clusters of same size, distance via matrices
 Gath & Geva: looks for hyper-ellipsoidal clusters
  of arbitrary size, distance via matrices
 Axis-parallel variations exist that use diagonal
  matrices (computationally less expensive and less
  loss of information when rules are created)

   März 2001           Rudolf Kruse
                                                      NF
                                                      SEURO



                                                         UZZY
Fuzzy Cluster Analysis with DataEngine




 März 2001      Rudolf Kruse
                                     NF
                                     S   EURO



                                           UZZY
       Construct Fuzzy Sets by Cluster Projection
u(x)                                             Approximation by a
                                                 triangular fuzzy set
  1
                                                 Convex hull of the discrete
                                                 degrees of membership

                                                 Connection of the discrete
                                                 degrees of membership




                                             x
  Projecting a cluster means to project the degrees of membership
  of the data on the single dimensions: Histograms are obtained.


        März 2001             Rudolf Kruse
                                                                        NF
                                                                        S     EURO



                                                                                UZZY
FCLUSTER: Tool for Fuzzy Cluster Analysis




  März 2001      Rudolf Kruse
                                        NF
                                        S   EURO



                                              UZZY
                 Introduction
 Building a fuzzy system requires
  prior knowledge (fuzzy rules, fuzzy sets)

  manual tuning: time consuming and error-prone

 Therefore: Support this process by learning
  learning fuzzy rules (structure learning)
  learning fuzzy set (parameter learning)

Approaches from Neural Networks can be used

   März 2001           Rudolf Kruse
                                                   NF
                                                   SEURO



                                                      UZZY
   Learning Fuzzy Sets: Problems in Control

 Reinforcement learning must be used to compute an error
  value
  (note: the correct output is unknown)

 After an error was computed, any fuzzy set learning
  procedures can be used

 Example: GARIC (Berenji/Kedhkar 1992)
  online approximation to gradient-descent

 Example: NEFCON (Nauck/Kruse 1993)
  online heuristic fuzzy set learning using a
  rule-based fuzzy error measure

   März 2001               Rudolf Kruse
                                                        NF
                                                        S   EURO



                                                              UZZY
März 2001   Rudolf Kruse
                           NF
                           SEURO



                              UZZY
 Neuro-Fuzzy Systems in Data Analysis

 Neuro-Fuzzy System:
  System of linguistic rules (fuzzy rules).
  Not rules in a logical sense, but function
   approximation.
  Fuzzy rule = vague prototype / sample.

 Neuro-Fuzzy-System:
  Adding a learning algorithm inspired by neural
   networks.
  Feature: local adaptation of parameters.

  März 2001            Rudolf Kruse
                                                    NF
                                                    SEURO



                                                       UZZY
Example:      Prognosis of the Daily Proportional Changes of the DAX at
              the Frankfurter Stock Exchange (Siemens)




 Database: time series from 1986 - 1997


 DAX                                   Composite DAX
 German 3 month interest rates         Return Germany
 Morgan Stanley index Germany          Dow Jones industrial index
 DM / US-$                             US treasury bonds
 Gold price                            Nikkei index Japan
 Morgan Stanley index Europe           Price earning ratio




  März 2001                      Rudolf Kruse
                                                                      NF
                                                                      S   EURO



                                                                            UZZY
            Fuzzy Rules in Finance
        Trend Rule
         IF      DAX = decreasing AND US-$ = decreasing
         THEN DAX prediction = decrease
         WITH high certainty
        Turning Point Rule
         IF       DAX = decreasing AND US-$ = increasing
         THEN DAX prediction = increase
         WITH low certainty
        Delay Rule
         IF      DAX = stable AND US-$ = decreasing
         THEN DAX prediction = decrease
         WITH very high certainty
        In general
         IF        x1 is m1 AND x2 is m2
         THEN y = h
         WITH weight k

März 2001                    Rudolf Kruse
                                                           NF
                                                           SEURO



                                                              UZZY
Classical Probabilistic Expert Opinion Pooling Method

  DM analyzes each source (human expert, data +
   forecasting model) in terms of (1) Statistical accuracy,
   and (2) Informativeness by asking the source to asses
   quantities (quantile assessment)

  DM obtains a “weight” for each source

  DM “eliminates” bad sources

  DM determines the weighted sum of source outputs

  Determination of “Return of Invest”
    März 2001            Rudolf Kruse
                                                        NF
                                                        S     EURO



                                                                UZZY
 E experts, R quantiles for N quantities
   each expert has to asses R·N values
 stat. Accuracy:
                                                                R      si
       C         1   R 2 N  I s, p ,
                        2
                                                   I s, p    si ln
                                                               i 0    p
 information score:
           1 N                     R 1             pr 1 
       I   lnvi, R 1  vi,o    pr 1 ln                
           N i 1                   r 1       vi,r  vi,r 1 
                                                   ce  I e  id ce 
 weight for expert e:             we 
                   E
                                          e1ce  I e  id e ce 
                                           E

 outputt=  we  outputt
                        e
                  e 1
           T
                             
 roi =  yt   sign outputt
                             DM
                                               
          t 1
    März 2001                           Rudolf Kruse
                                                                            NF
                                                                            SEURO



                                                                               UZZY
                Formal Analysis

 Sources of information
     R1     rule set given by expert 1
     R2     rule set given by expert 2
     D      data set (time series)

 Operator schema
     fuse (R1, R2) fuse two rule sets
     induce(D)            induce a rule set from D
     revise(R, D)         revise a rule set R by D



   März 2001            Rudolf Kruse
                                                     NF
                                                     SEURO



                                                        UZZY
                 Formal Analysis

 Strategies:
    fuse(fuse (R1, R2), induce(D))
    revise(fuse(R1, R2), D)            
    fuse(revise(R1, D), revise(R2, D))

 Technique: Neuro-Fuzzy Systems
    Nauck, Klawonn, Kruse, Foundations of Neuro-Fuzzy
     Systems, Wiley 97
    SENN (commercial neural network environment, Siemens)



   März 2001             Rudolf Kruse
                                                        NF
                                                        S    EURO



                                                               UZZY
      From Rules to Neural Networks

1. Evaluation of membership degrees




2. Evaluation of rules (rule activity)
                                  x   j 1 m c( ,js)  xi 
                  n        r
             ml: IR  [0,1] ,
                                          l    D



3. Accumulation of rule inputs and normalization
                                                   kl m l  x 
             NF: IR  IR, x  l 1 wl
                   n                   r


                                                         k j m j x 
                                                   r
                                                   j 1



 März 2001                      Rudolf Kruse
                                                                         NF
                                                                         SEURO



                                                                            UZZY
        Neuro-Fuzzy Architecture




März 2001        Rudolf Kruse
                                   NF
                                   SEURO



                                      UZZY
  The Semantics-Preserving Learning Algorithm

Reduction of the dimension of the weight space
1. Membership functions of different inputs share their parameters,
   e.g.
                          m dax  m cdax
                            stable  stable


2. Membership functions of the same input variable are not allowed to pass
   each other, they must keep their original order,
   e.g.
                    m decreasing  m stable  m increasing




Benefits:      the optimized rule base can still be interpreted
               the number of free parameters is reduced

  März 2001                      Rudolf Kruse
                                                                        NF
                                                                        S    EURO



                                                                               UZZY
Return-on-Investment Curves of the Different Models

                Validation data from March 01, 1994 until April 1997




    März 2001                      Rudolf Kruse
                                                                       NF
                                                                       SEURO



                                                                          UZZY
               A Neuro-Fuzzy System
 is a fuzzy system trained by heuristic learning
  techniques derived from neural networks

 can be viewed as a 3-layer neural network with fuzzy
  weights and special activation functions

 is always interpretable as a fuzzy system

 uses constraint learning procedures

 is a function approximator (classifier, controller)

   März 2001             Rudolf Kruse
                                                        NF
                                                        SEURO



                                                           UZZY
               Learning Fuzzy Rules
 Cluster-oriented approaches
  => find clusters in data, each cluster is a rule

 Hyperbox-oriented approaches
  => find clusters in the form of hyperboxes

 Structure-oriented approaches
  => used predefined fuzzy sets to structure the
     data space, pick rules from grid cells


   März 2001            Rudolf Kruse
                                                     NF
                                                     SEURO



                                                        UZZY
Hyperbox-Oriented Rule Learning
   y
                                Search for hyperboxes in
                                the data space
                                Create fuzzy rules by
                                projecting the hyperboxes
                                Fuzzy rules and fuzzy
                                sets are created at the
                                same time
                            x   Usually very fast




März 2001    Rudolf Kruse
                                                          NF
                                                          SEURO



                                                             UZZY
  Hyperbox-Oriented Rule Learning
       y              y                     y          y



                x              x                x              x



 Detect hyperboxes in the data, example: XOR function
 Advantage over fuzzy cluster anlysis:
    No loss of information when hyperboxes are represented as
     fuzzy rules
    Not all variables need to be used, don‘t care variables can be
     discovered
 Disadvantage: each fuzzy rules uses individual fuzzy sets,
  i.e. the rule base is complex.

    März 2001                Rudolf Kruse
                                                                   NF
                                                                   S  EURO



                                                                        UZZY
         Structure-Oriented Rule Learning
                y
                                                        Provide initial fuzzy sets for
large




                                                        all variables.
                                                        The data space is partitioned
                                                        by a fuzzy grid
medium




                                                        Detect all grid cells that
                                                        contain data (approach by
                                                        Wang/Mendel 1992)
                                                       Compute best consequents and
small




                                                       select best rules (extension by
                                                       Nauck/Kruse 1995,
                                                     x NEFCLASS model)




                     small   medium          large

         März 2001                    Rudolf Kruse
                                                                                     NF
                                                                                     S   EURO



                                                                                           UZZY
  Structure-Oriented Rule Learning
 Simple: Rule base available after two cycles through the
  training data
    1. Cycle: discover all antecedents
    2. Cycle: determine best consequents

 Missing values can be handled
 Numeric and symbolic attributes can be processed at the
  same time (mixed fuzzy rules)

 Advantage: All rules share the same fuzzy sets
 Disadvantage: Fuzzy sets must be given
   März 2001                Rudolf Kruse
                                                             NF
                                                             SEURO



                                                                UZZY
               Learning Fuzzy Sets
 Gradient descent procedures
  only applicable, if differentiation is possible, e.g.
  for Sugeno-type fuzzy systems.

 Special heuristic procedures that do not use
  gradient information.

 The learning algorithms are based on the idea of
  backpropagation.


   März 2001            Rudolf Kruse
                                                          NF
                                                          SEURO



                                                             UZZY
  Learning Fuzzy Sets: Constraints
 Mandatory constraints:
     Fuzzy sets must stay normal and convex
     Fuzzy sets must not exchange their relative
      positions (they must not „pass“ each other)
     Fuzzy sets must always overlap
 Optional constraints
     Fuzzy sets must stay symmetric
     Degrees of membership must add up to 1.0
 The learning algorithm must enforce these
  constraints.

   März 2001           Rudolf Kruse
                                                    NF
                                                    SEURO



                                                       UZZY
  Different Neuro-Fuzzy Approaches
 ANFIS (Jang, 1993)
  no rule learning, gradient descent fuzzy set learning, function approximator
 GARIC (Berenji/Kedhkar, 1992)
  no rule learning, gradient descent fuzzy set learning, controller
 NEFCON (Nauck/Kruse, 1993)
  structure-oriented rule learning, heuristic fuzzy set learning, controller
 FuNe (Halgamuge/Glesner, 1994)
  combinatorical rule learning, gradient descent fuzzy set learning, classifier
 Fuzzy RuleNet (Tschichold-Gürman, 1995)
  hyperbox-oriented rule learning, no fuzzy set learning, classifier
 NEFCLASS (Nauck/Kruse, 1995)
  structure-oriented rule learning, heuristic fuzzy set learning, classifier
 Learning Fuzzy Graphs (Berthold/Huber, 1997)
  hyperbox-oriented rule learning, no fuzzy set learning, function approximator
 NEFPROX (Nauck/Kruse, 1997)
  structure-oriented rule learning, heuristic fuzzy set learning, function approx.

    März 2001                         Rudolf Kruse
                                                                                     NF
                                                                                     SEURO



                                                                                        UZZY
         Example: Medical Diagnosis
 Results from patients tested for breast cancer
  (Wisconsin Breast Cancer Data).


 Decision support: Do the data indicate a malignant or a benign
  case?


 A surgeon must be able to check the classification for
  plausibility.


 We are looking for a simple and interpretable classifier:
  knowledge discovery.

    März 2001                Rudolf Kruse
                                                               NF
                                                               S   EURO



                                                                     UZZY
               Example: WBC Data Set
 699 cases (16 cases have missing values).

 2 classes: benign (458), malignant (241).

 9 attributes with values from {1, ... , 10}
  (ordinal scale, but usually interpreted as a numerical
  scale).

 Experiment: x3 and x6 are interpreted as nominal
  attributes.

 x3 and x6 are usually seen as „important“ attributes.

   März 2001             Rudolf Kruse
                                                           NF
                                                           SEURO



                                                              UZZY
               Applying NEFCLASS-J
 Tool for developing Neuro-Fuzzy Classifiers

 Written in JAVA

 Free version for research available

 Project started at Neuro-Fuzzy Group of University of
  Magdeburg, Germany




   März 2001            Rudolf Kruse
                                                     NF
                                                     S    EURO



                                                            UZZY
NEFCLASS: Neuro-Fuzzy Classifier


               Output variables (class labels)

               Unweighted connections

               Fuzzy rules

               Fuzzy sets (antecedents)

               Input variables (attributes)




 März 2001   Rudolf Kruse
                                                 NF
                                                 SEURO



                                                    UZZY
               NEFCLASS: Features
 Automatic induction of a fuzzy rule base from data
 Training of several forms of fuzzy sets
 Processing of numeric and symbolic attributes
 Treatment of missing values (no imputation)
 Automatic pruning strategies
 Fusion of expert knowledge and knowledge obtained
  from data


   März 2001            Rudolf Kruse
                                                       NF
                                                       SEURO



                                                          UZZY
           Representation of Fuzzy Rules
                                  Example: 2 Rules


             c1              c2   R1: if x is large and y is small, then class is c1.

                                  R2: if x is large and y is large, then class is c2.

                                  The connections x  R1 and x  R2
        R1              R2
                                  are linked.
             small
large
                        large     The fuzzy set large is a shared weight.

  x                 y
                                  That means the term large has always the
                                  same meaning in both rules.


        März 2001                      Rudolf Kruse
                                                                                        NF
                                                                                        SEURO



                                                                                           UZZY
       1. Training Step: Initialisation
Specify initial fuzzy partitions for all input variables
                                               y




                                      large
      c1        c2




                                      medium
                                      small                                 x

x                      y
                                                   small   medium   large



    März 2001                Rudolf Kruse
                                                                      NF
                                                                      S     EURO



                                                                                UZZY
              2. Training Step: Rule Base
Algorithm:
                                                 Variations:
for (all patterns p) do
        find antecedent A,                       Fuzzy rule bases can
        such that A( p) is maximal;              also be created by using
        if (A  L) then add A to L;              prior knowledge, fuzzy
end;                                             cluster analysis, fuzzy
                                                 decision trees, genetic
for (all antecedents A  L) do                   algorithms, ...
        find best consequent C for A;
        create rule base candidate R = (A,C);
        Determine the performance of R;
        Add R to B;
end;
Select a rule base from B;


       März 2001                  Rudolf Kruse
                                                                     NF
                                                                     S  EURO



                                                                            UZZY
                 Selection of a Rule Base

          e
Pe rformanc of a Rule :
                                              • Order rules by performance.
                                              • Either select

                         
         N
           1
     1                                          the best r rules or
Pr 
                  c
                      Rr x p , with             the best r/m rules per class.
     N   p 1
                                              • r is either given or is
                                                determined automatically such
                                                that all patterns are covered.
  0 if class(x p )  con( Rr ),
  
c
  1 otherwise.
  

     März 2001                 Rudolf Kruse
                                                                            NF
                                                                            S   EURO



                                                                                  UZZY
                   Rule Base Induction
NEFCLASS uses a modified Wang-Mendel procedure
                                               y




                                      large
         c1        c2




                                      medium
    R1        R2    R3

                                      small                                 x


x                        y
                                                   small   medium   large



     März 2001               Rudolf Kruse
                                                                      NF
                                                                      S     EURO



                                                                                UZZY
              Computing the Error Signal
     Error Signal            Fuzzy Error ( jth output):

                             E j  sgn(d )  1   (d ) , with d  t j  o j
                                                                             2
         c1        c2                                             ad   
                                                                
                                                                 d      
                                                                         
                             and  :   0, 1,  (d )  e       max   

                             (t : correct output, o : actual output)
    R1        R2    R3

                             Rule Error:

x                        y   Er   r 1   r    Econ( Rr ) , with 0    1



     März 2001                 Rudolf Kruse
                                                                                 NF
                                                                                 SEURO



                                                                                    UZZY
          3. Training Step: Fuzzy Sets
                                                       x a                 
                                                        b  a if x  [a, b) 
Example:                                                                    
triangular                                             c  x                
membership       m a ,b,c :   [0,1], m a ,b,c ( x)         if x  [b, c] 
function.                                              c  b                
                                                                            
                                                       0      otherwise
                                                                            

                      m ( x)            if E  0
                 f 
                      1  m ( x)  otherwise
Parameter
updates for an
antecedent       b  f  E  c  a   sgn(x  b)
fuzzy set.       a   f  E  b  a   b
                 c  f  E  c  b   b

    März 2001                Rudolf Kruse
                                                                           NF
                                                                           S     EURO



                                                                                   UZZY
                     Training of Fuzzy Sets
                                                                y




                                                       large
                    initial fuzzy set
m(x)
           reduce                  enlarge




                                                       medium
0.85

0.55




                                                       small
0.30
                                                                                              x



                               x
                                                                    small   medium   large


       Heuristics: a fuzzy set is moved away from x (towards x)
       and its support is reduced (enlarged), in order to
       reduce (enlarge) the degree of membership of x.

        März 2001                       Rudolf Kruse
                                                                                       NF
                                                                                       S     EURO



                                                                                                  UZZY
                  Training of Fuzzy Sets
Algorithm:
                                              Variations:
repeat
     for (all patterns) do                    • Adaptive learning rate
         accumulate parameter updates;        • Online-/Batch
         accumulate error;                      Learning
     end;
     modify parameters;                       • optimistic learning
until (no change in error);                     (n step look ahead)



                                local         Observing the error on
                              minimum         a validation set




      März 2001                Rudolf Kruse
                                                                  NF
                                                                  S    EURO



                                                                         UZZY
  Constraints for Training Fuzzy Sets

• Valid parameter values
• Non-empty intersection of                  1
  adjacent fuzzy sets
• Keep relative positions
                                             2
• Maintain symmetry
• Complete coverage
  (degrees of membership add up to
                                             3
  1 for each element)

                                             Correcting a partition after
                                             modifying the parameters


     März 2001                Rudolf Kruse
                                                                            NF
                                                                            SEURO



                                                                               UZZY
              4. Training Step: Pruning
Goal: remove variables, rules and fuzzy sets, in order to
improve interpretability and generalisation.




  März 2001              Rudolf Kruse
                                                            NF
                                                            SEURO



                                                               UZZY
                               Pruning
Algorithm:                                     Pruning Methods:

repeat                                         1. Remove variables
   select pruning method;                          (use correlations, information
                                                   gain etc.)
   repeat
       execute pruning step;                   2. Remove rules
       train fuzzy sets;                          (use rule performance)

        if (no improvement)                    3. Remove terms
        then undo step;                           (use degree of fulfilment)

   until (no improvement);                     4. Remove fuzzy sets
                                                  (use fuzziness)
until (no further method);


      März 2001                 Rudolf Kruse
                                                                               NF
                                                                               S    EURO



                                                                                      UZZY
     WBC Learning Result: Fuzzy Rules
R1: if uniformity of cell size is small and bare nuclei is fuzzy0 then benign
R2: if uniformity of cell size is large then malignant




      März 2001                        Rudolf Kruse
                                                                                NF
                                                                                SEURO



                                                                                   UZZY
   WBC Learning Result: Classification Performance

             Predicted Class
                malign            benign              not              sum
                                                   classified
   malign 228 (32.62%)         13      (1.86%) 0       (0%) 241         (34.99%)
   benign 15 (2.15%) 443 (63.38%) 0                    (0%) 458         (65.01%)
   sum    243 (34.76%) 456 (65.24%) 0                  (0%) 699        (100.00%)

      Estimated Performance on Unseen Data (Cross Validation)

 NEFCLASS-J:            95.42%              NEFCLASS-J (numeric): 94.14%
 Discriminant Analysis: 96.05%              Multilayer Perceptron:      94.82%
 C 4.5:                 95.10%              C 4.5 Rules:                95.40%


     März 2001                      Rudolf Kruse
                                                                               NF
                                                                               S   EURO



                                                                                     UZZY
WBC Learning Result: Fuzzy Sets
                              uniformity of cell size
                   sm               lg
            1.0


            0.5


            0.0
                  1.0   2.8        4.6      6.4     8.2   10.0

                                   bare nuclei
            1.0


            0.5


            0.0
                  1.0   2.8        4.6      6.4     8.2   10.0



März 2001                         Rudolf Kruse
                                                                 NF
                                                                 SEURO



                                                                    UZZY
            NEFCLASS-J




März 2001      Rudolf Kruse
                              NF
                              SEURO



                                 UZZY
                      Resources
Detlef Nauck, Frank Klawonn & Rudolf Kruse:

Foundations of Neuro-Fuzzy Systems
Wiley, Chichester, 1997, ISBN: 0-471-97151-0


Neuro-Fuzzy Software (NEFCLASS, NEFCON, NEFPROX):
http://www.neuro-fuzzy.de

Beta-Version of NEFCLASS-J:
http://www.neuro-fuzzy.de/nefclass/nefclassj


   März 2001                Rudolf Kruse
                                                NF
                                                S   EURO



                                                      UZZY
                      Conclusions
 Neuro-Fuzzy-Systems can be useful for knowledge discovery.

 Interpretability enables plausibility checks and improves
  acceptance.

 (Neuro-)Fuzzy systems exploit tolerance for sub-optimal
  solutions.

 Neuro-fuzzy learning algorithms must observe constraints in
  order not to jeopardise the semantics of the model.

 Not an automatic model creator, the user must work with the
  tool.

 Simple learning techniques support explorative data analysis.
    März 2001               Rudolf Kruse
                                                                  NF
                                                                  SEURO



                                                                     UZZY
               Download NEFCLASS-J
Download the free version of NEFCLASS-J at
http://fuzzy.cs.uni-magdeburg.de




   März 2001          Rudolf Kruse
                                             NF
                                             SEURO



                                                UZZY
Fuzzy Methods in Information Mining: Examples

here: Exploiting quantitative and qualitative
       information

 Fuzzy Data Analysis (Projects with Siemens)

 Information Fusion (EC Project)

 Dependency Analysis (Project with Daimler/Chrysler)




   März 2001            Rudolf Kruse
                                                  NF
                                                  S EURO



                                                        UZZY
Analysis of Daimler/Chrysler Database


 Database: ~ 18.500 passenger cars
            > 100 attributes per car

 Analysis of dependencies between special equipment and
  faults.


 Results used as a starting point for technical experts looking
  for causes.




   März 2001                Rudolf Kruse
                                                                   NF
                                                                   SEURO



                                                                      UZZY
        Learning Graphical Models

       data
         +
prior information




                          A        B


      Inducer                          C   local models




  März 2001         Rudolf Kruse
                                                     NF
                                                     S    EURO



                                                            UZZY
                      The Learning Problem
                                 known structure                   unknown structure
                                   A                   B               A                B

                                            C                                  C

 complete data            Statistical Parametric                Discrete Optimization over
                          Estimation (closed from eq.):         Structures (discrete search):
   A       B        C          statistical parameter fitting,        likelihood scores,
 <a4,      b3,      c1>        ML Estimation,                        MDL
                               Bayesian Inference, ...          Problem:
 <a3,      b2,      c4>                                         search complexity      heuristics

incomplete data Parametric Optimization:                        Combined Methods:
(missing values,               EM,                                  structured EM
hidden variables,...)          gradient descent, ...                only few approaches
                                                                Problems:
   A        B       C
                                                                    criterion for fit?
 <a4,       ?,      c1>                                             new variables?
 <a3,       b2,     ?>                                              local maxima?
                                                                    fuzzy values?

        März 2001                           Rudolf Kruse
                                                                                             NF
                                                                                             S      EURO



                                                                                                      UZZY
                     Possibility Theory
                                     fuzzy set induces possibility

                                           A   sup m 
                                                   A
      mcloudy
                                                              2
                                           0  55, 60 
1


                                                              3

                                     axioms
                                           0  0
       50       65   85   100   
                                              1
                                           A  B  max  A ,  B
                                           A  B  min  A ,  B



    März 2001                   Rudolf Kruse
                                                                      NF
                                                                      SEURO



                                                                            UZZY
General Structure of (most) Learning Algorithms for Graphical Models


 Use a criterion to measure the degree to which a
  network structure fits the data and the prior
  knowledge
  (model selection, goodness of hypergraph)

 Use a search algorithm to find a model that
  receives a high score by the criterion
  (optimal spanning tree, K2: greedy selection of
  parents, ...)


    März 2001                Rudolf Kruse
                                                                 NF
                                                                 S     EURO



                                                                         UZZY
Measuring the Deviation from an Independent Distribution

         Probability- and Information-based Measures

                  information gain *
                   identical with mutual information
                  information gain ratio *
                  g-function (Cooper and Herskovits)
                  minimum description length
                  gini index *

                       Possibilistic Measures
                  expected nonspecificity
                  specificity gain
                  specificity gain ratio
    (Measures marked with * originated from decision tree learning)
   März 2001                   Rudolf Kruse
                                                                      NF
                                                                      SEURO



                                                                         UZZY
       Data Mining Tool Clementine




März 2001        Rudolf Kruse
                                     NF
                                     SEURO



                                        UZZY
Analysis of Daimler/Chrysler Database

electrical       air con-      type of      type of     slippage
 roof top       ditioning      engine        tyres       control



             faulty           faulty           faulty
             battery        compressor         brakes


 Fictituous example:
 There are significantly more faulty batteries, if both
 air conditioning and electrical roof top are built
 into the car.

 März 2001                   Rudolf Kruse
                                                                   NF
                                                                   SEURO



                                                                      UZZY

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:8
posted:6/4/2012
language:English
pages:69