Neuro-Fuzzy Data Analysis by yurtgc548

VIEWS: 8 PAGES: 69

• pg 1
```									Industrial Applications of
Neuro-Fuzzy Networks

Prof. Dr. Rudolf Kruse
University of Magdeburg
Faculty of Computer Science
Magdeburg, Germany
rudolf.kruse@cs.uni-magdeburg.de

NF
SEURO

UZZY
Example: Continously Adapting Gear Shift Schedule in VW New Beetle

classification of driver / driving situation         gear shift
by fuzzy logic                        computation

fuzzification           inference        defuzzifi-     interpolation
machine          cation
accelerator pedal

filtered speed of
determination
accelerator pedal
of speed limits
rule                           for shifting      gear
number of                                  sport
base                           into higher or    selection
changes in                                 factor [t]
lower gear
pedal direction
depending on
sport factor

sport factor [t-1]

März 2001                    Rudolf Kruse
NF
S     EURO

UZZY
Continously Adapting Gear Shift Schedule: Technical Details

 Mamdani controller with 7 rules

 Optimized program                                           AG4

24 Byte RAM

702 Byte ROM
}  on Digimat

 Runtime 80 ms
12 times per second a new sport factor is assigned

 How to generate knowledge automatically from data?

März 2001                  Rudolf Kruse
NF
SEURO

UZZY
Learning from Examples (Observations, Databases)

 Statistics:          parameter fitting, structure
identification, inference method,
model selection
 Machine Learning:    computational learning (PAC
learning), inductive learning, learning
decision trees, concept learning, ...
 Neural Networks:     learning from data
 Cluster Analysis:    unsupervised classification

 Learning Problem is transformed into an optimization problem.
 How to use these methods in fuzzy systems?

März 2001             Rudolf Kruse
NF
SEURO

UZZY
Function Approximation with Fuzzy Rules

if x is large then y is large

y
output value

x
current input value

März 2001                 Rudolf Kruse
NF
SEURO

UZZY
How to Derive a Fuzzy Controller Automatically from Observed Process Data

ut
• Function approximation

tp
ou
input
current input value

• Perform fuzzy cluster analysis of input-output data (FCM, GK, GG, ...)
• Project clusters
• Obtain fuzzy rules of the kind: „If x is small then y is medium“

März 2001                         Rudolf Kruse
NF
SEURO

UZZY
Fuzzy Cluster Analysis
 Classification of a given data set X = {x1, ..., xn}  p into c
clusters.
 Membership degree of datum xk to class i is uik.
 Representation of cluster i by prototype vi  p.

Formal: Minimisation of functional:

JX, U, v    uik  d2 v i, x k 
c     n
m

i1 k 1

under constraints n
u     ik
 0 for all i   , ..., c
1
k 1
c

u     ik
 1 for all k   , ..., n
1

März 2001
i1

Rudolf Kruse
NF
SEURO

UZZY
Simplest Algorithm: Fuzzy-c-Means (FCM)

d v i , x k   v i  x k
2                           2

Iterative Procedure (with random initialisation of prototypes vi)

 u 
n
m
ik        xk
1
ui k                                         and       vi    k 1

 u 
1                             n

 d2 v i , x k  
m
c                         m 1

  d2 v , x 

j 1 
                                k 1
ik

j     k 

FCM is searching for equally large clusters in form of (hyper-)balls.

März 2001                            Rudolf Kruse
NF
SEURO

UZZY
Examples

März 2001    Rudolf Kruse
NF
SEURO

UZZY
Fuzzy Cluster Analysis
 Fuzzy C-Means: simple, looks for spherical
clusters of same size, uses Euclidean distance
 Gustafson & Kessel: looks for hyper-ellipsoidal
clusters of same size, distance via matrices
 Gath & Geva: looks for hyper-ellipsoidal clusters
of arbitrary size, distance via matrices
 Axis-parallel variations exist that use diagonal
matrices (computationally less expensive and less
loss of information when rules are created)

März 2001           Rudolf Kruse
NF
SEURO

UZZY
Fuzzy Cluster Analysis with DataEngine

März 2001      Rudolf Kruse
NF
S   EURO

UZZY
Construct Fuzzy Sets by Cluster Projection
u(x)                                             Approximation by a
triangular fuzzy set
1
Convex hull of the discrete
degrees of membership

Connection of the discrete
degrees of membership

x
Projecting a cluster means to project the degrees of membership
of the data on the single dimensions: Histograms are obtained.

März 2001             Rudolf Kruse
NF
S     EURO

UZZY
FCLUSTER: Tool for Fuzzy Cluster Analysis

März 2001      Rudolf Kruse
NF
S   EURO

UZZY
Introduction
 Building a fuzzy system requires
prior knowledge (fuzzy rules, fuzzy sets)

manual tuning: time consuming and error-prone

 Therefore: Support this process by learning
learning fuzzy rules (structure learning)
learning fuzzy set (parameter learning)

Approaches from Neural Networks can be used

März 2001           Rudolf Kruse
NF
SEURO

UZZY
Learning Fuzzy Sets: Problems in Control

 Reinforcement learning must be used to compute an error
value
(note: the correct output is unknown)

 After an error was computed, any fuzzy set learning
procedures can be used

 Example: GARIC (Berenji/Kedhkar 1992)
online approximation to gradient-descent

 Example: NEFCON (Nauck/Kruse 1993)
online heuristic fuzzy set learning using a
rule-based fuzzy error measure

März 2001               Rudolf Kruse
NF
S   EURO

UZZY
März 2001   Rudolf Kruse
NF
SEURO

UZZY
Neuro-Fuzzy Systems in Data Analysis

 Neuro-Fuzzy System:
System of linguistic rules (fuzzy rules).
Not rules in a logical sense, but function
approximation.
Fuzzy rule = vague prototype / sample.

 Neuro-Fuzzy-System:
Adding a learning algorithm inspired by neural
networks.
Feature: local adaptation of parameters.

März 2001            Rudolf Kruse
NF
SEURO

UZZY
Example:      Prognosis of the Daily Proportional Changes of the DAX at
the Frankfurter Stock Exchange (Siemens)

 Database: time series from 1986 - 1997

DAX                                   Composite DAX
German 3 month interest rates         Return Germany
Morgan Stanley index Germany          Dow Jones industrial index
DM / US-\$                             US treasury bonds
Gold price                            Nikkei index Japan
Morgan Stanley index Europe           Price earning ratio

März 2001                      Rudolf Kruse
NF
S   EURO

UZZY
Fuzzy Rules in Finance
 Trend Rule
IF      DAX = decreasing AND US-\$ = decreasing
THEN DAX prediction = decrease
WITH high certainty
 Turning Point Rule
IF       DAX = decreasing AND US-\$ = increasing
THEN DAX prediction = increase
WITH low certainty
 Delay Rule
IF      DAX = stable AND US-\$ = decreasing
THEN DAX prediction = decrease
WITH very high certainty
 In general
IF        x1 is m1 AND x2 is m2
THEN y = h
WITH weight k

März 2001                    Rudolf Kruse
NF
SEURO

UZZY
Classical Probabilistic Expert Opinion Pooling Method

 DM analyzes each source (human expert, data +
forecasting model) in terms of (1) Statistical accuracy,
and (2) Informativeness by asking the source to asses
quantities (quantile assessment)

 DM obtains a “weight” for each source

 DM “eliminates” bad sources

 DM determines the weighted sum of source outputs

 Determination of “Return of Invest”
März 2001            Rudolf Kruse
NF
S     EURO

UZZY
 E experts, R quantiles for N quantities
 each expert has to asses R·N values
 stat. Accuracy:
R      si
C         1   R 2 N  I s, p ,
2
I s, p    si ln
i 0    p
 information score:
1 N                     R 1             pr 1 
I   lnvi, R 1  vi,o    pr 1 ln                
N i 1                   r 1       vi,r  vi,r 1 
ce  I e  id ce 
 weight for expert e:             we 
E
e1ce  I e  id e ce 
E

 outputt=  we  outputt
e
e 1
T

 roi =  yt   sign outputt
DM

t 1
März 2001                           Rudolf Kruse
NF
SEURO

UZZY
Formal Analysis

 Sources of information
R1     rule set given by expert 1
R2     rule set given by expert 2
D      data set (time series)

 Operator schema
fuse (R1, R2) fuse two rule sets
induce(D)            induce a rule set from D
revise(R, D)         revise a rule set R by D

März 2001            Rudolf Kruse
NF
SEURO

UZZY
Formal Analysis

 Strategies:
 fuse(fuse (R1, R2), induce(D))
 revise(fuse(R1, R2), D)            
 fuse(revise(R1, D), revise(R2, D))

 Technique: Neuro-Fuzzy Systems
 Nauck, Klawonn, Kruse, Foundations of Neuro-Fuzzy
Systems, Wiley 97
 SENN (commercial neural network environment, Siemens)

März 2001             Rudolf Kruse
NF
S    EURO

UZZY
From Rules to Neural Networks

1. Evaluation of membership degrees

2. Evaluation of rules (rule activity)
x   j 1 m c( ,js)  xi 
n        r
ml: IR  [0,1] ,
l    D

3. Accumulation of rule inputs and normalization
kl m l  x 
NF: IR  IR, x  l 1 wl
n                   r

          k j m j x 
r
j 1

März 2001                      Rudolf Kruse
NF
SEURO

UZZY
Neuro-Fuzzy Architecture

März 2001        Rudolf Kruse
NF
SEURO

UZZY
The Semantics-Preserving Learning Algorithm

Reduction of the dimension of the weight space
1. Membership functions of different inputs share their parameters,
e.g.
m dax  m cdax
stable  stable

2. Membership functions of the same input variable are not allowed to pass
each other, they must keep their original order,
e.g.
m decreasing  m stable  m increasing

Benefits:      the optimized rule base can still be interpreted
 the number of free parameters is reduced

März 2001                      Rudolf Kruse
NF
S    EURO

UZZY
Return-on-Investment Curves of the Different Models

Validation data from March 01, 1994 until April 1997

März 2001                      Rudolf Kruse
NF
SEURO

UZZY
A Neuro-Fuzzy System
 is a fuzzy system trained by heuristic learning
techniques derived from neural networks

 can be viewed as a 3-layer neural network with fuzzy
weights and special activation functions

 is always interpretable as a fuzzy system

 uses constraint learning procedures

 is a function approximator (classifier, controller)

März 2001             Rudolf Kruse
NF
SEURO

UZZY
Learning Fuzzy Rules
 Cluster-oriented approaches
=> find clusters in data, each cluster is a rule

 Hyperbox-oriented approaches
=> find clusters in the form of hyperboxes

 Structure-oriented approaches
=> used predefined fuzzy sets to structure the
data space, pick rules from grid cells

März 2001            Rudolf Kruse
NF
SEURO

UZZY
Hyperbox-Oriented Rule Learning
y
Search for hyperboxes in
the data space
Create fuzzy rules by
projecting the hyperboxes
Fuzzy rules and fuzzy
sets are created at the
same time
x   Usually very fast

März 2001    Rudolf Kruse
NF
SEURO

UZZY
Hyperbox-Oriented Rule Learning
y              y                     y          y

x              x                x              x

 Detect hyperboxes in the data, example: XOR function
 Advantage over fuzzy cluster anlysis:
 No loss of information when hyperboxes are represented as
fuzzy rules
 Not all variables need to be used, don‘t care variables can be
discovered
 Disadvantage: each fuzzy rules uses individual fuzzy sets,
i.e. the rule base is complex.

März 2001                Rudolf Kruse
NF
S  EURO

UZZY
Structure-Oriented Rule Learning
y
Provide initial fuzzy sets for
large

all variables.
The data space is partitioned
by a fuzzy grid
medium

Detect all grid cells that
contain data (approach by
Wang/Mendel 1992)
Compute best consequents and
small

select best rules (extension by
Nauck/Kruse 1995,
x NEFCLASS model)

small   medium          large

März 2001                    Rudolf Kruse
NF
S   EURO

UZZY
Structure-Oriented Rule Learning
 Simple: Rule base available after two cycles through the
training data
 1. Cycle: discover all antecedents
 2. Cycle: determine best consequents

 Missing values can be handled
 Numeric and symbolic attributes can be processed at the
same time (mixed fuzzy rules)

 Advantage: All rules share the same fuzzy sets
 Disadvantage: Fuzzy sets must be given
März 2001                Rudolf Kruse
NF
SEURO

UZZY
Learning Fuzzy Sets
 Gradient descent procedures
only applicable, if differentiation is possible, e.g.
for Sugeno-type fuzzy systems.

 Special heuristic procedures that do not use
gradient information.

 The learning algorithms are based on the idea of
backpropagation.

März 2001            Rudolf Kruse
NF
SEURO

UZZY
Learning Fuzzy Sets: Constraints
 Mandatory constraints:
 Fuzzy sets must stay normal and convex
 Fuzzy sets must not exchange their relative
positions (they must not „pass“ each other)
 Fuzzy sets must always overlap
 Optional constraints
 Fuzzy sets must stay symmetric
 Degrees of membership must add up to 1.0
 The learning algorithm must enforce these
constraints.

März 2001           Rudolf Kruse
NF
SEURO

UZZY
Different Neuro-Fuzzy Approaches
 ANFIS (Jang, 1993)
no rule learning, gradient descent fuzzy set learning, function approximator
 GARIC (Berenji/Kedhkar, 1992)
no rule learning, gradient descent fuzzy set learning, controller
 NEFCON (Nauck/Kruse, 1993)
structure-oriented rule learning, heuristic fuzzy set learning, controller
 FuNe (Halgamuge/Glesner, 1994)
combinatorical rule learning, gradient descent fuzzy set learning, classifier
 Fuzzy RuleNet (Tschichold-Gürman, 1995)
hyperbox-oriented rule learning, no fuzzy set learning, classifier
 NEFCLASS (Nauck/Kruse, 1995)
structure-oriented rule learning, heuristic fuzzy set learning, classifier
 Learning Fuzzy Graphs (Berthold/Huber, 1997)
hyperbox-oriented rule learning, no fuzzy set learning, function approximator
 NEFPROX (Nauck/Kruse, 1997)
structure-oriented rule learning, heuristic fuzzy set learning, function approx.

März 2001                         Rudolf Kruse
NF
SEURO

UZZY
Example: Medical Diagnosis
 Results from patients tested for breast cancer
(Wisconsin Breast Cancer Data).

 Decision support: Do the data indicate a malignant or a benign
case?

 A surgeon must be able to check the classification for
plausibility.

 We are looking for a simple and interpretable classifier:
knowledge discovery.

März 2001                Rudolf Kruse
NF
S   EURO

UZZY
Example: WBC Data Set
 699 cases (16 cases have missing values).

 2 classes: benign (458), malignant (241).

 9 attributes with values from {1, ... , 10}
(ordinal scale, but usually interpreted as a numerical
scale).

 Experiment: x3 and x6 are interpreted as nominal
attributes.

 x3 and x6 are usually seen as „important“ attributes.

März 2001             Rudolf Kruse
NF
SEURO

UZZY
Applying NEFCLASS-J
 Tool for developing Neuro-Fuzzy Classifiers

 Written in JAVA

 Free version for research available

 Project started at Neuro-Fuzzy Group of University of
Magdeburg, Germany

März 2001            Rudolf Kruse
NF
S    EURO

UZZY
NEFCLASS: Neuro-Fuzzy Classifier

Output variables (class labels)

Unweighted connections

Fuzzy rules

Fuzzy sets (antecedents)

Input variables (attributes)

März 2001   Rudolf Kruse
NF
SEURO

UZZY
NEFCLASS: Features
 Automatic induction of a fuzzy rule base from data
 Training of several forms of fuzzy sets
 Processing of numeric and symbolic attributes
 Treatment of missing values (no imputation)
 Automatic pruning strategies
 Fusion of expert knowledge and knowledge obtained
from data

März 2001            Rudolf Kruse
NF
SEURO

UZZY
Representation of Fuzzy Rules
Example: 2 Rules

c1              c2   R1: if x is large and y is small, then class is c1.

R2: if x is large and y is large, then class is c2.

The connections x  R1 and x  R2
R1              R2
are linked.
small
large
large     The fuzzy set large is a shared weight.

x                 y
That means the term large has always the
same meaning in both rules.

März 2001                      Rudolf Kruse
NF
SEURO

UZZY
1. Training Step: Initialisation
Specify initial fuzzy partitions for all input variables
y

large
c1        c2

medium
small                                 x

x                      y
small   medium   large

März 2001                Rudolf Kruse
NF
S     EURO

UZZY
2. Training Step: Rule Base
Algorithm:
Variations:
for (all patterns p) do
find antecedent A,                       Fuzzy rule bases can
such that A( p) is maximal;              also be created by using
if (A  L) then add A to L;              prior knowledge, fuzzy
end;                                             cluster analysis, fuzzy
decision trees, genetic
for (all antecedents A  L) do                   algorithms, ...
find best consequent C for A;
create rule base candidate R = (A,C);
Determine the performance of R;
Add R to B;
end;
Select a rule base from B;

März 2001                  Rudolf Kruse
NF
S  EURO

UZZY
Selection of a Rule Base

e
Pe rformanc of a Rule :
• Order rules by performance.
• Either select

 
N
  1
1                                          the best r rules or
Pr 
c
Rr x p , with             the best r/m rules per class.
N   p 1
• r is either given or is
determined automatically such
that all patterns are covered.
0 if class(x p )  con( Rr ),

c
1 otherwise.


März 2001                 Rudolf Kruse
NF
S   EURO

UZZY
Rule Base Induction
NEFCLASS uses a modified Wang-Mendel procedure
y

large
c1        c2

medium
R1        R2    R3

small                                 x

x                        y
small   medium   large

März 2001               Rudolf Kruse
NF
S     EURO

UZZY
Computing the Error Signal
Error Signal            Fuzzy Error ( jth output):

E j  sgn(d )  1   (d ) , with d  t j  o j
2
c1        c2                                             ad   

d      

and  :   0, 1,  (d )  e       max   

(t : correct output, o : actual output)
R1        R2    R3

Rule Error:

x                        y   Er   r 1   r    Econ( Rr ) , with 0    1

März 2001                 Rudolf Kruse
NF
SEURO

UZZY
3. Training Step: Fuzzy Sets
x a                 
 b  a if x  [a, b) 
Example:                                                                    
triangular                                             c  x                
membership       m a ,b,c :   [0,1], m a ,b,c ( x)         if x  [b, c] 
function.                                              c  b                
                     
0      otherwise
                     

 m ( x)            if E  0
f 
 1  m ( x)  otherwise
Parameter
updates for an
antecedent       b  f  E  c  a   sgn(x  b)
fuzzy set.       a   f  E  b  a   b
c  f  E  c  b   b

März 2001                Rudolf Kruse
NF
S     EURO

UZZY
Training of Fuzzy Sets
y

large
initial fuzzy set
m(x)
reduce                  enlarge

medium
0.85

0.55

small
0.30
x

x
small   medium   large

Heuristics: a fuzzy set is moved away from x (towards x)
and its support is reduced (enlarged), in order to
reduce (enlarge) the degree of membership of x.

März 2001                       Rudolf Kruse
NF
S     EURO

UZZY
Training of Fuzzy Sets
Algorithm:
Variations:
repeat
for (all patterns) do                    • Adaptive learning rate
accumulate parameter updates;        • Online-/Batch
accumulate error;                      Learning
end;
modify parameters;                       • optimistic learning
until (no change in error);                     (n step look ahead)

local         Observing the error on
minimum         a validation set

März 2001                Rudolf Kruse
NF
S    EURO

UZZY
Constraints for Training Fuzzy Sets

• Valid parameter values
• Non-empty intersection of                  1
adjacent fuzzy sets
• Keep relative positions
2
• Maintain symmetry
• Complete coverage
(degrees of membership add up to
3
1 for each element)

Correcting a partition after
modifying the parameters

März 2001                Rudolf Kruse
NF
SEURO

UZZY
4. Training Step: Pruning
Goal: remove variables, rules and fuzzy sets, in order to
improve interpretability and generalisation.

März 2001              Rudolf Kruse
NF
SEURO

UZZY
Pruning
Algorithm:                                     Pruning Methods:

repeat                                         1. Remove variables
select pruning method;                          (use correlations, information
gain etc.)
repeat
execute pruning step;                   2. Remove rules
train fuzzy sets;                          (use rule performance)

if (no improvement)                    3. Remove terms
then undo step;                           (use degree of fulfilment)

until (no improvement);                     4. Remove fuzzy sets
(use fuzziness)
until (no further method);

März 2001                 Rudolf Kruse
NF
S    EURO

UZZY
WBC Learning Result: Fuzzy Rules
R1: if uniformity of cell size is small and bare nuclei is fuzzy0 then benign
R2: if uniformity of cell size is large then malignant

März 2001                        Rudolf Kruse
NF
SEURO

UZZY
WBC Learning Result: Classification Performance

Predicted Class
malign            benign              not              sum
classified
malign 228 (32.62%)         13      (1.86%) 0       (0%) 241         (34.99%)
benign 15 (2.15%) 443 (63.38%) 0                    (0%) 458         (65.01%)
sum    243 (34.76%) 456 (65.24%) 0                  (0%) 699        (100.00%)

Estimated Performance on Unseen Data (Cross Validation)

 NEFCLASS-J:            95.42%              NEFCLASS-J (numeric): 94.14%
 Discriminant Analysis: 96.05%              Multilayer Perceptron:      94.82%
 C 4.5:                 95.10%              C 4.5 Rules:                95.40%

März 2001                      Rudolf Kruse
NF
S   EURO

UZZY
WBC Learning Result: Fuzzy Sets
uniformity of cell size
sm               lg
1.0

0.5

0.0
1.0   2.8        4.6      6.4     8.2   10.0

bare nuclei
1.0

0.5

0.0
1.0   2.8        4.6      6.4     8.2   10.0

März 2001                         Rudolf Kruse
NF
SEURO

UZZY
NEFCLASS-J

März 2001      Rudolf Kruse
NF
SEURO

UZZY
Resources
Detlef Nauck, Frank Klawonn & Rudolf Kruse:

Foundations of Neuro-Fuzzy Systems
Wiley, Chichester, 1997, ISBN: 0-471-97151-0

Neuro-Fuzzy Software (NEFCLASS, NEFCON, NEFPROX):
http://www.neuro-fuzzy.de

Beta-Version of NEFCLASS-J:
http://www.neuro-fuzzy.de/nefclass/nefclassj

März 2001                Rudolf Kruse
NF
S   EURO

UZZY
Conclusions
 Neuro-Fuzzy-Systems can be useful for knowledge discovery.

 Interpretability enables plausibility checks and improves
acceptance.

 (Neuro-)Fuzzy systems exploit tolerance for sub-optimal
solutions.

 Neuro-fuzzy learning algorithms must observe constraints in
order not to jeopardise the semantics of the model.

 Not an automatic model creator, the user must work with the
tool.

 Simple learning techniques support explorative data analysis.
März 2001               Rudolf Kruse
NF
SEURO

UZZY
Download NEFCLASS-J
Download the free version of NEFCLASS-J at
http://fuzzy.cs.uni-magdeburg.de

März 2001          Rudolf Kruse
NF
SEURO

UZZY
Fuzzy Methods in Information Mining: Examples

here: Exploiting quantitative and qualitative
information

 Fuzzy Data Analysis (Projects with Siemens)

 Information Fusion (EC Project)

 Dependency Analysis (Project with Daimler/Chrysler)

März 2001            Rudolf Kruse
NF
S EURO

UZZY
Analysis of Daimler/Chrysler Database

 Database: ~ 18.500 passenger cars
> 100 attributes per car

 Analysis of dependencies between special equipment and
faults.

 Results used as a starting point for technical experts looking
for causes.

März 2001                Rudolf Kruse
NF
SEURO

UZZY
Learning Graphical Models

data
+
prior information

A        B

Inducer                          C   local models

März 2001         Rudolf Kruse
NF
S    EURO

UZZY
The Learning Problem
known structure                   unknown structure
A                   B               A                B

C                                  C

complete data            Statistical Parametric                Discrete Optimization over
Estimation (closed from eq.):         Structures (discrete search):
A       B        C          statistical parameter fitting,        likelihood scores,
<a4,      b3,      c1>        ML Estimation,                        MDL
Bayesian Inference, ...          Problem:
<a3,      b2,      c4>                                         search complexity      heuristics

incomplete data Parametric Optimization:                        Combined Methods:
(missing values,               EM,                                  structured EM
hidden variables,...)          gradient descent, ...                only few approaches
Problems:
A        B       C
criterion for fit?
<a4,       ?,      c1>                                             new variables?
<a3,       b2,     ?>                                              local maxima?
fuzzy values?

März 2001                           Rudolf Kruse
NF
S      EURO

UZZY
Possibility Theory
 fuzzy set induces possibility

 A   sup m 
A
mcloudy
2
 0  55, 60 
1

3

 axioms
 0  0
50       65   85   100   
    1
 A  B  max  A ,  B
 A  B  min  A ,  B

März 2001                   Rudolf Kruse
NF
SEURO

UZZY
General Structure of (most) Learning Algorithms for Graphical Models

 Use a criterion to measure the degree to which a
network structure fits the data and the prior
knowledge
(model selection, goodness of hypergraph)

 Use a search algorithm to find a model that
receives a high score by the criterion
(optimal spanning tree, K2: greedy selection of
parents, ...)

März 2001                Rudolf Kruse
NF
S     EURO

UZZY
Measuring the Deviation from an Independent Distribution

Probability- and Information-based Measures

 information gain *
identical with mutual information
 information gain ratio *
 g-function (Cooper and Herskovits)
 minimum description length
 gini index *

Possibilistic Measures
 expected nonspecificity
 specificity gain
 specificity gain ratio
(Measures marked with * originated from decision tree learning)
März 2001                   Rudolf Kruse
NF
SEURO

UZZY
Data Mining Tool Clementine

März 2001        Rudolf Kruse
NF
SEURO

UZZY
Analysis of Daimler/Chrysler Database

electrical       air con-      type of      type of     slippage
roof top       ditioning      engine        tyres       control

faulty           faulty           faulty
battery        compressor         brakes

Fictituous example:
There are significantly more faulty batteries, if both
air conditioning and electrical roof top are built
into the car.

März 2001                   Rudolf Kruse
NF
SEURO

UZZY

```
To top