Incorporating Prior Knowledge in Fuzzy c-Regression Models

Document Sample
Incorporating Prior Knowledge in Fuzzy c-Regression Models Powered By Docstoc
					 Incorporating Prior Knowledge in
    Fuzzy c-Regression Models
     — Application to System
           Identification
        Janos Abonyi, Robert Babuska,
         Tibor Chovan, Ferenc Szeifert
University of Veszprém, Department of Process
 Engineering, P.O. Box 158, H-8201, Hungary
          www.fmt.vein.hu/softcomp
                      Outline
•   Fuzzy clustering
•   Fuzzy c-regression method
•   Application to system identification
•   What makes a good fuzzy system ?
•   Constrained fuzzy c-regression clustering
•   Discussion and Conclusion

                 Department of Process Engineering,   2
                      University of Veszprem,
           Fuzzy Clustering
• CLUSTERS:
  – Groups with uncertain boundaries
  – Each item has some membership in all groups



• A method to classify items into groups upon
  a SIMILARITY CRITERION.

               Department of Process Engineering,   3
                    University of Veszprem,
          Concept of fuzziness
A real-life example                                Warm room example

Wife: Do you love me?                     1

Husband (Boolean logician): Yes.                                           Fuzzy
Wife: How much?
                                                                        Crisp

                                          0
                                          10           15        20       25         30
                                                            Temperature [°C]


                      Department of Process Engineering,                         4
                           University of Veszprem,
Fuzzy c-regression clustering
         y




                                                     x

      Rule i:    IF x is Ai THEN y = i x

PROBLEM: HOW TO CALCULATE i and Ai ?


                Department of Process Engineering,       5
                     University of Veszprem,
             The original algorithm
• The regression models                                     yk  f i ( x k , i )

• The prediction error (distance)                          Ei,k (θi )  ( yk  fi (xk , θi ))2

                                                                             c      N
• The objective function                                  Em (U,{θi }      i,k m Ei,k (θi )
                                                                           i 1 k 1
• The algorithm:
  0., Initialisation
  1., arg min Em (U,{θi }                       (weighted LS)
                θi
                                      1
        i , k (l )                                     , 1  i  c, 1  k  N
  2.
                         j 1
                            c
                               Ei,k / E j,k 2 /(m1)

                                  Department of Process Engineering,                             6
                                       University of Veszprem,
Application to system identification
 • NARX model
     yk  f (x k )  f ( yk 1, , yk  n y 1,uk  nd , , uk  nu  nd 1, )

 • Example: First-order Hammerstein system

               u                          v                  0.1z 1      y
                        vu   2                 G( z ) 
                                                           1  0.9 z 1


       rl : IF [uk , yk 1] is Al THEN            yk  al yk 1  bl uk  cl



                        Department of Process Engineering,                     7
                             University of Veszprem,
What is the problem?
             1


            0.8


            0.6


vs and i   0.4


            0.2


             0


        -0.2


        -0.4
                  0   0.1   0.2   0.3   0.4   0.5    0.6   0.7   0.8   0.9   1
                                              u(k)


                         a      b      c
                      0.8999 0.0824 -0.0131
                      0.8988 0.0794 -0.0025
                      0.8991 0.1382 -0.0420
                        Department of Process Engineering,                       8
                             University of Veszprem,
    What makes a good fuzzy system?
Accuracy
    Approximation or Classification error
    Certainty degree
    Local models/global models !!!

Transparency and Interpretability
    Moderate number of rules
    Distinguishability!!!
    Normality
    Coverage

                      Department of Process Engineering,   9
                           University of Veszprem,
   Explicit use of prior knowledge:
          Constrained parameter estimation
• Rule-consequents define a convex region (polytope).

• This polytope can be constrained by global linear
  constraints.

• The individual rule-consequents can be constrained
  separately local linear constraints.

Knowledge: The rule consequents should be different
     relative linear constraints.
                    Department of Process Engineering,   10
                         University of Veszprem,
       Graphical representation
rl : IF [uk , yk 1] is Al THEN                      yk   l ,1 yk 1   l ,2uk   l ,3
             Global constraints



             i,2
                                                            [3,1 ,3,2 ]


                                  [4,1 ,4,2 ]


1,2 <4,2                                  [2,1 ,2,2 ]

                                                                      Local constraints

                            [1,1 ,1,2 ]
  Relative
  constraints
                                                                            i,1
                           Department of Process Engineering,                                11
                                University of Veszprem,
    Quadratic programming
              1 T       T 
          min   H   d  
           
              2            
                   
• H and d contain the measured input-output data
•  and  represents the a priori knowledge based
  constraints

              Department of Process Engineering,   12
                   University of Veszprem,
          Applied Constraints
• Upper and lower bounds:
                               nb

                              b        i
                                         l


              K min 
                l              i 1
                                   na
                                              K max
                                                 l


                            1   ail
                                 i 1


• Inequality constraints for QP:
                           na
                               l
                                     nb
             K   l
                 min
                       1   ai    bil  0
                                
                        i 1  i 1
                           na
                               l
                                     nb
            K   l
                 max
                       1   ai    bil  0
                                
                        i 1  i 1

                       Department of Process Engineering,   13
                            University of Veszprem,
This is a good fuzzy model !
               1

              0.8

              0.6


  vs and i   0.4

              0.2

               0

          -0.2

          -0.4

          -0.6

          -0.8
                    0   0.1    0.2   0.3   0.4   0.5    0.6   0.7   0.8   0.9   1
                                                 u(k)

                                 a      b      c
                              0.8996 0.0204 0.0151
                              0.8996 0.0408 -0.0025
                              0.8996 0.1560 -0.0590
                              Department of Process Engineering,                    14
                                   University of Veszprem,
                  Conclusions
  Expert                Mechanistic                       Measured
knowledge               knowledge                           data


  Model       Parameter             Steady-state data      Dynamic data
 structure    constraints              and model         and (local) model


                       Optimization



                            Model


 Accuracy,             Interpretability,                Generalisation
performance              applicability                   (wide range)
                 Department of Process Engineering,                  15
                      University of Veszprem,