Self Tuning of Fuzzy Rules with Different Types of Membership by nikeborome

VIEWS: 4 PAGES: 8

									                                                                                                                                   1



Self-Tuning of Fuzzy Rules with Different Types of Membership Functions for
                   Multi Input and Output Learning Data
                                 ∗ Eikou Gonda , Hitoshi Miyata ,†Masaaki Ohkita
                                        ∗ Department of Electrical Engineering
                                       Yonago National College of Technology
                                    4448 Hikona, Yonago, Tottori 683-8502, Japan
                   ∗Phone: +81-859-24-5117, Fax: +81-859-24-5009, E-mail: gonda@yonago-k.ac.jp
                    Phone: +81-859-24-5121, Fax: +81-859-24-5009, E-mail: miyata@yonago-k.ac.jp
                                 †Department of Electrical and Electronic Engineering
                          Tottori University, 4-101 Koyama-Minami, Tottori 680-8552, Japan
                  Phone: +81-857-31-5699, Fax: +81-857-31-0880, E-mail: mohkita@ele.tottori-u.ac.jp


Abstract - With the progress of practical applications of the      and large scale change of fuzzy system for increase of the
fuzzy theory, it is becoming increasingly difficult to design and   number of inputs and outputs.
tune larger and more complicated systems with the help of          Therefore, we use a technique of genetic algorithm for
fuzzy knowledge alone. Various methods of tuning the fuzzy         the optimization of fuzzy reasoning using the steepest
reasoning rules used for the approximation of given input-
                                                                   descent method[GON 04]. This new method using the
output data intended for the solution of such problems have
been proposed. This proposal includes methods combining
                                                                   genetic algorithm can select some kinds of membership
fuzzy reasoning and neural networks or methods of tuning the       functions(MSFs), delete some lengthy rules, and optimize
fuzzy reasoning rules by using optimization techniques based       MSFs. Besides, this new method can improve generaliza-
on genetic algorithms. However, these methods cannot cope          tion ability as a result of selection of MSFs adapting to the
with many problems completely, for example, the problems of        model. The advantages of this proposal are proved by nu-
rapid increase of the number of rules and large scale change       merical examples with multi input and output learning data
of fuzzy system for increase of the number of inputs and out-      involving function approximations.
puts.
To overcome these problems, we use a technique of genetic
algorithm for the optimization of fuzzy reasoning using the
steepest descent method. This new method using the ge-              II. FUZZY REASONING USING DIFFERENT TYPES
                                                                                      OF MSFS
netic algorithm can select some kinds of membership func-
tions(MSFs), delete some lengthy rules, and hence optimize
MSFs. In addition, this new method can improve generaliza-         A. Simplified fuzzy reasoning
tion ability as a result of selection of MSFs adapting to the
model. The advantages of this new method are proved by nu-
merical examples with multi input and output learning data         A method of the fuzzy reasoning described in this paper is a
involving function approximations.                                 simplified fuzzy reasoning whose consequence is given by
                                                                   crisp values. An organization of the simplified fuzzy rea-
Keywords— fuzzy control, steepest descent method, optimiza-        soning is shown in a form of the neural network architec-
tion of fuzzy reasoning, selecting of shapes of membership
functions, genetic algorithm, multi inputs and outputs             ture. Now, let x1 , · · · , xm be input signals to the controller,
                                                                   and yk be the output, then the i-th rule of the simplified
                    I. INTRODUCTION                                fuzzy reasoning is as follows:

In the tuning of fuzzy rules, a method using a genetic algo-           Rule i : if x1 is A1i and x2 is A2i and · · ·
rithm[FUK 95],[OHK 98], a method combining fuzzy rea-                       and xj is Aji and · · · and xm is Ami
soning with a neural network[NOM 92],[MIY 96],[GON
03], and so on have been proposed. In a use of the former                               then yk is wki (i = 1, · · · , n)        (1)
by a genetic algorithm, the solutions are got necessarily
and theoretically. But the method searches the solutions in        where Aji (j = 1, 2, · · · , m) shows the j-th fuzzy vari-
global solutions’s space, so that it necessitates much time to     able of the premises in the i-th rule and wki shows the i-
get the best solutions. On the other hand the latter method        th consequence to the k(k = 1, 2, · · · , l)-th output with a
has the advantage of reaching the solutions using a steepest       real value. With an aid of the product-sum-gravity method,
descent method in a short time. But the solutions some-            for given input signals x1 , · · · , xm its compatibility µi for
times result in suboptimal solutions. Besides, these meth-         those rules can be determined in the hidden layer. The out-
ods cannot cope with many problems completely, for exam-           put yk can be obtained by estimating the center of gravity
ple, the problems of rapid increase of the number of rules         in the output layer.
                                                                                Aji (xj ) = exp{−(xj − aji )2 /bji }          (5)
                               n
                                     µi · wki                       The learning procedures for the fuzzy control are executed
                               i=1                                  by the steepest descent method. For its implementation, the
                      yk =           n                        (2)
                                                                    following performance criterion E should be minimized,
                                         µi                         which is expressed by square of error between given data
                                   i=1                               ∗
                                                                    yk and the reasoning output yk :

where                                                                                                  ∗
                                                                                                (yk − yk )2
                                                                                           E=                                 (6)
                                                                                                    2
            µi = A1i (x1 )A2i (x2 ) · · · Ami (xm )           (3)

The procedures determining these fuzzy rules are ex-                The procedures for adjusting the parameters of both isosce-
pressed by using a neural network. Fuzzy rules are tuned            les triangular MSFs and bell-shaped MSFs can be ex-
with supervised data.                                               pressed as follows:


                                                                                                                 ∂E
                                                                                  aji (t + 1) = aji (t) − Ka                  (7)
                                                                                                                 ∂aji

                                                                                                                 ∂E
                                                                                   bji (t + 1) = bji (t) − Kb                 (8)
                                                                                                                 ∂bji

                                                                                                                   ∂E
                                                                                 wki (t + 1) = wki (t) − Kw                   (9)
                                                                                                                  ∂wki

                                                                    where Ka , Kb and Kw are learning coefficients for adjust-
                                                                    ing the training cycles. The variable t denotes the t-th iter-
                                                                    ative computation. Through these adjustment, the desired
                                                                    inference rules can be obtained step by step. Suppose that
                                                                    we have P pairs of input/output samples of a learning data.
                                                                    Then the objective function Gt is defined as the sum of
                                                                    squared errors over the all samples:

                                                                                           P    l
                                                                                       1               ∗
                                                                                Gt =                 (ykp (t) − ykp (t))2    (10)
                                                                                       P   p=1 k=1
  Fig. 1. An organization of the simplified fuzzy reasoning.

                                                                    where subscript p indicates the value with respect to p-
B. Different types of MSFs                                          th sample. When a value of the objective function Gt is
                                                                    smaller than a threshold value ε, according to the progress
In this section two types of MSFs are considered for conve-         of the learning, the learning process is finished.
nience of the explanation. Figure 2(a) shows an isosceles
triangular MSF and Figure 2(b) shows an bell-shaped MSF,                                        Gt ≤ ε                       (11)
which are often used in the conventional fuzzy reasoning
rules. They are formally described as follows:                         III. REPLACING MSFS USING THE GA METHOD

                                                                    In the process of learning by the steepest descent method,
Aji (xj )                                                           there are cases in which the learning does not progress to
                                                                    a desired state because of a minute reduction of the mean
                 2|xj −aji |              b             bji         square error Gt . The reason of this phenomenon is consid-
            1−                 , aji − 2 < xj < aji +
                                        ji
                                                                    erd from the following facts: (1) the fact that the MSFs
   =                bji                                  2
            0                  , otherwise                          are not located in positions appropriate for the learning
                                                                    data, (2) the fact that lengthy MSFs exist and (3) the fact
                                                              (4)   that the MSFs’s shape does not adapt the learning data. In
               Fig. 2. Different types of MSFs.


such cases we propose that the MSFs have to be placed in
more appropriate locations by moving the MSF peaks aji
to other locations and propose to change the MSFs’s kinds                   Fig. 3. Application of the GA method.
and the number of MSFs.

                                                                 tion selected at random sets the boundary of the rule field.
A. Replacing MSFs using the GA method
                                                                 Step6 Carry out the mutation based on the bit-selection
As shown in Fig.3(a), an individual is composed of plu-          probability.
ral rules. The individual has the genetic code of the MSF        Step7 Select the individual of the most highest fitness
peaks aji at each rule. Besides, we insert genetic bits to the   and return to the steepest descent method. As shown in
individual. By this genetic bits, the propriety of MSFs can      Fig.3(b), evaluate whether the selected MSF is proper or
be decided and kinds of the MSFs are selected. The genetic       improper based on the on-off gene of the MSFs, and select
code is represented by binary number with finite bit length       some kinds of MSFs based on the kind’s gene of the MSFs.
and fixed point. In this paper, we use the simple GA, which       The gene of the peaks of the MSF exchanges binary num-
is given below.                                                  ber with finite bit length for denary number. The maximum
Step1 Establish the initial individuals using random num-        of the denary number divides input region into same pieces,
bers in Fig.3(a).                                                and the peaks of the MSF move the range of the value of
Step2 Establish the reciprocal of the objective function as      the gene of the peaks of the MSF from the side edge of the
the fitness.                                                      region.
Step3 Reproduct the individuals.                                              IV. COMPUTER SIMULATION
Step4 Select the individual with the high fitness value as
the next generation.                                             In order to prove the effectiveness of our proposal, the fol-
Step5 Carry out the one-point crossover. Crossover posi-         lowing functions are numerically approximated.
A. Shaping effect of the MSFs                                   bell-shaped MSFs, and n2 describes that by isosceles trian-
                                                                gular MSFs.
In order to prove the effectiveness of MSF’s shape, the fol-
lowing function is prepared.
                                                                       1


                     x+1    −1 ≤ x < 0
             y=                                          (12)
                     −x + 1 0 ≤ x < 1




                                                                     y
                       y = −x2 + 1                       (13)

Here, Equation(12) is the function which has a differential
impossible point. Equation(13) is the function which has a             0
smooth shape. The numerical approximation is performed                     -1                    0                     1
under the following conditions:                                                                  x

(a) Input values x for each function are generated by a ran-           1
dom number within [-1,1].
(b) Output value of each function y[0,1] is evaluated.
(c) A hundred pairs and ten pairs of input and output data
are prepared.




                                                                     y
Replacing MSFs using the GA method is performed under
the following conditions:
(a) The gene of the MSF peaks is expressed by eight bits.
                                                                       0
(b) The on-off gene of the MSFs is expressed by one bit.                   -1                    0                     1
                                                                                                 x
(c) The kind’s gene of the MSFs is expressed by one bit.
(d) Twenty individuals composed of a hundred bits (=ten
rules) are set.                                                 Fig. 4. Comparison between a hundred pairs of learning data
                                                                    and the model after learning for approximation of an example
(e) The crossover probability is 1.0 and the bit-selection          function given by Eq.(12).
probability is 0.1.
Initial positions of the centers of the ten MSFs for the
                                                                B. Multi input and output learning data
premise are set to be at equal intervals. We applied the
algorithm to a hundred pairs and ten pairs of input and         In order to prove the effectiveness of multi input and output
output data, and compared the solutions obtained by both        learning data, we applied the algorithm to two input and
isosceles triangular MSFs and the bell-shaped MSFs with         two output learning data.
those by each MSFs alone. The approximation results of
an example function given by Eq.(12) from Fig.4 to Fig.6
show learning data and the model after learning when the                        x(t + 1) = 1 − 1.4x(t)2 + y(t)
                                                                                                                           (14)
mean square error becomes the minimum value respec-                             y(t + 1) = 0.3x(t)
tively. The approximation results of an example function
given by Eq.(13) from Fig.7 to Fig.9 show learning data
and the model after learning when the mean square error         
becomes the minimum value respectively. Figure 4 and             x(t + 1) = 1 + 0.9(x(t)cos(θ(t)) − y(t)sin(θ(t)))
Figure 7 are the case that number of learning data are a          y(t + 1) = 0.9(x(t)sin(θ(t)) + y(t)cos(θ(t)))
                                                                  θ(t) = 0.4 − 6.0/(1 + x(t)2 + y(t)2 )
                                                                
hundred pairs using each MSFs alone. Figure 5 and Figure
8 are the case that number of learning data are ten pairs us-
                                                                                                                           (15)
ing each MSFs alone. Figure 6 and Figure 9 are the case
using both isosceles triangular MSFs and the bell-shaped
MSFs. Table I shows the number of rules in each example
                                                                 
from Fig.4 to Fig.6 for approximation of an example func-         x(t + 1) = 0.8x(t) + 0.2f (y(t)) − f (x(t)) + 0.68
tion given by Eq.(12). Table II shows the number of rules          y(t + 1) = 0.8x(t) + 0.2f (x(t)) − f (y(t)) + 0.68
in each example from Fig.7 to Fig.9 for approximation of         
                                                                   f (z) = 1/(1 + exp(−z/0.04))
an example function given by Eq.(13). As for (n1 , n2 ) in
Table I and Table II, n1 describes number of the rules of                                                                  (16)
       1                                                                1



     y




                                                                      y
       0                                                                0
           -1                   0                    1                      -1                  0                    1
                                x                                                               x

       1                                                                1
     y




                                                                      y
       0                                                                0
           -1                   0                    1                      -1                  0                    1
                                x                                                               x


Fig. 5. Comparison between ten pairs of learning data and the    Fig. 6. Comparison between learning data and the model after
    model after learning for approximation of an example func-       learning using both types of MSFs for approximation of an
    tion given by Eq.(12).                                           example function given by Eq.(12).


                          TABLE I                                                          TABLE II
       C OMPARISON OFTHE NUMBER OF RULES FOR                              C OMPARISON OF
                                                                                      THE NUMBER OF RULES FOR
   APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY                    APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY
                      E Q .(12)                                                        E Q .(13)




Here, Equation(14) shows the function of the Henon               (d) Output value are given by x(t+1),y(t+1) for each func-
map. Equation(15) shows the function of the Ikeda map.           tion.
Equation(16) shows the function of the chaotic neural net-       Replacing MSFs using the GA method is performed un-
work. These functions often use the chaos theory. The            der the same conditions before subsection. Initial positions
numerical approximation is performed under the following         of the centers of 16 MSFs for the premise are set to be at
conditions:                                                      equal intervals covering the input region. We applied the
                                                                 algorithm to three kinds of approximated functions, and
(a) Time series are generated by each function.
                                                                 compared the solutions obtained by both isosceles trian-
(b) A hundred data of (x(t),y(t),x(t+1),y(t+1)) are picked       gular MSFs and the bell-shaped MSFs with those by each
up from time series.                                             MSFs alone. The approximation results from Table III to
(c) Input values are given by x(t),y(t) for each function.       Table V show number of fuzzy rules, the minimum value of
       1

                                                                          1

     y




                                                                        y
       0
           -1                    0                     1
                                 x                                        0
                                                                                -1                 0                    1
       1                                                                                           x

                                                                            1
     y




                                                                         y
       0
           -1                    0                     1
                                 x                                          0
                                                                                -1                 0                    1
                                                                                                   x
Fig. 7. Comparison between a hundred pairs of learning data
    and the model after learning for approximation of an example
    function given by Eq.(13).
                                                                   Fig. 8. Comparison between ten pairs of learning data and the
                                                                       model after learning for approximation of an example func-
                                                                       tion given by Eq.(13).
the mean square error at the learning data and those at the
checking data after iteration respectively. The MSFs tuned
by our algorithm are illustrated in Fig.10.

                      V. DISCUSSION
                                                                                            TABLE III
In the function approximation of Eq.(12), as shown in                 C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN
                                                                        SQUARE ERROR AND THE NUMBER OF RULES FOR
Fig.4(a), there is a wide gap between learning data and               APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY
the model after learning at x = 0 using bell-shaped MSFs                                 E Q .(14)
alone. On the other hand, as shown in Fig.4(b), the model
after learning fits learning data at the whole region using
isosceles triangular MSFs alone. Smooth shape of bell-
shaped MSFs causes the gap in Fig.4(a). When num-
ber of learning data are decreased at the whole region, as
shown in Fig.5(a), the model after learning fits learning
data smoothly using bell-shaped MSFs alone. On the other
hand, as shown in Fig.5(b), though the model after learn-
ing goes through learning data using isosceles triangular
MSFs alone, that behaves jaggy at the region except learn-
ing data because isosceles triangular MSF has a straight
line and a differential point. As shown in Fig.6, in the case
of both types of MSFs, the problem using each MSF alone
is solved. Figure 6(a) shows the model after learning fits
learning data at the whole region like the case using isosce-
les triangular MSFs alone. This method using both types
of MSFs chooses isosceles triangular MSF appropriately as
shown in Table I. Figure 6(b) shows the model after learn-
       1

     y




       0
             -1                0                    1
                               x                                         1

         1




                                                                  y(t)
      y




                                                                         0



         0
             -1                 0                    1
                                x
                                                                                        0                    1
                                                                                                 x(t)
Fig. 9. Comparison between learning data and the model after
    learning using both types of MSFs for approximation of an
    example function given by Eq.(13).




                                                                         1
                         TABLE IV
   C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN
     SQUARE ERROR AND THE NUMBER OF RULES FOR
   APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY
                      E Q .(15)
                                                                 y(t)




                                                                         0




                                                                                        0                   1
                                                                                                 x(t)


                                                                Fig. 10. MSFs tuned by iterative learning for approximation of
                                                                     an example function given by Eq.(15).
                         TABLE V                               and the rate of increase of the mean square error between
   C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN                learning and checking data are the least in those. As shown
     SQUARE ERROR AND THE NUMBER OF RULES FOR
   APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY
                                                               in Fig.10(a), though there is a region that MSFs do not
                      E Q .(16)                                exist using isosceles triangular MSFs alone, as shown in
                                                               Fig.10(b), there is no such a region in the case of both bell-
                                                               shaped MSFs and isosceles triangular MSFs. As for the
                                                               number of rules and generalization ability, this method of
                                                               using different types of MSFs is superior to that of using
                                                               each MSFs alone.

                                                                                   VI. CONCLUSION

                                                               In this paper, we have applied the method of using different
                                                               types of MSFs efficiently to multi input and output learning
                                                               data. Each MSF has a characteristic that is or is not good
                                                               at adjusting the model.This algorithm can make proper use
                                                               of each MSF. Besides, in the case of multi input and output
                                                               learning data, as for the number of rules and generalization
                                                               ability, the effectiveness of this method of using different
                                                               types of MSFs has been proved.

                                                                                      REFERENCES
ing is similar to the function form given by Eq.(12) rather    [FUK 95] Fukuda T.,Hasegawa Y. and Shimojima Y.,
than the case using bell-shaped MSFs alone as shown in         Structure Organization of Hierarchical Fuzzy Model us-
Fig.5(a), because both types of MSFs are used as shown in      ing Genetic Algorithm, Trans. Japan Soc. Fuzzy Theory,
Table I.
                                                               vol.7,No.5,pp.988-996, 1995.(in Japanese)
In the function approximation of Eq.(13), as shown in
Fig.7, the model after learning fits learning data at the       [OHK 98] Ohki M.,Moriyama T. and Ohkita M., Optimiza-
whole region using each MSF alone. There is no difference
                                                               tion of Fuzzy Reasoning by Genetic Algorithm Using Vari-
between the case using bell-shaped MSFs alone and that         able Bit-Selection Probability, IEICE, Trans. Information
using isosceles triangular MSFs alone because of learning      and Communication Engineers, vol.J81-DII, No.1, pp.127-
data densely. When number of learning data are decreased       136, 1998.(in Japanese)
at the whole region, as shown in Fig.8(a), the model af-
ter learning fits learning data smoothly using bell-shaped
                                                               [NOM 92] Nomura H.,Hayashi I. and Wakami N., Method
MSFs alone. On the other hand, as shown in Fig.8(b),           of Self-tuning of Fuzzy Inferences derived from Delta
though the model after learning goes through learning data     Rules and its Application to Obstacle Avoidance, Trans.
using isosceles triangular MSFs alone, that behaves flat at
                                                               Japan Soc. Fuzzy Theory, vol.4,No.2,pp.379-388, 1992.(in
the edge of the region because the shape of isosceles trian-   Japanese)
gular MSF does not have a skirt in comparison with that
of bell-shaped MSF. As shown in Fig.9, in the case of          [MIY 96] Miyata H.,Ohki M. and Ohkita M., Self-tuning
both types of MSFs, the problem using each MSF alone           of Fuzzy Reasoning by the Steepest Descent Method and
is solved. Figure 9(a) shows the model after learning fits      its Application to a Parallel Parking, IEICE, Trans. Infor-
learning data at the whole region like the case using bell-    mation and Systems, vol.E79-D, No.5, pp.561-569, 1996.
shaped MSFs alone. This method using both types of MSFs
chooses bell-shaped MSF appropriately as shown in Table
                                                               [GON 03] Gonda E.,Miyata H. and Ohkita M., Self-tuning
II. Figure 9(b) shows the model after learning is similar to   of Fuzzy Rules When Learning Data Have a Radically
the function form given by Eq.(13) next to the case using      Changing Distribution, Trans. IEE Japan, vol.144, No.4,
bell-shaped MSFs alone as shown in Fig.8(a), because both      pp.63-74, 2003.
types of MSFs are used as shown in Table II.
Next, about the effectiveness of multi input and output        [GON 04] Gonda E.,Miyata H. and Ohkita M., Self-
learning data, as shown from Table III to Table V, in the      Tuning of Fuzzy Rules with Some Kinds of Membership
case of isosceles triangular MSFs alone, the mean square       Functions using Genetic Algorithm, Proceedings of In-
error for the learning data is the least in these three ex-
                                                               ternational Symposium on Bio-inspired Systems, Part 5
amples, but the number of total rules is the most in those.    Robotics and Motion Control, pp.89-95,2004.
On the other hand, in the case of both bell-shaped MSFs
and isosceles triangular MSFs, the number of total rules

								
To top