VIEWS: 4 PAGES: 8 POSTED ON: 4/1/2011 Public Domain
1 Self-Tuning of Fuzzy Rules with Different Types of Membership Functions for Multi Input and Output Learning Data ∗ Eikou Gonda , Hitoshi Miyata ,†Masaaki Ohkita ∗ Department of Electrical Engineering Yonago National College of Technology 4448 Hikona, Yonago, Tottori 683-8502, Japan ∗Phone: +81-859-24-5117, Fax: +81-859-24-5009, E-mail: gonda@yonago-k.ac.jp Phone: +81-859-24-5121, Fax: +81-859-24-5009, E-mail: miyata@yonago-k.ac.jp †Department of Electrical and Electronic Engineering Tottori University, 4-101 Koyama-Minami, Tottori 680-8552, Japan Phone: +81-857-31-5699, Fax: +81-857-31-0880, E-mail: mohkita@ele.tottori-u.ac.jp Abstract - With the progress of practical applications of the and large scale change of fuzzy system for increase of the fuzzy theory, it is becoming increasingly difﬁcult to design and number of inputs and outputs. tune larger and more complicated systems with the help of Therefore, we use a technique of genetic algorithm for fuzzy knowledge alone. Various methods of tuning the fuzzy the optimization of fuzzy reasoning using the steepest reasoning rules used for the approximation of given input- descent method[GON 04]. This new method using the output data intended for the solution of such problems have been proposed. This proposal includes methods combining genetic algorithm can select some kinds of membership fuzzy reasoning and neural networks or methods of tuning the functions(MSFs), delete some lengthy rules, and optimize fuzzy reasoning rules by using optimization techniques based MSFs. Besides, this new method can improve generaliza- on genetic algorithms. However, these methods cannot cope tion ability as a result of selection of MSFs adapting to the with many problems completely, for example, the problems of model. The advantages of this proposal are proved by nu- rapid increase of the number of rules and large scale change merical examples with multi input and output learning data of fuzzy system for increase of the number of inputs and out- involving function approximations. puts. To overcome these problems, we use a technique of genetic algorithm for the optimization of fuzzy reasoning using the steepest descent method. This new method using the ge- II. FUZZY REASONING USING DIFFERENT TYPES OF MSFS netic algorithm can select some kinds of membership func- tions(MSFs), delete some lengthy rules, and hence optimize MSFs. In addition, this new method can improve generaliza- A. Simpliﬁed fuzzy reasoning tion ability as a result of selection of MSFs adapting to the model. The advantages of this new method are proved by nu- merical examples with multi input and output learning data A method of the fuzzy reasoning described in this paper is a involving function approximations. simpliﬁed fuzzy reasoning whose consequence is given by crisp values. An organization of the simpliﬁed fuzzy rea- Keywords— fuzzy control, steepest descent method, optimiza- soning is shown in a form of the neural network architec- tion of fuzzy reasoning, selecting of shapes of membership functions, genetic algorithm, multi inputs and outputs ture. Now, let x1 , · · · , xm be input signals to the controller, and yk be the output, then the i-th rule of the simpliﬁed I. INTRODUCTION fuzzy reasoning is as follows: In the tuning of fuzzy rules, a method using a genetic algo- Rule i : if x1 is A1i and x2 is A2i and · · · rithm[FUK 95],[OHK 98], a method combining fuzzy rea- and xj is Aji and · · · and xm is Ami soning with a neural network[NOM 92],[MIY 96],[GON 03], and so on have been proposed. In a use of the former then yk is wki (i = 1, · · · , n) (1) by a genetic algorithm, the solutions are got necessarily and theoretically. But the method searches the solutions in where Aji (j = 1, 2, · · · , m) shows the j-th fuzzy vari- global solutions’s space, so that it necessitates much time to able of the premises in the i-th rule and wki shows the i- get the best solutions. On the other hand the latter method th consequence to the k(k = 1, 2, · · · , l)-th output with a has the advantage of reaching the solutions using a steepest real value. With an aid of the product-sum-gravity method, descent method in a short time. But the solutions some- for given input signals x1 , · · · , xm its compatibility µi for times result in suboptimal solutions. Besides, these meth- those rules can be determined in the hidden layer. The out- ods cannot cope with many problems completely, for exam- put yk can be obtained by estimating the center of gravity ple, the problems of rapid increase of the number of rules in the output layer. Aji (xj ) = exp{−(xj − aji )2 /bji } (5) n µi · wki The learning procedures for the fuzzy control are executed i=1 by the steepest descent method. For its implementation, the yk = n (2) following performance criterion E should be minimized, µi which is expressed by square of error between given data i=1 ∗ yk and the reasoning output yk : where ∗ (yk − yk )2 E= (6) 2 µi = A1i (x1 )A2i (x2 ) · · · Ami (xm ) (3) The procedures determining these fuzzy rules are ex- The procedures for adjusting the parameters of both isosce- pressed by using a neural network. Fuzzy rules are tuned les triangular MSFs and bell-shaped MSFs can be ex- with supervised data. pressed as follows: ∂E aji (t + 1) = aji (t) − Ka (7) ∂aji ∂E bji (t + 1) = bji (t) − Kb (8) ∂bji ∂E wki (t + 1) = wki (t) − Kw (9) ∂wki where Ka , Kb and Kw are learning coefﬁcients for adjust- ing the training cycles. The variable t denotes the t-th iter- ative computation. Through these adjustment, the desired inference rules can be obtained step by step. Suppose that we have P pairs of input/output samples of a learning data. Then the objective function Gt is deﬁned as the sum of squared errors over the all samples: P l 1 ∗ Gt = (ykp (t) − ykp (t))2 (10) P p=1 k=1 Fig. 1. An organization of the simpliﬁed fuzzy reasoning. where subscript p indicates the value with respect to p- B. Different types of MSFs th sample. When a value of the objective function Gt is smaller than a threshold value ε, according to the progress In this section two types of MSFs are considered for conve- of the learning, the learning process is ﬁnished. nience of the explanation. Figure 2(a) shows an isosceles triangular MSF and Figure 2(b) shows an bell-shaped MSF, Gt ≤ ε (11) which are often used in the conventional fuzzy reasoning rules. They are formally described as follows: III. REPLACING MSFS USING THE GA METHOD In the process of learning by the steepest descent method, Aji (xj ) there are cases in which the learning does not progress to a desired state because of a minute reduction of the mean 2|xj −aji | b bji square error Gt . The reason of this phenomenon is consid- 1− , aji − 2 < xj < aji + ji erd from the following facts: (1) the fact that the MSFs = bji 2 0 , otherwise are not located in positions appropriate for the learning data, (2) the fact that lengthy MSFs exist and (3) the fact (4) that the MSFs’s shape does not adapt the learning data. In Fig. 2. Different types of MSFs. such cases we propose that the MSFs have to be placed in more appropriate locations by moving the MSF peaks aji to other locations and propose to change the MSFs’s kinds Fig. 3. Application of the GA method. and the number of MSFs. tion selected at random sets the boundary of the rule ﬁeld. A. Replacing MSFs using the GA method Step6 Carry out the mutation based on the bit-selection As shown in Fig.3(a), an individual is composed of plu- probability. ral rules. The individual has the genetic code of the MSF Step7 Select the individual of the most highest ﬁtness peaks aji at each rule. Besides, we insert genetic bits to the and return to the steepest descent method. As shown in individual. By this genetic bits, the propriety of MSFs can Fig.3(b), evaluate whether the selected MSF is proper or be decided and kinds of the MSFs are selected. The genetic improper based on the on-off gene of the MSFs, and select code is represented by binary number with ﬁnite bit length some kinds of MSFs based on the kind’s gene of the MSFs. and ﬁxed point. In this paper, we use the simple GA, which The gene of the peaks of the MSF exchanges binary num- is given below. ber with ﬁnite bit length for denary number. The maximum Step1 Establish the initial individuals using random num- of the denary number divides input region into same pieces, bers in Fig.3(a). and the peaks of the MSF move the range of the value of Step2 Establish the reciprocal of the objective function as the gene of the peaks of the MSF from the side edge of the the ﬁtness. region. Step3 Reproduct the individuals. IV. COMPUTER SIMULATION Step4 Select the individual with the high ﬁtness value as the next generation. In order to prove the effectiveness of our proposal, the fol- Step5 Carry out the one-point crossover. Crossover posi- lowing functions are numerically approximated. A. Shaping effect of the MSFs bell-shaped MSFs, and n2 describes that by isosceles trian- gular MSFs. In order to prove the effectiveness of MSF’s shape, the fol- lowing function is prepared. 1 x+1 −1 ≤ x < 0 y= (12) −x + 1 0 ≤ x < 1 y y = −x2 + 1 (13) Here, Equation(12) is the function which has a differential impossible point. Equation(13) is the function which has a 0 smooth shape. The numerical approximation is performed -1 0 1 under the following conditions: x (a) Input values x for each function are generated by a ran- 1 dom number within [-1,1]. (b) Output value of each function y[0,1] is evaluated. (c) A hundred pairs and ten pairs of input and output data are prepared. y Replacing MSFs using the GA method is performed under the following conditions: (a) The gene of the MSF peaks is expressed by eight bits. 0 (b) The on-off gene of the MSFs is expressed by one bit. -1 0 1 x (c) The kind’s gene of the MSFs is expressed by one bit. (d) Twenty individuals composed of a hundred bits (=ten rules) are set. Fig. 4. Comparison between a hundred pairs of learning data and the model after learning for approximation of an example (e) The crossover probability is 1.0 and the bit-selection function given by Eq.(12). probability is 0.1. Initial positions of the centers of the ten MSFs for the B. Multi input and output learning data premise are set to be at equal intervals. We applied the algorithm to a hundred pairs and ten pairs of input and In order to prove the effectiveness of multi input and output output data, and compared the solutions obtained by both learning data, we applied the algorithm to two input and isosceles triangular MSFs and the bell-shaped MSFs with two output learning data. those by each MSFs alone. The approximation results of an example function given by Eq.(12) from Fig.4 to Fig.6 show learning data and the model after learning when the x(t + 1) = 1 − 1.4x(t)2 + y(t) (14) mean square error becomes the minimum value respec- y(t + 1) = 0.3x(t) tively. The approximation results of an example function given by Eq.(13) from Fig.7 to Fig.9 show learning data and the model after learning when the mean square error becomes the minimum value respectively. Figure 4 and x(t + 1) = 1 + 0.9(x(t)cos(θ(t)) − y(t)sin(θ(t))) Figure 7 are the case that number of learning data are a y(t + 1) = 0.9(x(t)sin(θ(t)) + y(t)cos(θ(t))) θ(t) = 0.4 − 6.0/(1 + x(t)2 + y(t)2 ) hundred pairs using each MSFs alone. Figure 5 and Figure 8 are the case that number of learning data are ten pairs us- (15) ing each MSFs alone. Figure 6 and Figure 9 are the case using both isosceles triangular MSFs and the bell-shaped MSFs. Table I shows the number of rules in each example from Fig.4 to Fig.6 for approximation of an example func- x(t + 1) = 0.8x(t) + 0.2f (y(t)) − f (x(t)) + 0.68 tion given by Eq.(12). Table II shows the number of rules y(t + 1) = 0.8x(t) + 0.2f (x(t)) − f (y(t)) + 0.68 in each example from Fig.7 to Fig.9 for approximation of f (z) = 1/(1 + exp(−z/0.04)) an example function given by Eq.(13). As for (n1 , n2 ) in Table I and Table II, n1 describes number of the rules of (16) 1 1 y y 0 0 -1 0 1 -1 0 1 x x 1 1 y y 0 0 -1 0 1 -1 0 1 x x Fig. 5. Comparison between ten pairs of learning data and the Fig. 6. Comparison between learning data and the model after model after learning for approximation of an example func- learning using both types of MSFs for approximation of an tion given by Eq.(12). example function given by Eq.(12). TABLE I TABLE II C OMPARISON OFTHE NUMBER OF RULES FOR C OMPARISON OF THE NUMBER OF RULES FOR APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY E Q .(12) E Q .(13) Here, Equation(14) shows the function of the Henon (d) Output value are given by x(t+1),y(t+1) for each func- map. Equation(15) shows the function of the Ikeda map. tion. Equation(16) shows the function of the chaotic neural net- Replacing MSFs using the GA method is performed un- work. These functions often use the chaos theory. The der the same conditions before subsection. Initial positions numerical approximation is performed under the following of the centers of 16 MSFs for the premise are set to be at conditions: equal intervals covering the input region. We applied the algorithm to three kinds of approximated functions, and (a) Time series are generated by each function. compared the solutions obtained by both isosceles trian- (b) A hundred data of (x(t),y(t),x(t+1),y(t+1)) are picked gular MSFs and the bell-shaped MSFs with those by each up from time series. MSFs alone. The approximation results from Table III to (c) Input values are given by x(t),y(t) for each function. Table V show number of fuzzy rules, the minimum value of 1 1 y y 0 -1 0 1 x 0 -1 0 1 1 x 1 y y 0 -1 0 1 x 0 -1 0 1 x Fig. 7. Comparison between a hundred pairs of learning data and the model after learning for approximation of an example function given by Eq.(13). Fig. 8. Comparison between ten pairs of learning data and the model after learning for approximation of an example func- tion given by Eq.(13). the mean square error at the learning data and those at the checking data after iteration respectively. The MSFs tuned by our algorithm are illustrated in Fig.10. V. DISCUSSION TABLE III In the function approximation of Eq.(12), as shown in C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN SQUARE ERROR AND THE NUMBER OF RULES FOR Fig.4(a), there is a wide gap between learning data and APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY the model after learning at x = 0 using bell-shaped MSFs E Q .(14) alone. On the other hand, as shown in Fig.4(b), the model after learning ﬁts learning data at the whole region using isosceles triangular MSFs alone. Smooth shape of bell- shaped MSFs causes the gap in Fig.4(a). When num- ber of learning data are decreased at the whole region, as shown in Fig.5(a), the model after learning ﬁts learning data smoothly using bell-shaped MSFs alone. On the other hand, as shown in Fig.5(b), though the model after learn- ing goes through learning data using isosceles triangular MSFs alone, that behaves jaggy at the region except learn- ing data because isosceles triangular MSF has a straight line and a differential point. As shown in Fig.6, in the case of both types of MSFs, the problem using each MSF alone is solved. Figure 6(a) shows the model after learning ﬁts learning data at the whole region like the case using isosce- les triangular MSFs alone. This method using both types of MSFs chooses isosceles triangular MSF appropriately as shown in Table I. Figure 6(b) shows the model after learn- 1 y 0 -1 0 1 x 1 1 y(t) y 0 0 -1 0 1 x 0 1 x(t) Fig. 9. Comparison between learning data and the model after learning using both types of MSFs for approximation of an example function given by Eq.(13). 1 TABLE IV C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN SQUARE ERROR AND THE NUMBER OF RULES FOR APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY E Q .(15) y(t) 0 0 1 x(t) Fig. 10. MSFs tuned by iterative learning for approximation of an example function given by Eq.(15). TABLE V and the rate of increase of the mean square error between C OMPARISON OFTHE MINIMUM VALUES OF THE MEAN learning and checking data are the least in those. As shown SQUARE ERROR AND THE NUMBER OF RULES FOR APPROXIMATION OF AN EXAMPLE FUNCTION GIVEN BY in Fig.10(a), though there is a region that MSFs do not E Q .(16) exist using isosceles triangular MSFs alone, as shown in Fig.10(b), there is no such a region in the case of both bell- shaped MSFs and isosceles triangular MSFs. As for the number of rules and generalization ability, this method of using different types of MSFs is superior to that of using each MSFs alone. VI. CONCLUSION In this paper, we have applied the method of using different types of MSFs efﬁciently to multi input and output learning data. Each MSF has a characteristic that is or is not good at adjusting the model.This algorithm can make proper use of each MSF. Besides, in the case of multi input and output learning data, as for the number of rules and generalization ability, the effectiveness of this method of using different types of MSFs has been proved. REFERENCES ing is similar to the function form given by Eq.(12) rather [FUK 95] Fukuda T.,Hasegawa Y. and Shimojima Y., than the case using bell-shaped MSFs alone as shown in Structure Organization of Hierarchical Fuzzy Model us- Fig.5(a), because both types of MSFs are used as shown in ing Genetic Algorithm, Trans. Japan Soc. Fuzzy Theory, Table I. vol.7,No.5,pp.988-996, 1995.(in Japanese) In the function approximation of Eq.(13), as shown in Fig.7, the model after learning ﬁts learning data at the [OHK 98] Ohki M.,Moriyama T. and Ohkita M., Optimiza- whole region using each MSF alone. There is no difference tion of Fuzzy Reasoning by Genetic Algorithm Using Vari- between the case using bell-shaped MSFs alone and that able Bit-Selection Probability, IEICE, Trans. Information using isosceles triangular MSFs alone because of learning and Communication Engineers, vol.J81-DII, No.1, pp.127- data densely. When number of learning data are decreased 136, 1998.(in Japanese) at the whole region, as shown in Fig.8(a), the model af- ter learning ﬁts learning data smoothly using bell-shaped [NOM 92] Nomura H.,Hayashi I. and Wakami N., Method MSFs alone. On the other hand, as shown in Fig.8(b), of Self-tuning of Fuzzy Inferences derived from Delta though the model after learning goes through learning data Rules and its Application to Obstacle Avoidance, Trans. using isosceles triangular MSFs alone, that behaves ﬂat at Japan Soc. Fuzzy Theory, vol.4,No.2,pp.379-388, 1992.(in the edge of the region because the shape of isosceles trian- Japanese) gular MSF does not have a skirt in comparison with that of bell-shaped MSF. As shown in Fig.9, in the case of [MIY 96] Miyata H.,Ohki M. and Ohkita M., Self-tuning both types of MSFs, the problem using each MSF alone of Fuzzy Reasoning by the Steepest Descent Method and is solved. Figure 9(a) shows the model after learning ﬁts its Application to a Parallel Parking, IEICE, Trans. Infor- learning data at the whole region like the case using bell- mation and Systems, vol.E79-D, No.5, pp.561-569, 1996. shaped MSFs alone. This method using both types of MSFs chooses bell-shaped MSF appropriately as shown in Table [GON 03] Gonda E.,Miyata H. and Ohkita M., Self-tuning II. Figure 9(b) shows the model after learning is similar to of Fuzzy Rules When Learning Data Have a Radically the function form given by Eq.(13) next to the case using Changing Distribution, Trans. IEE Japan, vol.144, No.4, bell-shaped MSFs alone as shown in Fig.8(a), because both pp.63-74, 2003. types of MSFs are used as shown in Table II. Next, about the effectiveness of multi input and output [GON 04] Gonda E.,Miyata H. and Ohkita M., Self- learning data, as shown from Table III to Table V, in the Tuning of Fuzzy Rules with Some Kinds of Membership case of isosceles triangular MSFs alone, the mean square Functions using Genetic Algorithm, Proceedings of In- error for the learning data is the least in these three ex- ternational Symposium on Bio-inspired Systems, Part 5 amples, but the number of total rules is the most in those. Robotics and Motion Control, pp.89-95,2004. On the other hand, in the case of both bell-shaped MSFs and isosceles triangular MSFs, the number of total rules