VIEWS: 10 PAGES: 11 CATEGORY: Engineering POSTED ON: 11/8/2011
Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 Contents lists available at ScienceDirect Artiﬁcial Intelligence in Medicine journal homepage: www.elsevier.com/locate/aiim Suppressed fuzzy-soft learning vector quantization for MRI segmentation Wen-Liang Hung a,∗ , De-Hua Chen b , Miin-Shen Yang c a Department of Applied Mathematics, National Hsinchu University of Education, Hsin-Chu 30014, Taiwan b Institute of Statistical Science, Academia Sinica, Taipei 11529, Taiwan c Department of Applied Mathematics, Chung Yuan Christian University, Chung-Li 32023, Taiwan a r t i c l e i n f o a b s t r a c t Article history: Objective: A self-organizing map (SOM) is a competitive artiﬁcial neural network with unsupervised Received 12 November 2009 learning. To increase the SOM learning effect, a fuzzy-soft learning vector quantization (FSLVQ) algorithm Received in revised form 3 January 2011 has been proposed in the literature, using fuzzy functions to approximate lateral neural interaction of Accepted 27 January 2011 the SOM. However, the computational performance of FSLVQ is still not good enough, especially for large data sets. In this paper, we propose a suppressed FSLVQ (S-FSLVQ) using suppression with a parameter Keywords: learning schema. We then apply the S-FSLVQ to MRI segmentation and compare it with several existing Self-organizing map methods. Learning vector quantization Mean squared error Methods and materials: The proposed S-FSLVQ algorithm and some existing methods, such as FSLVQ, CPU time generalized LVQ, revised generalized LVQ and alternative LVQ, are compared using numerical data and Magnetic resonance image segmentation MRI images. The numerical data are generated by a mixture of normal distributions. The MRI data sets are from a 2-year-old female patient who was diagnosed with retinoblastoma of her left eye, a congenital malignant neoplasm of the retina with frequent metastasis beyond the lacrimal cribrosa. To evaluate the performance of these algorithms, two criteria for accuracy and computational efﬁciency are used. Results: Comparing S-FSLVQ with FSLVQ, generalized LVQ, revised generalized LVQ and alternative LVQ, the numerical results indicate that the S-FSLVQ algorithm is better than the other algorithms in accuracy and computational efﬁciency. Moreover, the proposed S-FSLVQ can reduce the computation time and increase accuracy compared to existing methods in segmenting these ophthalmological MRIs. Conclusions: The proposed S-FSLVQ is a good competitive learning algorithm that is very suitable for seg- menting the ophthalmological MRI data sets. Therefore, the S-FSLVQ algorithm is highly recommended for use in MRI segmentation as an aid for supportive diagnoses. Crown Copyright © 2011 Published by Elsevier B.V. All rights reserved. 1. Introduction response based on the networks has ability to organize itself. Com- petitive learning, on the other hand, occurs when artiﬁcial neurons Artiﬁcial neural networks are a data processing system consist- compete among themselves, and only the one that yields the best ing of a large number of simple, highly interconnected processing response to a given input modiﬁes its weight to become more like elements in an architecture inspired by the structure of the cere- the input. bral cortex in the human brain. This system performs two major Lippmann [1] presented a good tutorial review on neural com- functions: learning and recall. Learning is the process of adapting puting with six important neural net models that can be used for connection weights in an artiﬁcial neural network to produce the pattern classiﬁcation. Of these neural net models, the Kohonen’s desired output vector in response to a stimulus vector presented self-organizing map (SOM) is one of the most important ones since to the input buffer. Recall is the process of accepting an input stim- it is unsupervised with a competitive learning neural network that ulus and producing an output response in accordance with the uses the neighborhood interaction set to approximate lateral neu- network weight structure. The learning rules of neural computa- ral interaction and discover topological structures hidden in the tion indicate how connection weights are adjusted in response to data [2,3]. Learning vector quantization (LVQ) is the simplest case of a learning example. In supervised learning, an artiﬁcial neural net- SOM and its learning rule is the well-known winner-take-all (WTA) work is trained to give the desired response to a speciﬁc input principle, which gives crisp excitation states for each neuron. When stimulus. In unsupervised no speciﬁc response is sought, but the the data sets under consideration become large, the computation time of the algorithm is more important. To see the convergence rate and speed up the learning, Kohonen [2] proposed a batch ∗ Corresponding author. Tel.: +886 3 5213132; fax: +886 3 5611128. version SOM. Cheng [4] studied the convergence and ordering prop- E-mail address: wlhung@mail.nhcue.edu.tw (W.-L. Hung). erties of the batch SOM. This motivates us to propose another LVQ 0933-3657/$ – see front matter. Crown Copyright © 2011 Published by Elsevier B.V. All rights reserved. doi:10.1016/j.artmed.2011.01.004 34 W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 algorithm. Adopting the ideas of Fan et al. [5] and Hung et al. [6], tering algorithms are widely used for MRI segmentation, such as we modiﬁed Wu and Yang’s [7] fuzzy-soft LVQ (FSLVQ) algorithm brain MRI [15–19] and ophthalmological MRI segmentations [20]. to propose a suppressed FSLVQ (S-FSLVQ). To select the param- However, these fuzzy clustering algorithms always suffer from eter in S-FSLVQ algorithm, we propose a learning process which sensitivity to initials, parameters and noise. Thus, various neural is based on an exponential separation strength between weights network approaches have been proposed for the MRI segmentation with updating at each iteration. Numerical examples are used to to overcome these problems. In these neural network approaches, illustrate the effectiveness of the proposed S-FSLVQ algorithm. Kohonen’s SOM is most used in MRI segmentation. Lin et al. [21] In oncology, physicians depend on different clinical frameworks, generalized Kohonen’s SOM with fuzzy and fuzzy-soft types called different types of anatomical evidence and different theoretical fuzzy Kohonen’s competitive learning and fuzzy-soft Kohonen’s approaches to diagnose patients. Magnetic resonance image (MRI) competitive learning. They applied these generalized Kohonen’s approaches are particularly helpful in clinical oncology to support competitive learning algorithms to MRI and magnetic resonance the diagnosis of retinoblastoma, a congenital oncological disease in angiographies (MRA) ophthalmological segmentations. Recently, ophthalmology, which usually shows its symptoms in early child- Yang et al. [22] applied the FSLVQ algorithm to three MRI data sets of hood. In this paper, the proposed S-FSLVQ algorithm is applied for real cases: (i) a 2-year-old girl who was diagnosed with retinoblas- the segmentation of the MRI of a retinoblastoma patient who was toma in her left eye; (ii) a 55-year-old woman who developed diagnosed using MRI in the ophthalmology ﬁeld. complete left side oculomotor palsy immediately after a motor The rest of this paper is organized as follows. Section 2 is a vehicle accident; and (iii) an 84-year-old man who was diagnosed brief survey of related work on LVQ algorithms. In Section 3, we with Alzheimer’s disease. Yang et al. [22] also compared the per- introduce the batch FSLVQ algorithm. In Section 4, the S-FSLVQ formance of FSLVQ algorithm with the generalized Kohonen’s SOM algorithm with a parameter learning schema is proposed and algorithms proposed by Lin et al. [21]. The results indicated that the experimental results in comparison with a variety of data sets are FSLVQ algorithm is better than these generalized Kohonen’s SOM presented. In Section 5, we apply the S-FSLVQ to an MRI segmenta- algorithms. tion in the case study of a patient diagnosed with retinoblastoma of her left eye. In comparison with some existing LVQ algorithms for 3. Batch fuzzy-soft LVQ these MRI segmentation results, we ﬁnd that the proposed algo- rithm provides better detection of abnormal tissue than others, The SOM [2,3] is a useful tool for visualizing high-dimensional especially in reducing the CPU time. Finally, discussions and con- data. Basically, it produces a similarity graph of input data and clusions are presented in Sections 6 and 7, respectively. converts the nonlinear statistical relationships between high- dimensional data into simple geometric relationships of their image points on a low-dimensional display, usually a regular 2. Related work two-dimensional grid of nodes. That is, SOM is a two-layer feed- forward competitive learning neural network that can discover a Kohonen’s SOM is an unsupervised competitive neural network topological structure hidden in the data and display it in one or two- that uses the neighborhood interaction set to approximate lateral dimensional space. Assume that Wk in an p-dimensional Euclidean neural interaction and discovers the topological structure hidden space Rp with its ordinary Euclidean norm ||·|| is the speciﬁed weight in the data for visual display in one or two dimensional space [3]. of the node k and the feature vector xj ∈ Rp is shown on-line at time Although Kohonen’s competitive learning network originally was t, the winner neuron k among all neurons is produced by the nearest not a clustering method, it could be used as a prototype gener- neighbor condition ation algorithm called a LVQ [3]. Because LVQ updates only the winner node, Pal et al. [8] proposed a generalization of LVQ (GLVQ) ||xj − Wk (t − 1)|| = min||xj − Wi (t − 1)||. (1) i that updates all nodes for given input data with the learning rule depending on the degree of distance match to the winner node. Eq. (1) indicates that the weight of node k matches best with xj . Based on the winner-take-most competing strategy, Karayiannis Then the self-organization used the following learning rule: and Pai [9], Karayiannis [10] and Karayiannis et al. [11] proposed Wi (t) = Wi (t − 1) + ˛i (t)hi,j,k (xj − Wi (t − 1)), (2) some fuzzy generalized LVQ (FGLVQ) algorithms. Zhou et al. [12] discussed the disadvantages of GLVQ and FGLVQ algorithms and where ˛i (t) is the learning rate of the node i and is a monotonically then proposed a revised generalized LVQ (RGLVQ) algorithm to decreasing function of time t. The neighborhood function hi,j,k is the overcome these disadvantages. lateral neural interaction phenomenon and the degree of excitation Yair et al. [13] applied the stochastic relaxation concept to of the neuron. Usually, hi,j,k is deﬁned as modify the learning rate and neighborhood function in SOM and proposed a soft competitive learning network. Combining the 1, if the node i belongs to Nk (t), hi,j,k = (3) competitive learning with soft competition and fuzzy c-means 0, otherwise. membership functions, Wu and Yang [7] proposed a batch com- Nk (t) is called the neighborhood set of the winner neuron k and is petitive learning method called fuzzy-soft LVQ (FSLVQ). Recently, a decreasing function of time t because it needs to match the WTA Wu and Yang [14] discussed the inﬂuences of noise and outliers principle. It means that when t → ∞, on the SOM quality. To reduce the inﬂuence of noise and outliers on SOM, they proposed an alternative LVQ (ALVQ) algorithm, and 1, if i = k, numerical results have shown ALVQ performs well. hi,j,k = (4) 0, / if i = k. Image segmentation is a way to partition image pixels into dif- ferent cluster regions with similar intensity image values. Cluster It is well-known that LVQ for unlabelled data can be viewed as analysis is a method of clustering a data set into groups dis- a special case of SOM. In LVQ, the neighborhood set of each node playing similar characteristics. It is an approach to unsupervised will contain the winner node and hence the vectorial location of the learning and has become one of the major techniques used in node is negligible. This competitive learning rule is called the WTA pattern recognition. Therefore, clustering algorithms would natu- and the neighborhood function is deﬁned as Eq. (4). From Eq. (2), we rally be applied in image segmentation. Because most MRIs present know that, after an on-line feature vector xj is input, the weights overlapping gray-scale intensities for different tissues, fuzzy clus- of the nodes in both SOM and LVQ will update toward the input W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 35 vector with a step size ˛i (t)hi,j,k . Furthermore, LVQ uses the WTA S1 Initialize Wi (0) for each prototype and set the iteration counter principle to approximate the neural interaction as Eq. (4), which t = 1. Specify the maximum number of iteration max NI and ε. shows a crisp labelling for each node. Because the neighborhood S2 Compute i (xj ) using Eq. (9), ∀i, j. set of the winner neuron k is negligible, the index k in hi,j will be S3 Compute hi,j using Eq. (8), ∀i, j. deleted. Now we rewrite Eq. (4) as S4 Compute Wi (t) using Eq. (12). c p S5 If i=1 l=1 |Wil (t) − Wil (t − 1)| < ε or t = max NI, THEN stop 1, if Wi satisﬁes (1), ELSE t = t + 1 and go to Step 2. hi,j,k = (5) 0, if otherwise. 4. Suppressed FSLVQ To extend this crisp type competition, Yair et al. [13] proposed hi,j with When the size of the data set under consideration becomes large, the computation time of the FSLVQ algorithm will grow rapidly. A 2 good algorithm should perform well with a large data set, as often e−ˇ(t)||xj −Wi || hi,j = c , (6) occurs in real application systems. To speed up the FSLVQ, we adopt e−ˇ(t)||xj −Wk || 2 k=1 the suppression idea of Fan et al. [5] to the FSLVQ and then create the S-FSLVQ algorithm. We ﬁrst modify hi,j in Eq. (8) to be where ˇ(t) corresponds to a temperature which can be speciﬁed by −1 users. Based on the learning formula Eq. (2), Yair et al. [13] created a (1 − ˛ + ˛ i (xj ))(cf (t)) , if i is winner neuron, soft competition scheme (SCS). The learning rate in this sequential hi,j = (13) (˛ i (xj ))f (t)/c , otherwise. learning is where 0 ≤ ˛ ≤ 1. Note that (i) when ˛ = 0, hi,j is reduced to the WTA 1, if t is a perfect square, principle and (ii) when 0 < ˛ ≤ 1, hi,j is also reduced to the WTA ˛i (t) = 1 (7) principle as t → ∞. However, the problem of selecting a suitable , otherwise. [1/˛i (t − 1)] + hi,j parameter ˛ in S-FSLVQ constitutes an important part of imple- menting the S-FSLVQ algorithm for real applications. Based on the To solve Yair et al.’s [13] problems such as the selection of the neigh- concept of machine learning, we use the idea of Hung et al. [6] borhood functions and unstable reinitialization in the SCS, Wu and to solve this problem. That is, the learning process of ˛ is based Yang [7] apply a fuzzy function to approximate hi,j with on an exponential separation strength between prototypes with updating at each iteration. Therefore, we choose ˛ as follows: 1+f (t)/c i (xj ) mini = k ||Wi − Wk ||2 hi,j = , (8) ˛ = exp − / , (14) max1≤k≤c k (xj ) s where where n n −1 j=1 ||xj − x||2 ¯ x j=1 j c ||xj − Wi ||2/(m−1) s= , x= ¯ . = , n n i (xj ) (9) ||xj − Wk ||2/(m−1) Thus, the S-FSLVQ algorithm is summarized as follows: k=1 S-FSLVQ algorithm and the inhibition function f(t) > 0 is a monotone strict function of t that is used to control the degree of inhibition within the neural S1 Initialize Wi (0) for each prototype and set the iteration counter lateral interaction. They also suggested the learning rate with t = 1. Specifying the maximum number of iteration max NI and ε. ˛i (t) S2 Compute i (xj ) using Eq. (9), ∀i, j. ˛i (t + 1) = (10) 1 + ˛i (t)hi,j S3 Compute ˛ using Eq. (14). S4 Compute hi,j using Eq. (13), ∀i, j. which is equivalent to the optimal learning rate in SOM [3]. The S5 Compute Wi (t) using Eq. (12). constraint of f(t), that is, limt→∞ f(t) = ∞, leads to the learning rule c p S6 If i=1 W (t) − Wil (t − 1)| < ε or t = max NI, THEN stop l=1 il of this on-line FSLVQ tending toward WTA as t → ∞. ELSE t = t + 1 and go to Step 2. Assume that the sample size of feature vectors is ﬁxed, the batch version of FSLVQ is the same construction of batch SOM assum- In order to examine and compare the performance of S-FSLVQ ing that the algorithm will converge to a stationary state W*. This with FSLVQ, ALVQ, GLVQ and RGLVQ algorithms, the two criteria means that the expectation values of Wi (t) and Wi (t − 1) in FSLVQ of accuracy and computational efﬁciency are used. The accuracy of must be equal as t → ∞. That is, an algorithm is measured by the mean squared error (MSE), that is, the average sum of squared error between the true parameter and E(hi,j (xj − Wi∗ )) = 0. (11) the estimated in N repeated trials. The computational efﬁciency of an algorithm is measured by the average numbers of iterations (NI) Using the empirical function to solve the above equation, the batch and the CPU time in N repeat trials. Next, we consider the normal learning formula of FSLVQ is obtained as mixture distribution consisting of 2 subpopulations, that is, h x j i,j j pN(0, 1) + (1 − p)N( , 1) Wi∗ = , (12) h j i,j where 0 < p < 1. The data are generated by this distribution with sample size n = 100 and the number of trials N = 200. Table 1 shows where hi,j is deﬁned as Eq. (8). Wi∗ is the weighted mean of all fea- various p and . ture vectors whose weighted functions are the lateral interaction Similar to Wu and Yang [7], we consider three inhibition func- √ functions. The algorithm of batch FSLVQ is as follows: tions f(t) with t, t and t2 to control the neural lateral interaction in FSLVQ algorithm FSLVQ and S-FSLVQ. According to the three kinds of f(t), these two 36 W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 Table 1 Normal mixture distributions for the numerical tests. Test A A1 A2 A3 p 0.1 0.1 0.1 1 1.5 2.0 Test B B1 B2 B3 p 0.3 0.3 0.3 1 1.5 2.0 Test C C1 C2 C3 p 0.5 0.5 0.5 1 1.5 2.0 √ Fig. 2. Values of CPU time for S-FLVQ and FLVQ algorithms with f (t) = t. algorithms and the ALVQ, GLVQ and RGLVQ algorithms are imple- mented under the same specifying initial values Wi (0), ε = 0.001, max NI = 50 and m = 2. The MSE is deﬁned as N 2 (k) 2 k=1 i=1 (ˆi − i) 2N (k) where i is the true subpopulation mean and ˆ i is the estimated value of i for the k-th trial. Tables 2, 3 and 5 show the simula- tion results of FSLVQ, S-FSLVQ, ALVQ, GLVQ and RGLVQ algorithms, respectively. To compare the performance of S-FSLVQ, we consider the ratios of MSE, NI and CPU time. They are deﬁned as follows: MSE obtained by S-FSLVQ r(MSE) = , MSE obtained by others NI obtained by S-FSLVQ r(NI) = , Fig. 3. Values of MSE for S-FLVQ and FLVQ algorithms with f(t) = t. NI obtained by others CPU time obtained by S-FSLVQ r(time) = . CPU time obtained by others Clearly, if these ratio values are less than 1 then the perfor- mance of S-FSLVQ is better than others according to the criteria of accuracy and computational efﬁciency. Tables 4 and 6 show these ratio values. From Table 2 (i.e. S-FSLVQ algorithm), f(t) = t √ has higher accurate than the use of f (t) = t, t 2 and has better computational efﬁciency. From√ Table 3 (i.e. FSLVQ algorithm), the slow increasing rate of f (t) = t, t has also high accuracy than the use of f(t) = t2 . The fast increasing rate of f(t) = t2 has bet- ter computational efﬁciency. The MSEs √ and the CPU times for S-FLVQ and FLVQ algorithms with f (t) = t, t and t2 are depicted √ in Figs. 1–6. From these ﬁgures, we ﬁnd that (i) except f (t) = t, S-FSLVQ algorithm has better accuracy than FSLVQ algorithm for most of cases and (ii) the S-FSLVQ algorithm has better com- Fig. 4. Values of CPU time for S-FLVQ and FLVQ algorithms with f(t) = t. putational efﬁciency than FSLVQ algorithm. Furthermore, it can be found that the S-FSLVQ algorithm has better computational Fig. 5. Values of MSE for S-FLVQ and FLVQ algorithms with f(t) = t2 . √ Fig. 1. Values of MSE for S-FLVQ and FLVQ algorithms with f (t) = t. W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 37 Table 2 Values of MSE, NI and CPU time for the S-FSLVQ algorithm. √ Test f (t) = t f(t) = t f(t) = t2 MSE NI CPU time MSE NI CPU time MSE NI CPU time A1 0.2944 9.41 0.797 0.2868 5.53 0.484 0.3078 5.00 0.469 A2 0.3860 9.58 0.813 0.3771 5.54 0.468 0.3906 4.82 0.453 A3 0.5258 9.79 0.828 0.5232 5.62 0.485 0.5182 4.82 0.437 B1 0.2080 9.47 0.813 0.2029 5.64 0.484 0.2123 5.08 0.469 B2 0.1448 9.64 0.813 0.1408 5.73 0.484 0.1490 5.17 0.469 B3 0.1142 9.50 0.813 0.1105 5.72 0.484 0.1152 5.10 0.453 C1 0.1927 9.59 0.828 0.1939 5.67 0.485 0.1950 5.15 0.484 C2 0.1074 9.42 0.797 0.1082 5.40 0.468 0.1087 4.69 0.437 C3 0.0690 9.33 0.797 0.0688 5.34 0.453 0.0685 4.57 0.422 Table 3 Values of MSE, NI and CPU time for the FSLVQ algorithm. √ Test f (t) = t f(t) = t f(t) = t2 MSE NI CPU time MSE NI CPU time MSE NI CPU time A1 0.2729 28.66 1.281 0.2931 18.99 0.860 0.3058 9.21 0.484 A2 0.3700 30.16 1.328 0.3774 19.81 0.891 0.3852 9.11 0.469 A3 0.5181 32.21 1.422 0.4976 21.10 0.937 0.4957 9.44 0.484 B1 0.1836 28.91 1.297 0.2049 19.43 0.875 0.2115 9.26 0.485 B2 0.1267 31.31 1.375 0.1417 20.78 0.922 0.1465 9.48 0.484 B3 0.1011 32.38 1.422 0.1073 21.80 0.969 0.1099 9.57 0.500 C1 0.1692 30.42 1.344 0.1938 20.11 0.907 0.1986 9.36 0.500 C2 0.0907 30.70 1.375 0.1105 21.19 0.953 0.1121 9.34 0.484 C3 0.0560 31.02 1.453 0.0690 21.62 0.953 0.0699 9.18 0.469 Table 4 Ratio values of MSE, NI and CPU time for the S-FSLVQ algorithm. √ Test f (t) = t f(t) = t f(t) = t2 r(MSE) r(NI) r(time) r(MSE) r(NI) r(time) r(MSE) r(NI) r(time) A1 1.0788 0.3283 0.6222 0.9785 0.2912 0.5628 1.0065 0.5429 0.9690 A2 1.0432 0.3176 0.6122 0.9992 0.2797 0.5253 1.0140 0.5291 0.9659 A3 1.0149 0.3039 0.5823 1.0514 0.2664 0.5176 1.0454 0.5106 0.9029 B1 1.1329 0.3276 0.6268 0.9902 0.2903 0.5531 1.0038 0.5486 0.9670 B2 1.1429 0.3079 0.5913 0.9936 0.2757 0.5249 1.0171 0.5454 0.9690 B3 1.1296 0.2934 0.5717 1.0298 0.2624 0.4995 1.0482 0.5329 0.9060 C1 1.1389 0.3153 0.6161 1.0005 0.2819 0.5347 0.9819 0.5502 0.9680 C2 1.1841 0.3068 0.5796 0.9792 0.2548 0.4911 0.9697 0.5021 0.9029 C3 1.2321 0.3008 0.5485 0.9971 0.2470 0.4753 0.9800 0.4978 0.8998 Table 5 Values of MSE, NI and CPU time for the ALVQ, GLVQ and RGLVQ algorithms. Algorithm ALVQ GLVQ RGLVQ Test MSE NI CPU time MSE NI CPU time MSE NI CPU time A1 0.4569 26.97 39.141 41.5391 50.00 29.734 0.5867 50.00 34.766 A2 0.5888 26.47 38.047 183.0225 50.00 29.579 0.4553 50.00 34.703 A3 0.7765 27.58 39.844 58.3944 50.00 29.359 0.9353 50.00 34.750 B1 0.4031 24.18 35.156 218.8366 50.00 29.547 0.2415 50.00 34.703 B2 0.3988 28.86 41.406 38.2463 50.00 29.547 0.1670 50.00 35.157 B3 0.4505 30.44 44.703 5.8623 50.00 29.813 0.3793 50.00 35.094 C1 0.2620 24.05 34.797 82.7799 49.99 29.391 0.2627 50.00 34.594 C2 0.2353 27.45 39.734 93.2109 50.00 29.500 0.2360 50.00 34.829 C3 0.2431 30.57 44.344 6.1174 50.00 29.578 0.2438 50.00 34.687 Table 6 Ratio values of MSE, NI and CPU time for the S-FSLVQ algorithm. Test ALVQ GLVQ RGLVQ r(MSE) r(NI) r(time) r(MSE) r(NI) r(time) r(MSE) r(NI) r(time) A1 0.6634 0.2050 0.0124 0.0073 0.1106 0.0163 0.8580 0.1106 0.0139 A2 0.6596 0.2093 0.0123 0.0021 0.1108 0.0158 0.8531 0.1108 0.0135 A3 0.6738 0.2038 0.0122 0.0090 0.1124 0.0165 0.5594 0.1124 0.0140 B1 0.5220 0.2333 0.0138 0.0010 0.1128 0.0164 0.8712 0.1128 0.0139 B2 0.3694 0.1985 0.0117 0.0039 0.1146 0.0164 0.8820 0.1146 0.0138 B3 0.2542 0.1879 0.0108 0.0195 0.1144 0.0162 0.3019 0.1144 0.0138 C1 0.7401 0.2358 0.0139 0.0023 0.1134 0.0165 0.7381 0.1134 0.0140 C2 0.4598 0.1967 0.0118 0.0012 0.1080 0.0159 0.4585 0.1080 0.0134 C3 0.2830 0.1747 0.0102 0.0112 0.1068 0.0153 0.2822 0.1068 0.0131 38 W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 Although GLVQ (Fig. 7(5)) can distinguish the tumor, its perfor- mance is not good, and RGLVQ (Fig. 7(6)) cannot distinguish the tumor from the healthy tissue. MRI medical imaging uncertainty is widely present in the col- lected data because of noise in the partial volume effects originating from the low resolution of the sensors. Another factor causing uncertainty is the fact that the eyeball moves during the imaging and it is difﬁcult to control this movement, especially in younger patients. A distorted MR image, shown in Fig. 8, is used here to illustrate how an algorithm can detect tumorous tissue, despite uncertainty of the image. Using the AFCM with m = 2 (Fig. 8(1)), S-FSLVQ (Fig. 8(2)), FSLVQ (Fig. 8(3)) and ALVQ (Fig. 8(4)), we can still detect the tumorous tissue. But GLVQ (Fig. 8(5)) and RGLVQ (Fig. 8(6)) fail. Fig. 6. Values of CPU time for S-FLVQ and FLVQ algorithms with f(t) = t2 . Fig. 9 in the second MRI data set was processed at 283 × 292 pixels. From this picture, one lesion was clearly seen in the MR efﬁciency and is more accurate than the FSLVQ algorithm when image. However, some fuzzy shadows of lesions were suspected f(t) = t. to be tumor invasion though these suspected abnormalities cannot Next, we compare the performance of the S-FSLVQ algorithm easily be ascertained as tumorous. To detect these abnormal tis- (f(t) = t) with the ALVQ, GLVQ and RGLVQ algorithms according the sues, a window of the area around the chiasma was selected from ratios of MSE, NI and the CPU time. The results of Table 6 indicate the original MR images as shown in Fig. 9. First, we applied AFCM, that the S-FSLVQ algorithm has better computational efﬁciency and S-FSLVQ and FSLVQ to the window selection as illustrated in Fig. is more accurate than the ALVQ, GLVQ and RGLVQ algorithms. 9(1)–(3). We can see that occult lesions (red circles) were clearly As supported by the above experiments, the S-FSLVQ algorithm enhanced with AFCM (m = 2) in Fig. 9(1) and S-FSLVQ in Fig. 9(2). has better computational efﬁciency and is more accurate than the The results shown in Fig. 9(3) show FLVQ can indicate these occult FSLVQ, ALVQ, GLVQ and RGLVQ algorithms when f(t) = t. In the next lesions. Next, we use the ALVQ, GLVQ and RGLVQ algorithms to section, we will apply these algorithms to an MRI segmentation. segment this image and the segmentation results, respectively, as depicted in Fig. 9(4)–(6). With ALVQ Fig. 9(4) does not clearly indi- 5. Application to ophthalmological MRI segmentation cate the occult lesions on the right upper corner; with GLVQ Fig. 9(5) shows poor segmentation performance; while Fig. 9(6) shows that Segmentation of medical images obtained from MRI is a pri- RGLVQ can indicate these occult lesions. mary step in most applications of computer vision for medical In order to examine and compare the performance of S-FSLVQ, image analysis. Yang et al. [20] applied an alternative fuzzy c-means the following two criteria are used: the number of iterations (NI) (AFCM) algorithm, proposed by Wu and Yang [23], in a real case and the CPU time until convergence as an indication of the needed study of MRI segmentation to differentiate normal and abnormal computing time. Table 7 shows the computational performance of tissue in ophthalmology. The MRI data sets were from a 2-year- these algorithms. From Table 7, we ﬁnd that (i) in Fig. 7, the pro- old female patient. She was diagnosed with retinoblastoma of her posed S-FLVQ algorithm reduces the CPU time by 345.2, 7682.3, left eye, an inborn malignant neoplasm of the retina with frequent 10779.1, 12557%, compared with FSLVQ, ALVQ, GLVQ and RCLVQ metastasis beyond the lacrimal cribrosa. The MRI images showed an algorithms, respectively; (ii) in Fig. 8, the proposed S-FLVQ algo- intra-muscle cone tumor mass with high T1-weight signal images rithm reduces the CPU time by 334.7, 7659.1, 10901.2, 12612.9%, and low T2-weight signal images in the left eyeball. The tumor compared with FSLVQ, ALVQ, GLVQ and RCLVQ algorithms, respec- measured 20 mm in diameter and occupied nearly the entire vit- tively; and (iii) in Fig. 9, the proposed S-FLVQ algorithm reduces reous cavity. There was a shady signal abnormality all along the the CPU time by 870.4, 26638.7, 24460.2, 27975.3%, compared with optic nerve reaching as far as the optic chiasma near the brain. In FSLVQ, ALVQ, GLVQ and RCLVQ algorithms, respectively. These this current study we use S-FSLVQ, FSLVQ, ALVQ, GLVQ and RGLVQ results show the superiority of the proposed S-FLVQ algorithm over algorithms with max NI = 500 to analyze these MRI data sets. The the other algorithms. ﬁrst MRI data set is illustrated in Figs. 7 and 8 and the second MRI On the other hand, to evaluate detection of abnormal tissue, data set is shown in Fig. 9. We ﬁrst attempt to cluster the full size it is necessary to make a quantitative comparison of the image, images (Figs. 7 and 8) into the same ﬁve clusters used by Yang et al. segmented by each algorithm, with a reference image. Because [20]: muscle tissue, connective tissue, nerve tissue, the lens, and the segmentation results with AFCM (m = 2) in Figs. 7–9 can suc- tumor tissue. According to Yang et al. [20], a window as shown in cessfully differentiate the tumor from the normal tissues [20], Fig. 9 can be used to enhance areas of the tumor to better detect Figs. 7(1), 8(1) and 9(1) are considered as reference images. The small tumors. We also apply the above algorithms to segmenting comparison score S [16,24] for each algorithm is deﬁned as the image shown in Fig. 9. The lens and muscle tissue are excluded from the window so that the original ﬁve categories are reduced to three; connective tissue, nervous tissue and tumor tissue. A gray |A ∩ Aref | S= scale histogram comparison shows that there are actually three |A ∪ Aref | peaks appearing for the image. The two pictures (Figs. 7 and 8) were processed at 400 × 286 pixels. The sample size of these two pictures is 114,400 and the where A represents the set of pixels belonging to the tumor tissue data take values on the gray levels from 0 to 255. From the red found by a algorithm and Aref represents the set of pixels belonging circle on the full-sized two-dimensional MRI in Fig. 7, we can to the tumor tissue in the reference segmented image. The value clearly detect white tumor tissue at the chiasma. The segmenta- of S (0 ≤ S ≤ 1) describes the degree of similarity between A and tion results by AFCM with m = 2 (Fig. 7(1)), S-FSLVQ with f(t) = t Aref , where the greater the value of S, the higher the segmentation’s (Fig. 7(2)), FSLVQ with f(t) = t (Fig. 7(3)), ALVQ (Fig. 7(4)) can dis- quality. Moreover, adopting the similar idea of false negative and tinguish the tumor from the healthy tissue using ﬁve clusters. false positive from Fernández-García et al. [25], we may also deﬁne W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 39 Fig. 7. Original MR image. (1) Segmentation result of AFCM (S = 1, FN = 0, FP = 0). (2) Segmentation result of S-FSLVQ (S = 0.9091, FN = 0.0909, FP = 0). (3) Segmentation result of FSLVQ (S = 0.9091, FN = 0.0909, FP = 0). (4) Segmentation result of ALVQ (S = 0.9091, FN = 0.0909, FP = 0). (5) Segmentation result of GLVQ (S = 0.6667, FN = 0, FP = 0.0067). (6) Segmentation result of RGLVQ (S = 0.0133, FN = 0, FP = 1). the following two error types based on A and Aref : the proposed S-FSLVQ algorithm can detect tumorous tissue more accurately than the others. |Aref ∩ Ac | |Ac ∩ A| ref False Negative (FN) = , False Positive (FP) = |Aref | |Ac | ref 6. Discussion where Aref ∩ Ac represents that the set of pixels in Aref has not been Combining the concept of suppression [5] and machine learn- detected as tumor tissue by a algorithm, and Ac ∩ A represents ref ing [6], we propose using the S-FSLVQ algorithm to speed up the that the set of pixels in Ac has been detected as tumor tissue ref LVQ calculations. First, we reviewed the related LVQ algorithms, by a algorithm, and Ac and Ac represent the complements of A ref which include ALVQ, GLVQ and RGLVQ algorithms. Furthermore, and Aref , respectively. The values of S, FN and FP corresponding the technical details of the FSLVQ algorithm are described in Section to these algorithms are shown in Table 6. The results show that 3. These algorithms were implemented on a variety of 2-mixture 40 W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 Fig. 8. Distorted MR image. (1) Segmentation result of AFCM (S = 1, FN = 0, FP = 0). (2) Segmentation result of S-FSLVQ (S = 07374, FN = 0.2626, FP = 0). (3) Segmentation result of FSLVQ (S = 0.6403, FN = 0.3597, FP = 0). (4) Segmentation result of ALVQ (S = 0.7374, FN = 0.2626, FP = 0). (5) Segmentation result of GLVQ (S = 0.1679, FN = 0, FP = 1). (6) Segmentation result of RGLVQ (S = 0.1679, FN = 0, FP = 1) normal data sets. The proposed S-FSLVQ algorithm based on the the segmented images. Table 6 shows that the proposed algorithm criteria of accuracy and computation time outperformed others. is more accurate than other algorithms for detecting tumorous tis- To evaluate how well these algorithms worked for large data sue. sets, we applied them to a real case study of MRI segmentation to Furthermore, Alzheimer’s disease (AD) and vascular dementia differentiate normal and abnormal tissue in ophthalmology. Details are the two most common diseases causing dementia. A change of the data set are given in Section 5. The key issue in the present in hippocampal volume is found by brain MRI in individuals with analysis was whether the proposed algorithm could improve the mild cognitive impairment (MCI). Although these MCI individuals computation time. As depicted in Table 7, the proposed algorithm did not have stroke nor neurological deﬁcits, about two-thirds of provided a signiﬁcant improvement in computational efﬁciency, as them had at least one subcortical lacunar infarct. The proposed measured by the NI and CPU time. To analyze how far these seg- risk factors for converting MCI to AD include apolipoprotein E4, mentation results of the different algorithms are from the reference delayed recall, smaller hippocampal volume and decreased blood segmented image, the values of S, FN and FP were computed over ﬂow on brain HMPAO-SPECT, but there is no universal agreement. W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 41 Table 7 The execution time and a quantitative comparison of the algorithms S-FSLVQ, FSLVQ, ALVQ, GLVQ and RGLVQ for Figs. 1–3. Criterion Figures S-FSLVQ FSLVQ ALVQ GLVQ RGLVQ NI Fig. 1 28 163 188 500 500 Fig. 2 28 159 187 500 500 Fig. 3 16 246 274 500 500 CPU time(s) Fig. 1 22.062 98.219 1716.937 2400.140 2792.391 Fig. 2 22.063 95.906 1711.891 2427.203 2804.844 Fig. 3 5.969 57.922 1596.031 1466.000 1675.812 S Fig. 1 0.9091 0.9091 0.9091 0.6667 0.0133 Fig. 2 0.7374 0.6403 0.7374 0.1679 0.1679 Fig. 3 1 0.9103 0.8138 0.3034 0.7360 FN Fig. 1 0.0909 0.0909 0.0909 0 0 Fig. 2 0.2626 0.3597 0.2626 0 0 Fig. 3 0 0.0897 0.1862 0.6966 0 FP Fig. 1 0 0 0 0.0067 1 Fig. 2 0 0 0 1 1 Fig. 3 0 0 0 0 0.0122 Therefore, the accuracy of calculating the hippocampal volume in in the remedial temporal lobe. Thus, the segmentation efﬁciency the remedial temporal lobe is a main concern for predicting the and accuracy are very important for the AD MRI data sets. In our conversion of MCI to AD. Many MRI data sets which contain the hip- future work, we will apply the proposed S-FSLVQ algorithm to real pocampus are added together to calculate the hippocampal volume AD MRI data sets and make more advanced analysis. Fig. 9. Original MR image and its window selection. (1) Segmentation result of AFCM (S = 1, FN = 0, FP = 0). (2) Segmentation result of S-FSLVQ (S = 1, FN = 0, FP = 0). (3) Segmentation result of FSLVQ (S = 0.9103, FN = 0.0897, FP = 0). (4) Segmentation result of ALVQ (S = 0.8138, FN = 0.1862, FP = 0). (5) Segmentation result of GLVQ (S = 0.3034, FN = 0.6966, FP = 0). (6) Segmentation result of RGLVQ (S = 0.7360, FN = 0, FP = 0.0122). 42 W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 Fig. 9. (Continued). 7. Conclusions References In this paper we modify the FSLVQ proposed by Wu and Yang [1] Lippmann RP. An introduction to computing with neural nets. IEEE ASSP Mag- azine 1987:4–22. [7] and then propose a suppressed version of FSLVQ algorithm. In [2] Kohonen T. The self-organizing map. Neurocomputing 2003;21:1–6. the S-FSLVQ, we use a learning technique to search for the param- [3] Kohonen T. Self-organizing map. Berlin: Springer; 2001. eter ˛. From the simulation results, we ﬁnd that the advantages [4] Cheng Y. Convergence and ordering of Kohonen’s batch map. Neural Compu- tation 1997;9:1667–76. of S-FSLVQ algorithm are more accurate and reduce computation [5] Fan JL, Zhen WZ, Xie WX. Suppressed fuzzy c-means clustering algorithm. Pat- time. Finally, the S-FSLVQ algorithm is applied in the segmenta- tern Recognition Letters 2003;24:1607–12. tion of the MRI of an ophthalmic patient. The results show that the [6] Hung WL, Yang MS, Chen DH. Parameter selection for suppressed fuzzy c- means with an application to MRI segmentation. Pattern Recognition Letters S-FSLVQ provides better detection of abnormal tissue than FSLVQ, 2006;27:424–38. ALVQ, GLVQ and RGLVQ. Furthermore, the S-FSLVQ can reduce the [7] Wu KL, Yang MS. A fuzzy-soft learning vector quantization. Neurocomputing number of iterations and the CPU time. Therefore, the proposed 2003;55:681–97. [8] Pal NR, Bezdek JC, Tsao ECK. Generalized clustering networks and Kohonen’s S-FSLVQ is a good algorithm for real applications and is highly rec- self-organizing scheme. IEEE Transactions on Neural Networks 1993;4:549–57. ommended for use in MRI segmentation as an aid for supportive [9] Karayiannis NB, Pai PI. Fuzzy algorithms for learning vector quantization. IEEE diagnoses. Transactions on Neural Networks 1996;7:1196–211. W.-L. Hung et al. / Artiﬁcial Intelligence in Medicine 52 (2011) 33–43 43 [10] Karayiannis NB. A methodology for constructing fuzzy algorithms for learning images of a hemorrhagic glioblastoma multiforme. Magnetic Resonance Imag- vector quantization. IEEE Transactions on Neural Network 1997;8:505–18. ing 1995;13:277–90. [11] Karayiannis NB, Pai PI, Zervos N. Image compress based on fuzzy algorithms for [19] Suckling J, Sigmundsson T, Greenwood K, Bullmore ET. A modiﬁed fuzzy clus- learning vector quantization and wavelet image decomposition. IEEE Transac- tering algorithm for operator independent brain tissue classiﬁcation of dual tions on Image Processing 1998;7:1223–30. echo MR images. Magnetic Resonance Imaging 1999;17:1065–76. [12] Zhou SS, Wang WW, Zhou LH. A new technique for generalized learning vector [20] Yang MS, Hu YJ, Lin KCR, Lin CCL. Segmentation techniques for tissue differen- quantization algorithm. Image and Vision Computing 2006;24:649–55. tiation in MRI of ophthalmology using fuzzy clustering algorithms. Magnetic [13] Yair E, Zeger K, Gersho A. Competitive learning and soft competition for vector Resonance Imaging 2002;20:173–9. quantizer design. IEEE Transactions on Signal Processing 1992;40:294–309. [21] Lin KCR, Yang MS, Liu HC, Lirng JF. Generalized Kohonen’s competitive learning [14] Wu KL, Yang MS. Alternative learning vector quantization. Pattern Recognition algorithms for ophthalmological MR image segmentation. Magnetic Resonance 2006;39:351–62. Imaging 2003;21:863–70. [15] Ahmed MN, Yamany SM, Mohamed N, Farag AA, Moriarty T. A modiﬁed fuzzy [22] Yang MS, Lin KCR, Liu HC, Lirng JF. Magnetic resonance imaging segmentation c-means algorithm for bias ﬁeld estimation and segmentation of MRI data. IEEE techniques using batch-type learning vector quantization algorithms. Magnetic Transactions on Medical Imaging 2002;21:193–9. Resonance Imaging 2007;25:265–77. [16] Masulli F, Schenone A. A fuzzy clustering based segmentation system as [23] Wu KL, Yang MS. Alternative c-means clustering algorithms. Pattern Recogni- support to diagnosis in medical imaging. Artiﬁcial Intelligence in Medicine tion 2002;35:2267–78. 1999;16:129–47. [24] Zhang DQ, Chen SC. A novel kernelized fuzzy c-means algorithm with appli- [17] Pham DL, Prince JL. Adaptive fuzzy segmentation of magnetic res- cation in medical image segmentation. Artiﬁcial Intelligence in Medicine onance images. IEEE Transactions on Medical Imaging 1999;18: 2004;32:37–50. 737–52. [25] Fernández-García NL, Medina-Carnicer R, Carmona-Poyato A, Madrid-Cuevas [18] Philips WE, Velthuizen RP, Phuphanich S, Hall LO, Clarke LP, Silbiger ML. Appli- FJ, Prieto-Villegas M. Characterization of empirical discrepancy evaluation cation of fuzzy c-means segmentation technique for differentiation in MR measures. Pattern Recognition Letters 2004;25:35–47.