193 Vol. 15 No. 5 October 2003 APPLICATIONS, BASIS & COMMUNICATIONS 194
USING TEXTURE AND SHAPE FEATURES TO shape  or color and space , etc, by some
comparison methods to abstract the features of various
RETRIEVE SETS OF SIMILAR MEDICAL IMAGES images as the basis for image retrieval, so that the
image of the maximum similarity can be found out.
Besides, whether the size of image data will cause
JIANN-DER LEE, LI-PENG LOU network jam during data transfer and the space
required for saving the image data are also problems;
therefore, the image data should be compressed. The
Department of Electrical Engineering so-called image compression is reducing the
Chang Gung University, Tao-Yuan, Taiwan redundancy of image data, including psychological,
inter-image pixel and code redundancy, and leaving
over the data that we are interested in. JPEG is the
image compression standard that are adopted most
widely at present, and the newly constituted JPEG2000
ABSTRACT will be the standard format for image compression in
the coming 21st century. Recently, Cheng  proposed
a retrieval method aiming at the uncompressed and Fig. 1 The flowchart of the proposed scheme for
In this paper, a novel scheme has been proposed for image retrieval task using the feature compressed JPEG or the Wavelet Transformed image. image retrieval.
extracted directly from a compressed or uncompressed image. The texture information is first Extension of the previous researches, in this paper, we
extracted by exploiting the multiresolution nature of wavelet decomposition, which represent the not only proposed a novel image retrieval scheme
horizontal, vertical and diagonal frequency distribution of an image. We then calculate the mean and suitable for various image format including BMP, 2. IMAGE RETRIEVAL STRATEGY
standard deviation of wavelet coefficients of each sub-band as texture features. In additions, we also JPEG and JPEG2000, but also made retrieval aiming at
extract shape feature by using the fixed-resolution block representation, which divides the image into the whole similar compressed medical image set to In this paper, the major retrieval data are the
isometric blocks and calculate the overlapped degree of each block with binary codes. The find out the features of image that can represent the similar medical image sets; most of these images are
experimental results show that the retrieval efficiency is considerably improved by the proposed whole image set; herein, the users only have to find out the simple combination of object and background.
approach. which set that the image belongs to before he compare Therefore, on consideration of the above objectives
which image it is. The objective that we hoped to and the practical application, we designed an image
Biomed Eng Appl Basis Comm, 2003 (October); 15: 192-198. achieve is the image retrieval system that can reduce retrieval strategy, which made retrieval of the medical
Keywords: Image retrieval, shape feature, texture feature, similar medical image the quantity of image data and computation, and image separately aiming at the whole set of different
enhance the correctness and effectiveness of image formats including uncompressed and compressed
access and retrieval. JPEG or JPEG2000 using texture and shape features
As shown in Fig. 1, this paper presents a content- extracted from medical images. The flowchart of the
1. INTRODUCTION diagnosing the patients and inquiring about the relative based image retrieval strategy, in expectation of retrieval strategy is shown in Fig.1.
illness. An image needs many memory bits for store, finding out the features of shape and texture from all
for example, a digital image with 256 x 256 pixels and the compressed data; therefore, a composite image 2.1 Classification Method of Similar Images
The rapid development of computer network has retrieval scheme is built as the model for retrieval of Taking the medical image as an example, when
256 gray levels needs the storage of 65536 bytes.
produced many convenient applications. For example, the similar compressed medical image set, which taking CT and MRI scanning, it usually takes tens of
Confronted with the increasingly numerous and
in modern hospital, the patient record is made extracts the features image from the medical images of pieces once. If the inter-distance between the adjacent
various medical images, the users want to find simply
electronically and applied with various recourses in different format in the whole set to represent the images is too large, the difference between them will
and rapidly the similar medical image of a certain
teleconsultation. Many medical images, i.e. original image and then finds out the features of shape be too large and they cannot be considered as similar
interested illness for diagnosis and analysis; it is a big
ultrasonography, CT (computerized tomography) and and texture from the feature image for comparison. images; therefore, we defined a simple criterion to
problem at present.
MRI (magnetic resonance image), are used to diagnose From the experimental results, it is proved that the classify the images. That is, based on the statistic
In general, there are two kinds of features
the patient's status in the hospitals. Thus there are large proposed method can effectively complete the task of relation of images, we can calculate their correlation
common used to represent the image contents. One is
numbers of films for storing medical image in the image retrieval. coefficient and assign each image to its corresponding
obtained in the spatial domain, the objective
hospitals. Now by PACS (Picture Archiving and The remainder of this paper is organized as below. category.
characteristics of image pixel distribution, i.e. object
Communication System), the whole image diagnosis Section 2 illustrates the proposed image retrieval Supposing there are two pieces of image data,
shape , texture characteristic , color distribution
system in the hospital can be integrated, the medical strategy, which describes how to classify the images which are A (a 1 ,a 2 ,...a s ) and B (b 1 ,b 2 ,...b s )
[3-4], and other image contents, are used to represent
images can be digitized, and the medical image into the corresponding sets according to their separately, therefore the linear correlation coefficient
the characteristic of image. The other is obtained in the
database can be built, so as to make it convenient for correlation coefficients. The method to extract the of the two images is written as Eq.(1).
frequency domain, because some images have more
distinct features in frequency domain than that in texture and shape feature of a feature image is also
spatial domain; therefore, the results of some included in this section. The experimental results are
Received: Sept 17, 2003; Accepted: Oct 1, 2003 shown in Section 3. Finally, the conclusion is given in
transforms, i.e. Fast Fourier Transform , Discrete
Correspondence: Jiann-Der Lee, Ph. D., Professor Cosine Transform  or Discrete Wavelet Transform Section 4.
Department of Electrical Engineering, Chang Gung (1)
[7-8], can be recorded as the features of image. Most
University, Tao-Yuan, Taiwan relative researches combine some different image
E-mail: firstname.lastname@example.org features, i.e. relation of color and texture , color and
195 Vol. 15 No. 5 October 2003 APPLICATIONS, BASIS & COMMUNICATIONS 196
Herein, am and bm represent the average value of A 2.3.1 Original Image (BMP) from a compressed image (JPEG2000) is described 2.4.2 Shape-based Feature Extraction
and B data sets, respectively; S is the total pixel The image with BMP format is first divided into herer. It is known that the message will be multiplied
number of an image. In order to make it convenient for blocks of 8 x 8 pixels, and the average value of the by 2 when it is made number i step two-dimensional Here, we uses Fixed-Resolution format for shape
observation and avoid producing negatives, we blocks is calculated as a pixel of the feature image. Wavelet Transform. Therefore, in order to find out the representation and derive the required shape features.
replaced A,B with 2A,B. Therefore, if the two images That is, we reduce a 512*512 to a 64*64 feature image. representative image from JPEG2000 similar with that The details of the extraction step are described as
are very similar, 2A,B will tend towards 1; on the The detailed formula is expressed as Eq. (2). from the original image, we divide 2 from LL i of below.
contrary, if the two images have no relativity, 2A,B number (m, n) block plus displacement (DC), so that Step 1: Divide the image into isometric blocks
nearly equals to 0. we can get the approximate average value of the image that contain N x N pixels.
Supposing there is a set of successive medical (2) using Eq. (4). Step 2: Judge whether there are over p % (p is
images I1, I2, I3...IN, the correlation coefficients between between 1~100) pixels greater than a certain critical
I 1 and I 2, I 1 andI 3...I 1 and I j+1 are calculated; if the where FIm,n represents the pixel of coordinate (4) value in each block; if it is true, the index of this block
correlation coefficient is smaller than a certain (m,n) in the feature image, OI represents the original will be set to 1, if it is not, it will be set to 0.
threshold, we will classify the images from I1 to Ij as image. 2.4 Extracting Image Features from the Step 3: Judge whether the shapes of the two
the same category. Then the correlation coefficients Feature Image objects are similar; comparing the block index
between Ij+1 and Ij+2, Ij+1 and Ij+3...Ij+1 and Ik are calculated; produced in step 2, if they are different, then add 1 to
if the correlation coefficient between Ij+1 and I k+1 is 2.3.2 Compressed Image (JPEG ) the result; the smaller result represents that the shapes
smaller than a certain threshold, the images from Ij+1 to The image with JPEG format is also divided into In order to find out the feature that is sufficient to
represent the whole set of image, first we analyze the are more similar. In short, the decision rule is shown as
Ik are classified as the same category; accordingly, the blocks in advance and Discrete Cosine Transform below:
images can be classified correctly. (DCT) are performed to these blocks. The DC content characteristic of image and extract the
coefficient and AC coefficient of these blocks after characteristics of its shape and texture, etc, and then
2.2 Representing the Whole Image Set with DCT are then coded separately; therefore, DC combine these two features for later retrieval task.
Centroid Image coefficient can be adopted as the representative image 2.4.1 Texture-based Feature Extraction
The so-called Centroid Method  extracts the during image retrieval. In order to make the feature
pixels in the same position of each image of an image The image is calculated by three-level Discrete
image found out from JPEG similar with that found out Wavelet Transform and then sub-images are obtained.
set and sorts them in order; thus the centroid image of from original BMP image, we multiply the DC
the similar image set can be found out. The Centroid Nine sets of Wavelet coefficients, including LH1, HL1,
coefficient of number (m, n) block of the image by HH1, LH2, HL2, HH2, LH3, HL3, HH3, are taken out.
Method but not Average method is adopted because it (1/8) plus 128, i.e., [(1/8) FIm,n(0,0)+128], 128 is
can avoid the effect caused by huge variation of the Their mean and standard deviation  are calculated
added back because it is subtracted from the image in as image features, as Eq. (5) and (6).
gray level among the images in an image set; In other JPEG compression as displacement. In short, the JPEG
words, it can remove the extreme gray value due to the format image is divided into 8*8 blocks and the
noise. Fig. 2 shows the flowchart of Centroid Method. calculation step is summarized as below. (5)
2.3 Representing Images of Different Formats Supposing Fm,n(x, y) is the value of number (m, n)
with Feature Image block of the image after Fast Fourier Transform, where Si represents the i step wavelet coefficient,
therefore DC coefficient is F m,n(0,0), among which i =1,2,3.
Because the computation of feature information N=8.
for the whole image set is too huge, in order to reduce Standard deviation For example: An image with 256 x 256 pixels, if
the computation of these image features, we attempt to N equals to 32, the image is divided into 8 x 8 blocks,
replace the whole set of images by a feature image (6) each block contains 32 x 32 pixels; if N equals to 16,
without affecting the retrieval accuracy. At the same the image is divided into 16 x 16 blocks, each block
time the feature image can be sorted out from images contains 16 x 16 pixels. Therefore, if there are more
of different formats, including compressed and The feature vector is then obtained from the mean blocks, in spite of more data quantity, more specific
uncompressed images, then the image features of the and standard deviation as follows object shape can be recorded and the resolution is
feature image is recorded as the retrieval index. higher. Fig. 3 is an example to illustrate the concept of
(3) shape representation using Fixed-Resolution format.
Various image formats used in this approach are (7)
described as below:
After the feature vector of image is obtained, the 3. EXPERIMENTAL RESULTS AND
distance between image I1 and I2 is calculated. PERFORMANCE EVALUATION
(8) In the experiment, a medical image database,
which consists of MRI images and CT images of
2.3.3 Compressed Image (JPEG2000) and represent the kth features of image I1 and human body partially obtained from Chang Gung
I2. Memorial Hospital, are employed. Six examples of the
test images are shown in Fig. 4. More specifically,
In order to achieve the goal that the feature image MRI images include the six principal parts of human
obtained from images with various formats should be body; each part has several similar image sets, totaled
Fig. 2 The flowchart of Centroid Method. as same as possible, the derivation of the feature image to 580 pieces; and the resolution of each image is 256
197 Vol. 15 No. 5 October 2003 APPLICATIONS, BASIS & COMMUNICATIONS 198
shown in Table 1-5. Comparing Table 1 and Table 2, Table 3. Performance evaluation using difference-
we can find the retrieval accuracy is better while the image features extracted from the median image of
texture feature extracted from the feature image. a set of medical images and extracted from the
Besides, we added the noises ranging from 8, 16 feature image of median image.
and 32, which are produced at random, to the
retrieved images; the retrieval accuracy is shown as
Tables 3~5; It is obvious that the similar images can be
effectively retrieved by the features of shape and
In summary, the experimental results show that
retrieval accuracy using both shape and texture is
better than only using one. Clearly, the performance of
the proposed method is fast and efficient for medical
image retrieval task
Table 4. Performance evaluation using difference-
4. CONCLUSIONS image features extracted from the whole image.
Different from the traditional image retrieval
system that retrieves the single piece of static image
one by one, the proposed retrieval strategy finds out
Fig. 3 An example to illustrate the concept of shape the centroid image of the whole set of similar medical
representation using Fixed-Resolution format. (a) a images, and then finds out the representative image
source image, (b) the shape representation of (a) aiming at the centroid images of different formats
(compressed and uncompressed) to represent the set of
with the block size 32 32 pixels, (c) the shape medical images. From the experimental results, the
representation of (a) with the block size 16 16 proposed scheme can effectively achieve the retrieval Table 5. Performance evaluation using difference-
pixels. tasks no matter aiming at the whole set of compressed image features extracted from the feature image.
and uncompressed images.
x 256 pixels with 8 bits/pixels. We classify the whole The future research will further make retrieval
set of images into 145 sets, each has 4 slices, by using aiming at the medical images of local illness, i.e. liver
their correlation coefficients. The element of this MRI Fig. 4 Six examples of test images used in the tumor, connect the image collection system, image
image database is illustrated as follow: experiment. processing work station and image database system in
1. Head, 64 slices, totaled to 16 sets. retrieved in the total retrieval times. Based on this Table 1. Performance evaluation using the
2. Shoulder, 116 slices, totaled to 29sets. concept, the factor Precision is defined as Eq. (9). difference image features extracted from the whole
3. Trunk, 128 slices, totaled to32sets. Precision = (Times that the images are correctly image.
4. Thigh, 108 slices, totaled to27sets. retrieved/ Total retrieval times) x 100% (9)
5. Legs, 68 slices, totaled to 17sets. the hospital, and combine all the various medical
6. Feet, 96 slices, totaled to 24 sets. Similarly, supposing there are N candidate images diagnosis image systems in the hospital, so as to make
similar with the retrieval image in the database, the so- full use of the resources. When the doctor reads the
The other source of the used image database is CT called similarity with the retrieval image means their electrical patient record, he can also read the relative
images. Each of CT images has 256 x 256 pixels with corresponding correlation coefficients are greater than images to further obtain the integral data of the patient.
8 bits/pixels, including a set of head images with 49 a certain critical value. Here, we define the factor As a result, an effective and accepted diagnosis and
slices and a set of chest images with 15 slices. Recall to be the percentage of the candidate images treat environment is created for the doctors.
To evaluate the performance of the proposed similar with the retrieval image in the first N candidate Table 2. Performance evaluation using the
scheme for image retrieval, we use the common used image. The formula is expressed as Eq. (10). ACKNOWLEDGEMENT
factor, i.e. Precision and Recall, to represent the difference-image features extracted from the
retrieval accuracy. Recall =(Number of retrieved similar image feature image.
Supposing the retrieval image is one of the images This work is supported by Chang Gung Memorial
pieces /Number of total similar image pieces) x 100%
stored in the database, we define the accuracy to be Hospital under the contract CMRP1365.
100% if the first candidate image provided by the
system is just the desired image; otherwise, the Using these images of the medical image database
accuracy is 0%. Consequently, we want to know the described as above, we evaluate the performance of
percentage of times that the images are correctly proposed scheme and the experimental results are
199 Vol. 15 No. 5 October 2003 APPLICATIONS, BASIS & COMMUNICATIONS 200
REFERENCE THE APPLICATION OF FULL-SCALE 3D
1. Del Bimbo A. and Pala P., "Effective image retrieval 8. Kobayakwa M., Hoshi M. and Ohmori T, "Robust ANTHROPOMETRIC DIGITAL MODEL SYSTEM ON
using deformable templates", Pattern Recognition, texture image retrieval using hierarchical correlation
vol. 3, pp. 120 -124, 1996. of wavelet coefficients", Pattern Recognition, vol. 3, BREAST RECONSTRUCTION OF PLASTIC SURGERIES
2. Patrice, B., Konik H. "Texture Similarity Queries pp. 391 -396, 2000.
and Relevance Feedback for Image Retrieval", 9. Yang C.C. and Chan M.C, "Color image retrieval
Pattern Recognition, vol. 4, pp. 55 -58, 2000. based on textural and chromatic features", IEEE SHUH-PING SUN1, JING-SHYR CHEN2
3. Kim B.Y., Kim H.J. and Jang S.J., "Image Retrieval International Conference, vol. 4, pp.922-927, 1999.
Based on Color Coherence", Proceedings of the 10. Gevers T. and Smeulders A.W.M., "PicToSeek: 1
Department of Biomedical Engineering, I-Shou University
IEEE Region 10 Conference (TENCON 99), vol. 1, Combing color and shape invariant features for 2
pp. 178 -181, 1999. image retrieval", IEEE Trans. Image Processing, Department of Plastic Surgery, Kaohsiung Veterans General Hospital
4. Le Bourgeois F., "Content based image retrieval vol. 91, pp. 102-119, Jan 2000. Kaohsiung, Taiwan
using gradient color fields", Pattern Recognition, 11. Kwok-Wah Hung and Aw-Yong M, "A Content-
vol. 1, pp. 1027 -1030, 2000. based Image Retrieval System Integrating
5. Daescu O., Kogge P.M. and Chen D.Z., "Parallel Color,Shape and Spatial Analysis", IEEE
content-based image analysis on PIM processors", International Conference on Systems Man and ABSTRACT
IEEE Workshop on Content-Based Access of Image Cybernetics, vol.2, pp. 1481-1488, 2000.
and Video Libraries, pp. 73 -77, 1998. 12. Ruey-Feng Cheng, Wen-Jia Kuo and Hung-Chi
6. Lay J.A. and Ling Guan, "Image retrieval based on Tsai, "Image Retrieval on Uncompressed and A full-scale 3D anthropometric digital model system is a set of technology that combined with
energy histograms of the low frequency DCT Compressed Domains", International Conference 3D digital imaging system, computer 3D image processing system, reverse engineering and
coefficients", IEEE International Conference, vol. 6, on Image Processing, vol. 2, pp. 546 -549, Jan. Computer-Aided-Design. The purpose of this studied is to make a full size solid breast model by
pp. 3009 -3012, 1999. 2001. using the 1:1 anthropometric digital model technique to assist breast reconstruction plastic surgery
7. Tolias Y., Panas S. and Tsoukalas L.H., "FSMIQ: 13. K. Karadimitriou et al., "Centroid method for colon the same size of symmetrical breast of the patient. The full-sized simulating breast model
fuzzy similarity matching for image queries", compressing sets of similar images", Pattern created in this studied not only can assist plastic surgeons by making more symmetric breasts on the
International Conference on Information Intelligence Recognition Letters, vol. 19, no. 7, pp.585-593, other side during the reconstruction of the breast surgeries, but also can go into a process of
and Systems, pp. 249 -254, 1999. 1998. analyzing if the two sides of the breasts are symmetrical. This studied is used the 3D optics scanner
to scan on patients' breasts to obtain the breast 3D image data and then used reverse engineering
technique and CAD software to simulate and to analyze the 3D image model of the reconstruction
breasts. If this type of solid model is needed during the medical treatment, it can apply 3D digital
data into a Rapid Prototyping machine to make the full-sized solid model. Doctors can using this
solid model go into a process of evaluation and planning before the surgeries and consider being an
important reference on breast reconstruction surgeries so that it can reduce patients' inconvenience
and promote the medical treatment qualities.
Biomed Eng Appl Basis Comm, 2003 (October); 15: 199-205.
Keywords: Reverse Engineering, 3D Anthropometric Measurement, Plastic Surgeries, Computer-
1. INTRODUCTION females eliminated their breast due to breast cancers, at
this time, they will be facing the impact of breast
cancer recovers and mastectomy afterwards at the
As for humans, breasts held a unique position.
same time. Moreover, the spirit of females will suffer
Not only do the breasts have normal physiological
under a great pressure. According to what the scholars
functions, but also they represent the symbols of
have found is that simply used extraneous fake breasts
females. Sometimes mental setbacks can be more
cannot improve or ease the feeling of patients' body
painful than physical wound when losing breasts. If
images. However, breast reconstruction surgeries can
lower the impact to the lowest level. In general, about
Received: Aug 25, 2003; Accepted: Oct 9, 2003 sixty percent of patients who have breast cancers
Correspondence: Shuh-Ping Sun, Professor receive various methods of reconstruction after
Department of Biomedical Engineering, eliminating the breasts [1-8]. There are more and more
I-Shou University, Kaohsiung County, Taiwan 840 breast cancers patients after eliminating the breasts to
Email: email@example.com receive the breasts reconstruction surgeries.