National Conference on Role of Cloud Computing Environment in Green Communication 2012                                                553

                                         S.M.ASHA JELBHIN, R.MERCY
       Abstract— Content-based image retrieval (CBIR) is an important and widely studied topic since it can have significant
      impact on multimedia information retrieval. Recently, nonseparable wavelet transform has been applied to the problem
      of CBIR. The wavelet transform based method has been compared with other methods such as neural network (NN) and
      logistic regression, and has shown good results. Simulated Annealing (SA) has been increasingly applied in conjunction
      with other AI techniques. However, few studies have dealt with the combining Simulated Annealing and wavelet
      transform, though there is a great potential for useful applications in this area. This project focuses on an image signature
      is derived from an adapted non-separable wavelet transform, using four different lattices. The performances of the
      adapted wavelet filter bank over the non-adapted wavelet filter bank are higher for every database. The system is
      compared to a similar CBIR system, but using an adapted SA based separable wavelet transform. In this study, we show
      that the proposed approach outperforms the image classification problem for CBIR.
      Index Terms—Content-based image retrieval, relevance feedback, query point movement, query expansion, navigation
      pattern mining, simulated annealing
      MULTIMEDIA contents are growing explosively and the need for multimedia retrieval is occurring more and more
      frequently in our daily life. Due to the complexity of multimedia contents, image understanding is a difficult but
      interesting issue in this field. Extracting valuable knowledge from a large-scale multimedia repository, so-called
      multimedia mining, has been recently studied by some researchers. Typically, in the development of an image
      requisition system, semantic image retrieval relies heavily on the related captions, e.g., file-names, categories,
      annotated keywords, and other manual descriptions [19], [20]. Unfortunately, this kind of textual-based image
      retrieval always suffers from two problems: high-priced manual annotation and inappropriate automated annotation.
      On one hand, high priced manual annotation cost is prohibitive in coping with a large-scale data set. On the other
      hand, inappropriate automated\annotation yields the distorted results for semantic image retrieval. As a result, a
      number of powerful image retrieval algorithms have been proposed to deal with such

      problems over the past few years. Content-Based Image Retrieval (CBIR) is the mainstay of current image retrieval
      systems. In general, the purpose of CBIR is to present an image conceptually, with a set of low-level visual features
      such as color, texture, and shape. These conventional approaches for image retrieval are based on the computation of
      the similarity between the user’s query and images via a query by example (QBE) system [21]. Despite the power of
      the search strategies, it is very difficult to optimize the retrieval quality of CBIR within only one query process. The
      hidden problem is that the extracted visual features are too diverse to capture the concept of the user’s query. To
      solve such problems, in the QBE system, the users can pick up some preferred images to refine the image
      explorations iteratively. The feedback procedure, called Relevance Feedback (RF),repeats until the user is satisfied
      with the retrieval results. Although a number of RF studies [1], [11], [12], [16] have been made on interactive CBIR,
      they still incur some common problems, namely redundant browsing and exploration convergence. First, in terms of
      redundant browsing, most existing RF methods focus on how to earn the user’s satisfaction in one query process.
      That is, existing methods refine the query again and again by analyzing the specific relevant images picked up by
      the users. Especially for the compound and complex images, the users might go through a long series of feedbacks
      to obtain the desired To resolve the aforementioned problems, we propose a novel method named Navigation-
      Pattern-based Relevance Feedback (NPRF) to achieve the high retrieval quality of CBIR with RF by using the
      discovered navigation patterns. In terms of efficiency, the navigation patterns mined from the user query log can be
      viewed as the shortest paths to the user’s interested space. According to the discovered patterns, the users can obtain
      a set of relevant images in an online query refinement process. Thus, the problem of redundant browsing is
      successfully solved. In terms of effectiveness,
      the proposed navigation-pattern-based search algorithm (NPRFSearch) merges three query refinement strategies,
      including Query Point Movement (QPM), Query Reweighting (QR), and Query Expansion (QEX), to deal with the
      problem of exploration convergence. In short, the

 Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012                                            554

      discovered navigation pattern in NPRFSearch can be regarded as an
      optimized search path to converge the search space toward the user’s intention effectively. As a whole, through
      NPRF, the optimal results can be attained in very few feedbacks.
      Relevance feedback [5], [17], [25], in principle, refers to a set of approaches learning from an assortment of users’
      browsing behaviors on image retrieval [10]. Some earlier studies for RF make use of existing machine learning
      techniques to achieve semantic image retrieval, including Statistics, EM, KNN, etc.
      2.1 Query Reweighting
      Some previous work keeps an eye on investigating what visual features are important for those images (positive
      examples) picked up by the users at each feedback (also called iteration in this paper). The notion behind QR is that,
      if the ith feature fi exists in positive examples frequently, the system assigns the higher degree to fi.
      2.2 Query Point Movement
      Another solution for enhancing the accuracy of image retrieval is moving the query point toward the contour of the
      user’s preference in feature space. QPM regards multiple positive examples as a new query point at each feedback.
      After several forceful changes of location and contour, the query point should be close to a convex region of the
      user’s interest.
      2.3 Query EXpansion
      Because QR and QPM cannot elevate the quality of RF, QEX has been another hot technique in the solution space
      of RF recently. That is, straightforward search strategies, such as QR and QPM, cannot completely cover the user’s
      interest spreading in the broad feature space. As a result, diverse results for the same concept are difficult to obtain.
      For this reason, the modified version of MARS [9] groups the similar relevant points into several clusters, and
      selects good representative points from these clusters to construct the multipoint query. Wu et al. [22] proposed
      FALCON, which is designed to handle disjunctive queries within arbitrary metric spaces. Qcluster, developed by
      Kim and Chung [8], intends to handle the disjunctive queries by employing adaptive classification and cluster
      merging methods. As experimented in earlier studies, the effectiveness of QEX is better than those of QPM and QR.
      Nevertheless, there are still some problems unsolved for QEX. For MARS, inappropriate search regions cannot deal
      with complex queries. For FALCON, the relevant query points are too many to be efficient. Adjusting the
      disjunctive queries causes the expensive search cost and the results cannot escape from the restricted range (clusters)
      that the users are able to specify. On the whole, QEX brings out higher computation cost and more feedbacks in RF.
      2.4 Hybrid RF
      In addition to past studies already described, another type of RF approach emphasizes the integration of various
      search strategies [2], [3], [4], [7], [18]. However, this kind of method is instinctive, and very little hybridized work
      focuses on the accumulated information (long-term usage log) coming from various users. Moreover, the greater
      effectiveness of the multisystem requires a higher computation cost, due to multiple processings. One of the hybrid
      RF strategies is IRRL. IRRL, proposed by Yin et al. [23], addresses the important empirical question of how to
      precisely capture the user’s interest at each feedback. In IRRL, exploiting knowledge from the long-term experience
      of users can facilitate the selection of multiple RF techniques to get the best results. The derived problems from

 Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012                                         555

      IRRL are: the selection of optimal RF technique cannot avoid the overhead of long iterations of feedback. Also, the
      visual diversity existing in the global feature space cannot be resolved with an optimal RF technique alone.
      As elaborated above, the critical issue of SA can be chiefly summarized thus: how to achieve effective and efficient
      image retrieval. To deal with this issue, we describe how our proposed approach simulated Annealing provide
      optimal weight to the feature so that image can be retrieved based on the feature of the image.
      3.1 Overview of Simulated Annealing
      The major difference between our proposed approach and other contemporary approaches is that we approximate an
      optimal solution to resolve the problems existing in current RF, such as redundant browsing and exploration
      convergence. To this end, the approximated solution takes advantage of exploited knowledge (navigation patterns)
      to assist the proposed search strategy in efficiently hunting the desired images. Generally, the task of the proposed
      approach can be divided into two major operations, namely offline knowledge discovery and online image retrieval..
      For online operation, once a query image is submitted to this system, the system first finds the most similar images
      without considering any search strategy, and then returns a set of the most similar images. The first query process is
      called initial feedback. Next, the good examples picked up by the user deliver the valuable information to the image
      search phase, including new feature weights, new query point, and the user’s intention. Then, by using the
      navigation patterns, three
      search strategies, with respect to QPM, QR, and QEX, are hybridized to find the desired images. Overall, at each
      feedback, the results are presented to the user and the related browsing information is stored in the log database.
      After accumulating long-term users’ browsing behaviors, offline operation for knowledge discovery is triggered to
      perform navigation pattern mining and pattern indexing.
      3.1.1 Online Image Retrieval
      Initial Query Processing Phase: Without considering the feature weight, this phase extracts the visual features from
      the original query image to find the similar images. Afterward, the good examples (also called positive examples in
      this paper) picked up by the user are further analyzed at the first feedback (also called iteration 0 in this
      paper).Image Search Phase: Behind the search phase, our intent is to extend the one search point to multiple search
      points by integrating the navigation patterns and the proposed search algorithm SA. Thus, the diverse inclusion of
      the user’s interest can be successfully implied. In this phase, a new query point at each feedback is generated by the
      preceding positive examples. Then, the k-nearest images to the new query point can be found by expanding the
      weighted query. The search procedure does not stop unless the user is satisfied with the retrieval results.
      3.1.2 Offline Knowledge Discovery
      Knowledge Discovery Phase: Learning from users’ behaviors in image retrieval can be viewed as one type of
      knowledge discovery. Consequently, this phase primarily concerns the construction of the navigation model by
      discovering the implicit navigation patterns from users’ browsing behaviors. This navigation model can provide
      image search with a good support
      to predict optimal image browsing paths.
      Data Storage Phase: The databases in this phase can be regarded as the knowledge marts of a knowledge warehouse,
      which store integrated, time-variant, and nonvolatile collection of useful data including images, navigation patterns,
      log files, and image features. The knowledge warehouse is very helpful to improve the quality of image retrieval.
      Note that the procedure of constructing rule base from the image databases can be conducted periodically to
      maintain the validity of the proposed approach.
      3.2 Offline Knowledge Discovery
      In fact, usage mining has been made on how to generate users’ browsing patterns to facilitate the web pages
      retrieval. Similarly, for web image retrieval, the user has to submit a query term to the search engine, so-called
      textual-based image search. Then the user can obtain a set of most relevant web images according to the metadata or
      the browsing log. However, if the result does not satisfy the user, the query refinement can be easily incorporated
      into the query procedure This is why CBIR using RF has been the focus of the researchers in the field of image
      retrieval. As far as the usage log of CBIR is concerned, the challenge mainly lies on: how to generate and utilize the
      discovered patterns. In this paper, we develop a navigation-patternbased data structure permeated by the query point
      movement aspect, which has never been proposed by past studies. Through the special data structure, the user’s
      intention can be caught more quickly and precisely. In detail, the data structure can be viewed as a hierarchy,
      including positive images, query points, and clusters. A query session contains a set of iterative feedbacks
      (iterations), which is referred to a navigation path. At each feedback, the positive examples, which indicate the
      results picked up by the user, are used to derive a referred visual
      query point by averaging the positive visual features. Finally, the query sessions, iterations, positive examples, and
      visual query points are stored into the original log database. If the original log data are ready, the next task is to

 Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012                                         556

      discover navigation patterns from the original log data. Basically, navigation pattern discovery consists of two
      stages: data transformation and navigation
      3.2.1 Data Transformation
      To date, very few significant studies have succeeded in semantic image retrieval or image recognition because of the
      complicated visual contents. To handle the vagueness in image presentation, data transformation for visual content is
      a fundamental and important operation because it can simplify both the description of visual query points and the
      discovery of navigation patterns.

      Fig. 2.The query point dictionary of the proposed approach

      In other words, without the data transformation, we have to consider all positive images of each query session in the
      log database. If all positive images are considered for navigation pattern mining, too many items make the frequent
      itemsets (navigation patterns) hard to find. Also, the mining cost is expensive. As a result, the aim of data
      transformation is to generate Query Point Dictionary (QPD) to reduce the kinds of items on the transaction list.
      3.2.2 Navigation Patterns Mining
      This stage focuses on the discovery of relations among the users’ browsing behaviors on RF. Basically, the frequent
      patterns mined from the user logs are regarded as the useful browsing paths to optimize the search direction on RF.
      Through these navigation patterns, the user’s intention can be precisely captured in a shorter query process. In this
      phase, the Apriori-like algorithm is performed to exploit navigation patterns using the transformed data. The task for
      establishing the navigation model can be decomposed into two steps:
      3.2.3 Pattern Indexing
      In this stage, we describe how to build the navigation pattern tree with the discovered navigation patterns. A point
      indicates a set of positive images. In particular, to decrease the complexities of both pattern search and pattern
      storage, the redundant navigation patterns have to be pruned further. After eliminating the redundant patterns, the
      trimmed navigation pattern tree reduces the search cost significantly. Based on the navigation pattern tree, the
      desired images can be captured more promptly without repeating the scan of the whole image database at each
      feedback, especially for the large-scale image data. The entity-relationship data model for partitioning the log data.
      3.3 Online Image Search
      3.3.1 Basic Idea
      As we can recall from previous explanations, the aim of the search strategy is to attack the weakness of the
      traditional approaches, including redundant browsing and exploration convergence. Indeed, these unsolved problems
      result in large limitation in RF. Perhaps, the aged hybrid systems fusing the results generated by multiple query
      refinement systems can look for the better results than individual systems. Nevertheless, the expensive computation
      cost makes it impractical in real applications. Instead, we attempt to approximate the optimal solution, namely SA,

 Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012                                           557

      to resolve such problems by using the generated navigation patterns. For the problem of exploration convergence,
      our proposed approach extends the search range from a query point to a number of relevant navigation paths. As a
      result, each iterative search can escape from the local optimal space and further move toward the global optimal
      space for the user’s interest. For the problem of redundant browsing, the discovered navigation patterns are adopted
      as the shortest paths to derive the superior results in a shorter feedback process. Additionally, the expensive
      navigation cost is saved further, especially for the massive image data. In general, the SA algorithm can be
      recognized as a very important part of our proposed iterative solution to RF, which merges QEX, QP, and QR
      4.1 Experimental Data
      The experimental data came from the collection of the Corel image database and the web images. We prepared
      seven data sets composed of different kinds of categories. Each category contains 200 images. In our experimental
      logs, we initially performed QPM to collect the log on the queries. Then, the navigation patterns are obtained by
      adopting our pattern discovery mechanism. Incrementally, the knowledge discovered from the navigation patterns
      can be enhanced once the new query is submitted to NPRF. Indeed, it does need time to gather the usage logs.
      However, a larger log that needs longer collection time can help achieve higher retrieval quality. An alternative way
      to reduce the whole collection time is to increase the size of collected logs incrementally such that the precision will
      also be enhanced gradually. To analyze the effectiveness of our proposed approach, two major criteria, namely
      precision and coverage, are used to measure the related experimental evaluations..
      4.2 Experimental Results
      In practice, the primary intentions behind our experiments are: 1) evaluations for parameter settings, 2) comparisons
      between NPRF and other existing RF approaches in terms of effectiveness and efficiency, and 3) the promise of the
      performance for the scale-up data.
      To deal with the long iteration problem of CBIR with RF, we have presented a new approach named SA. In
      summary, the main feature of NPRF is to efficiently optimize the retrieval quality of interactive CBIR. On one hand,
      the navigation patterns derived from the users’ long term browsing behaviors are used as a good support for
      minimizing the number of user feedbacks. On the other hand, the proposed algorithm SA performs the navigation-
      pattern-based search to match the user’s intention by merging three query refinement strategies. As a result,
      traditional problems such as visual diversity and exploration convergence are solved.
      This research was supported by the National Science Council, Taiwan, R.O.C., under grant no. NSC 98-2631-H-
      006-001, and by the Ministry of Economic Affairs, Taiwan, R.O.C., under grant no. 97-EC-17-A-02-S1-024.
      [1] M.D. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D.
      Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-
      32, Sept. 1995.
      [2] R. Fagin, “Combining Fuzzy Information from Multiple Systems,” Proc. Symp. Principles of Database Systems
      (PODS), pp. 216-226, June 1996.
      [3] R. Fagin, “Fuzzy Queries in Multimedia Database Systems,” Proc. Symp. Principles of Database Systems
      (PODS), pp. 1-10, June 1998.
      [4] J. French and X-Y. Jin, “An Empirical Investigation of the Scalability of a Multiple Viewpoint CBIR System,”
      Proc. Int’l Conf. Image and Video Retrieval (CIVR), pp. 252-260, July 2004.
       [5] D. Harman, “Relevance Feedback Revisited,” Proc. 15th Ann. Int’l ACM SIGIR
      Conf. Research and Development in Information Retrieval, pp. 1-10, 1992.
      [6] Y. Ishikawa, R. Subramanya, and C. Faloutsos, “MindReader: Querying Databases through Multiple Examples,”
      Proc. 24th Int’l Conf. Very Large Data Bases (VLDB), pp. 218-227, 1998.
      [7] X. Jin and J.C. French, “Improving Image Retrieval Effectiveness via Multiple Queries,” Multimedia Tools and
      Applications, vol. 26, pp. 221-245, June 2005.
      [8] D.H. Kim and C.W. Chung, “Qcluster: Relevance Feedback Using Adaptive Clustering for Content-Based Image
      Retrieval,” Proc. ACM SIGMOD, pp. 599-610, 2003.
      [9] K. Porkaew, K. Chakrabarti, and S. Mehrotra, “Query Refinement for Multimedia Similarity Retrieval in
      MARS,” Proc. ACM Int’l Multimedia Conf. (ACMMM), pp. 235-238, 1999.
      [10] J. Liu, Z. Li, M. Li, H. Lu, and S. Ma, “Human Behaviour Consistent Relevance Feedback Model for Image
      Retrieval,” Proc. 15th Int’l Conf. Multimedia, pp. 269-272, Sept. 2007.
      [11] A. Pentalnd, R.W. Picard, and S. Sclaroff, “Photobook: Content- Based Manipulation of Image Databases,”
      Int’l J. Computer Vision (IJCV), vol. 18, no. 3, pp. 233-254, June 1996.

 Department of CSE, Sun College of Engineering and Technology
National Conference on Role of Cloud Computing Environment in Green Communication 2012                                   558

      [12] T. Qin, X.D. Zhang, T.Y. Liu, D.S. Wang, W.Y. Ma, and H.J. Zhang, “An Active Feedback Framework for
      Image Retrieval,” Pattern Recognition Letters, vol. 29, pp. 637-646, Apr. 2008.
       [13] Y. Rui, T. Huang, and S. Mehrotra, “Content-Based Image Retrieval with Relevance Feedback in MARS,”
      Proc. IEEE Int’l Conf. Image Processing, pp. 815-818, Oct. 1997.
      [14] Y. Rui, T. Huang, M. Ortega, and S. Mehrotra, “Relevance Feedback: A Power Tool for Interactive Content-
      Based Image Retrieval,” IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 644-655, Sept.
      [15] J.R. Smith and S.F. Chang, “VisualSEEK: A Fully Automated
      Content-Based Image Query System,” Proc. ACM Multimedia Conf., Nov. 1996.
      [16] G. Salton and C. Buckley, “Improving Retrieval Performance by Relevance Feedback,” J. Am. Soc.
      Information Science, vol. 41, no. 4, pp. 288-297, 1990.
      [17] H.T. Shen, S. Jiang, K.L. Tan, Z. Huang, and X. Zhou, “Speed up
      Interactive Image Retrieval,” VLDB J., vol. 18, no. 1, pp. 329-343, Jan. 2009.
      [18] V.S. Tseng, J.H. Su, J.H. Huang, and C.J. Chen, “Integrated
      Mining of Visual Features, Speech Features and Frequent Patterns for Semantic Video Annotation,” IEEE Trans.
      Multimedia, vol. 10, no. 2, pp. 260-267, Feb. 2008.
      [19] V.S. Tseng, J.H. Su, B.W. Wang, and Y.M. Lin, “Web Image
      Annotation by Fusing Visual Features and Textual Information,”
      Proc. 22nd ACM Symp. Applied Computing, Mar. 2007.
      [20] K. Vu, K.A. Hua, and N. Jiang, “Improving Image Retrieval
      Effectiveness in Query-by-Example Environment,” Proc. 2003 ACM Symp. Applied Computing, pp. 774-781, 2003.
      [21] L. Wu, C. Faloutsos, K. Sycara, and T.R. Payne, “FALCON: Feedback Adaptive Loop for Content-Based
      Retrieval,” Proc. 26th Int’l Conf. Very Large Data Bases (VLDB), pp. 297-306, 2000.
      [22] P.Y. Yin, B. Bhanu, K.C. Chang, and A. Dong, “Integrating
      Relevance Feedback Techniques for Image Retrieval Using
      Reinforcement Learning,” IEEE Trans. Pattern Analysis and Machine
      Intelligence, vol. 27, no. 10, pp. 1536-1551, Oct. 2005.
      [23] H. You, E. Chang, and B. Li, “NNEW: Nearest Neighbor Expansion by Weighting in Image Database
      Retrieval,” Proc. IEEE Int’l Conf. Multimedia and Expo, pp. 245-248, Aug. 2001.
      [24] X.S. Zhou and T.S. Huang, “Relevance Feedback for ImageRetrieval: A Comprehensive Review,” Multimedia
      Systems, vol. 8, no. 6, pp. 536-544, Apr. 2003.

 Department of CSE, Sun College of Engineering and Technology

To top