retrieval

Document Sample
retrieval Powered By Docstoc
					             Agenda
• Introduction
• Bag-of-words models
• Visual words with spatial location
• Part-based models
• Discriminative methods
• Segmentation and recognition
• Recognition-based image retrieval
• Datasets & Conclusions
           Retrieval domains

Internet image search


Video search for
people/objects



Searching home photo collections
• Learning from Internet Image Search

• Joint learning of text and images

• Large scale retrieval
Noisy labels
 Improving Google’s Image Search
• Fergus, Fei-Fei, Perona, Zisserman, ICCV 2005
• Variant of pLSA
  that includes
  spatial information
       Re-ranking result: Motorbike
Topics in model          Automatically chosen topic
          Animals on the Web
Berg and Forsyth, CVPR 2006
Gather images using text search
Use LDA to discover “good” images using
  features based on nearby text, shape, color
       Boostrapping of Image Search
 Schroff, Zisserman, Criminisi, Harvesting Image Databases from the Web, ICCV 2007

Images returned with PENGUIN query                             Final ranking
                                                               using SVM




Removal of drawings and abstract images




Naives Bayes ranking using noisy metadata




Train SVM…….
                            OPTIMOL
Li, Wang, Fei-Fei CVPR 07
• Learning from Internet Image Search

• Joint learning of text and images

• Large scale retrieval
        Matching Words and Pictures

• Barnard, Duygulu, de Freitas, Forsyth, Blei,
  Jordan, JMLR 2003
Text to Images
                  Images to text
• Use Blobworld or nCuts to segments images into regions

• Need to deduce labels attached to each image
Images to text result
        Names and Faces in the News
Berg, Berg, Edwards, Maire, White, Teh, Learned-Miller, Forsyth. CVPR 2004

Collected 500,000 images and text captions from Yahoo! News




1.   Find faces (standard face detector), rectify them to same pose.
2.   Perform Kernel PCA and Linear Discriminant Analysis (LDA).
3.   Extract names from text.
4.   Cluster faces, with each name corresponding to a cluster.
5.   Use language model to refine results
• Initial
  clusters
• Clusters
  refined with
  language model
• Learning from Internet Image Search

• Joint learning of text and images

• Large scale retrieval
                   Vocabulary tree
Nistér & Stewénius CVPR 2006.

KD-tree in descriptor space

Inverse lookup of features

Specific object recognition
 Not category-level
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
Slide from D. Nister
            Pyramid Match Hashing
• Grauman & Darell, CVPR 2007

• Combines Pyramid Match Kernel
  (efficient computation of
  correspondences between two set of
  vectors) with Locality Sensitive Hashing
  (LSH) [Indyk & Motwani 98]

• Allows matching of the set of features
  in a query image to sets of features in
  other images in time that is sublinear
  in # images

• Theoretical guarantees
             Semantic Hashing
• Salakhutdinov and Hinton, SIGIR 2007
• Torralba, Fergus, Weiss, CVPR 2008

• Map images to
  compact binary codes
• Hash codes for fast
  lookup

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:15
posted:7/22/2012
language:English
pages:41