Query by image

摘要

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing queries made up of images. In one aspect, a method includes indexing images by image descriptors. The method further includes associating descriptive n-grams with the images. In another aspect, a method includes receiving a query, identifying text describing the query, and performing a search according to the text identified for the query.

主权项

1. A method performed by data processing apparatus, the method comprising:
receiving a group of images, each image having associated metadata; generating one or more local descriptor vectors for each image of the group of images; building an initial index for the group of images using the local descriptor vectors, wherein the initial index is configured to return images of the group of images that are visually similar to a query image; generating a set of matching images for each image using the initial index, wherein a matching image is determined based on a match score that is correlated to a visual similarity between images using the local descriptor vectors; using the group of images and their respective matches to determine a final image selection, including:
generating an undirected graph using the group of images wherein each vertex is an image and each edge is a match between images and wherein each edge is weighted according to the respective match scores, using the undirected graph to determine pairwise image affinity based on the edge weights along a path between images in the undirected graph, andselecting a final image selection from the group of images using the pairwise image affinity; calculating a set of n-grams and corresponding n-gram scores for each image using image affinity and image metadata; and generating a new index using the images of the final image selection and including corresponding n-grams for each image.