Interactive concept learning in image search

摘要

An interactive concept learning image search technique that allows end-users to quickly create their own rules for re-ranking images based on the image characteristics of the images. The image characteristics can include visual characteristics as well as semantic features or characteristics, or may include a combination of both. End-users can then rank or re-rank any current or future image search results according to their rule or rules. End-users provide examples of images each rule should match and examples of images the rule should reject. The technique learns the common image characteristics of the examples, and any current or future image search results can then be ranked or re-ranked according to the learned rules.

主权项

1. A computer-implemented process that finds one or more images of sought image characteristics, comprising:
obtaining a set of images; receiving user-selected example images from the set of images with sought visual image characteristics; interactively learning one or more weighted user-created rules for ranking images based only on the visual image characteristics of the user-selected example images that each rule matches and each rule rejects via a machine learning technique using a set of image histograms for each image of the image set, the overall structure of each image of the image set and the texture of each image of the image set, wherein each rule is based on learning a distance metric which computes a score for each image returned in the set of images that indicates how similar each image returned in the set of images is to the examples of images with image characteristics sought comprising:
computing a set of image histograms computed over the pixels in each image in the set of images;computing two distance metrics for each histogram of the set of image histograms;computing a structure histogram that represents the overall structure of each image in the set of images by applying a shape descriptor to the whole image;computing a texture histogram for each image in the set of images; andusing the computed image histograms, the distance metrics for each of the computed image histograms, the structure histogram and the texture histogram for each image to compute a score for each image in the set of images that indicates how similar each image in the set of images is to the example images with image characteristics sought and example images not having image characteristics sought;displaying image distance metrics and weights for each of the one or more learned rules on a display;changing one or more of the rules based on an edit to a displayed image distance metric or weight received from a user; andranking the set of images based only on the sought visual image characteristics and the one or more interactively learned user-created changed rules.