RECOGNITION OF ITEMS DEPICTED IN IMAGES

摘要

Products (e.g., books) often include a significant amount of informative textual information that can be used in identifying the item. An input query image is a photo (e.g., a picture taken using a mobile phone) of a product. The photo is taken from an arbitrary angle and orientation, and includes an arbitrary background (e.g., a background with significant clutter). From the query image, the identification server retrieves the corresponding clean catalog image from a database. For example, the database may be a product database having a name of the product, image of the product, price of the product, sales history for the product, or any suitable combination thereof. The retrieval is performed by both matching the image with the images in the database and matching text retrieved from the image with the text in the database.

主权项

1. A system comprising:
a memory having instructions embodied thereon; and one or more processors configured by the instructions to perform operations comprising:
storing a plurality of records for a plurality of corresponding items, each record of the plurality of records including text data and image data for the item corresponding to the record;accessing a first image depicting a first item;generating a first set of candidate matches for the first item from the plurality of items based on the first image and the image data of the plurality of records;recognizing text in the first image;generating a second set of candidate matches for the first item from the plurality of items based on the recognized text and the text data of the plurality of records;combining the first set of candidate matches and the second set of candidate matches into a combined set of candidate matches; andidentifying a top-ranked candidate match of the combined set of candidate matches.