COMPUTER IMPLEMENTED SYSTEM AND METHOD FOR RECOGNIZING AND COUNTING PRODUCTS WITHIN IMAGES

摘要

The system and method for recognizing and counting products within images is disclosed. The products are recognized by carrying out a nearest-neighbor search in the template feature space using a k-d tree and the product count is then obtained by using a maximum feature repeatability index for each identified feature. The system also obtains product arrangement by fitting bounding boxes around each identified product. The count of boxes thus obtained may also provide the exact number of discrete products visible in an image. A second stage of grid-based search is also carried out in the neighborhood of each detected product to detect new products that might have been missed out in the previous step. This detection is based on a confidence measure including information like histogram matching and spatial location of products. This system is also useful in verifying planogram compliance for a given product.

主权项

1. A computer implemented system for recognizing and counting stock within a defined environment comprising an area of shelves and aisles between the shelves, said shelves adapted to removably hold said stock comprising clusters of products, said system comprising:
an intransient repository configured to store a set of rules to create a k-d tree having descriptors from a plurality of pre-determined product templates, identify descriptors in captured images, compare the descriptors in the k-d tree with the identified descriptors and compare number of descriptors with a first threshold value and a second threshold value, said intransient repository further configured to store a look-up table containing the plurality of pre-determined product templates corresponding to the products having pre-determined descriptors, the first threshold value and the second threshold value; a processor configured to cooperate with the intransient repository to receive the set of rules and further configured to use the received set of rules to generate a set of commands; a k-d tree creator configured to cooperate with the processor to receive the set of commands and to create, based on the received set of commands, the k-d tree containing descriptors, said k-d tree creator further configured to store the created k-d tree in the intransient repository; at least one monitoring device adapted to move between the aisles, said monitoring device associated with an image capturing device configured to capture a plurality images of the clusters of products; a descriptor identifier configured to cooperate with the processor to receive the set of commands and with the monitoring device to receive the captured images and further configured to use the received set of commands to identify descriptors in the captured images, wherein said descriptors are pre-determined interest point features in the captured images; a transient repository configured to cooperate with the descriptor identifier to receive and store the identified descriptors; a first threshold comparator configured to cooperate with the processor to receive the set of commands, with the transient repository to receive the stored captured images having the identified descriptors and with the intransient repository to receive the stored first threshold value, and further configured use the received set of commands to compare number of the identified descriptors in the captured images with the received first threshold value to obtain valid identified descriptors; a first comparator configured to cooperate with the processor, the threshold comparator and the intransient repository, and further configured to compare based on the set of commands, the valid identified descriptors in the captured images with the descriptors in the stored k-d tree to obtain matching descriptors; a second threshold comparator configured to cooperate with the processor to receive the set of commands, with the first comparator to receive captured images having the obtained matching descriptors and with the intransient repository to receive the stored second threshold value, and further configured use the received set of commands to compare number of the matching descriptors in the captured images with the received second threshold value to obtain suitable descriptors; a template identifier configured to cooperate with the intransient repository and said second threshold comparator, and further configured to identify the plurality of product templates corresponding to the suitable descriptors from the stored look-up table; a product identifier configured to cooperate with the template identifier to receive the identified product templates and the intransient repository to receive the stored look-up table, and further configured to recognize products within the captured images from the stored look-up table based on the identified product templates; and a product counter configured to cooperate with the processor to receive the set of commands and with the product identifier to receive the captured images containing the recognized products, and further configured to use the received set of commands to count products based on the number of recognized within the captured images.