END-TO-END VISUAL RECOGNITION SYSTEM AND METHODS

摘要

We describe an end-to-end visual recognition system, where “end-to-end” refers to the ability of the system of performing all aspects of the system, from the construction of “maps” of scenes, or “models” of objects from training data, to the determination of the class, identity, location and other inferred parameters from test data. Our visual recognition system is capable of operating on a mobile hand-held device, such as a mobile phone, tablet or other portable device equipped with sensing and computing power. Our system employs a video based feature descriptor, and we characterize its invariance and discriminative properties. Feature selection and tracking are performed in real-time, and used to train a template-based classifier during a capture phase prompted by the user. During normal operation, the system scores objects in the field of view based on their ranking.

主权项

1. A visual recognition apparatus for identifying objects captured in a video stream having a captured time period, the apparatus comprising:
a computer processor; and programming in a non-transitory computer readable medium and executable on the computer processor for: capturing a video stream on an electronic device having an image sensor, said video stream comprising a plurality of temporally adjacent images; enabling a user of the electronic device to select, from said video stream, a target object or scene for training; associating each frame in an image with a corresponding frame in temporally adjacent images, or in images taken from nearby vantage points; and temporally aggregating statistics computed at one or more collections of temporally corresponding frames, into a descriptor.