Adaptive multi-modal detection and fusion in videos via classification-based-learning

摘要

Described is a system for object detection using classification-based learning. A fusion method is selected, then a video sequence is processed to generate detections for each frame, wherein a detection is a representation of an object candidate. The detections are fused to generate a set of fused detections for each frame. The classification module generates a classification score labeling each fused detection based on a predetermined classification threshold. Otherwise, a token indicating that the classification module has abstained from generating a classification score is generated. The scoring module produces a confidence score for each fused detection based on a set of learned parameters from the learning module and the set of fused detections. The set of fused detections are filtered by the accept-reject module based on one of the classification score or the confidence score. Finally, a set of final detections representing an object is output.

主权项

1. A system for object detection for video sequences using classification-based learning, the system comprising:
one or more processors and a non-transitory memory having instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of: selecting a fusion method; processing an input video sequence to generate a plurality of detections for each frame in the input video sequence, wherein a detection is a representation of an object candidate within the frame of the input video sequence; fusing the plurality of detections to generate a set of fused detections for each frame using the selected fusion method; sending the set of fused detections to a classification module, a scoring module, and an accept-reject module; wherein the classification module:
a) generates a classification score for each fused detection, labeling the fused detection based on a predetermined classification threshold, and passes the classification score to a learning module and the accept-reject module; orb) generates a token indicating that the classification module has abstained from generating a classification score and passes the token to the learning module and the accept-reject module; wherein the scoring module produces a confidence score for each fused detection based on a set of learned parameters from the learning module and the set of fused detections; filtering of the set of fused detections by the accept-reject module based on one of the classification score or the confidence score; and outputting a set of final detections representing an object.