Systems and Methods for Semantically Classifying and Extracting Shots in Video

摘要

The present disclosure relates to systems and methods for classifying videos based on video content. For a given video file including a plurality of frames, a subset of frames is extracted for processing. Frames that are too dark, blurry, or otherwise poor classification candidates are discarded from the subset. Generally, material classification scores that describe type of material content likely included in each frame are calculated for the remaining frames in the subset. The material classification scores are used to generate material arrangement vectors that represent the spatial arrangement of material content in each frame. The material arrangement vectors are subsequently classified to generate a scene classification score vector for each frame. The scene classification results are averaged (or otherwise processed) across all frames in the subset to associate the video file with one or more predefined scene categories related to overall types of scene content of the video file.

主权项

1. A method for classifying videos based on video content, comprising the steps of:
receiving a video file, the video file including a plurality of frames, where each frame includes a plurality of pixels; extracting a set of frames from the video file; for each frame in the extracted set of frames, determining whether the frame comprises a poor classification frame, removing the one or more poor classification frames from the extracted set of frames; dividing each frame in the extracted set of frames into one or more segments, where each segment includes relatively uniform image content; extracting image features from each segment to form a feature vector associated with each segment; generating a material classification score vector for each segment via one or more material classifiers based on the feature vector associated with each segment, where each material classification score vector includes one or more material classification scores associated with one or more predefined material content categories; and assigning each material classification score vector associated with its respective segment to each pixel in each respective segment for each respective frame in the set of frames.