Classification, search, and retrieval of complex video events

摘要

A complex video event classification, search and retrieval system can generate a semantic representation of a video or of segments within the video, based on one or more complex events that are depicted in the video, without the need for manual tagging. The system can use the semantic representations to, among other things, provide enhanced video search and retrieval capabilities.

主权项

1. A video search assistant embodied in one or more machine readable storage media and accessible by a computing system to assist a user with a video search by:
receiving a user-specified search request; determining a higher level complex event of interest, based on the user-specified search request; accessing a video event model, the video event model comprising: (i) a plurality of semantic elements associated with a plurality of higher level complex events depicted in the plurality of videos, each higher level complex event evidenced by at least two different lower level complex events, each of the semantic elements describing one or more of a scene, an action, an actor, and an object depicted in one or more of the videos, and (ii) data indicative of evidentiary relationships of different combinations of semantic elements forming the lower level complex events, wherein the video event model is derived by: (i) executing one or more event classifiers on one or more dynamic low level features of the videos to identify one or more semantic elements associated with the dynamic low level features, (ii) computing a strength of association of the one or more semantic elements with ones of the lower level complex events, (iii) and computing a strength of association of the associated ones of the lower level complex events with the higher level complex event; determining, based on the video event model, one or more semantic elements of interest associated with the higher level complex event of interest; and formulating a search for one or more videos depicting the higher level complex event of interest, the search comprising one or more of the semantic elements of interest.