System for media correlation based on latent evidences of audio

摘要

A method for determining a relatedness between a query video and a database video is provided. A processor extracts an audio stream from the query video to produce a query audio stream, extracts an audio stream from the database video to produce a database audio stream, produces a first-sized snippet from the query audio stream, and produces a first-sized snippet from the database audio stream. An estimation is made of a first most probable sequence of latent evidence probability vectors generating the first-sized audio snippet of the query audio stream. An estimation is made of a second most probable sequence of latent evidence probability vectors generating the first-sized audio snippet of the database audio stream. A similarity is measured between the first sequence and the second sequence producing a score of relatedness between the two snippets. Finally a relatedness is determined between the query video and a database video.

主权项

1. A method for determining a relatedness between a query video and a database video, the method comprising the steps of:
extracting an audio stream from the query video to produce a query audio stream; extracting an audio stream from the database video to produce a database audio stream; producing a first-sized snippet from the query audio stream; producing a first-sized snippet from the database audio stream; generating a collection of single state HMM (Hidden Markov Model) for sounds within the snippet from the query audio stream and the snippet from the database audio stream; estimating a first most probable sequence of a HMM state probability vectors generating the first-sized audio snippet of the query audio stream; estimating a second most probable sequence of HMM state probability vectors generating the first-sized audio snippet of the database audio stream; measuring a similarity between the first sequence and the second sequence to produce a score of relatedness between the first-sized query audio snippet and the first-sized database audio snippet, wherein the step of measuring the similarity between the between the first sequence and the second sequence to produce a score of relatedness comprises the step of using a Viterbi algorithm and Kernalized Locality-Sensitive Hashing to determine whether two videos are related; and determining a relatedness between the query video and a database video based on the measure of similarity.