METHOD OF CLASSIFYING A MULTIMODAL OBJECT

摘要

A method of classifying a multimodal test object described according to at least one first and one second modality is provided, including offline construction by classification of a multimedia dictionary, defined by a plurality of multimedia words, based on a recoding matrix of representatives of the first modality forming a dictionary of the first modality including a plurality of words of the first modality, the recoding matrix constructed to express the frequency of each word of the second modality of a dictionary of the second modality including a plurality of words of the second modality, for each word of the first modality, classification of a multimodal test object performed online by recoding each representative of the first modality relating to the multimedia object considered on the multimedia dictionary base, followed by aggregating representatives of the first modality coded in the recoding in a single vector representative of the multimodal object.

主权项

1. A method of classifying a multimodal test object termed a multimedia test object described according to at least one first and one second modality, characterized in that it includes a step of offline construction by unsupervised classification of a multimedia dictionary (Wm), defined by a plurality Km of multimedia words, on the basis of a recoding matrix (X) of representatives of the first modality forming a dictionary of the first modality including a plurality KT of words of the first modality, the recoding matrix (X) being constructed so that each of the components thereof forms information representative of the frequency of each word of the second modality of a dictionary of the second modality including a plurality Kv of words of the second modality, for each word of the first modality, the classification of a multimedia test object being performed online by means of a step of recoding each representative of the first modality relating to the multimedia object considered on the multimedia dictionary (Wm) base, followed by a step of aggregating the representatives of the first modality coded in the recoding step in a single vector (BoMW) representative of the multimedia object considered.