Multimodal interactions based on body postures

摘要

In one example, a method for multimodal human-machine interaction includes sensing a body posture of a participant using a camera (605) and evaluating the body posture to determine a posture-based probability of communication modalities from the participant (610). The method further includes detecting control input through a communication modality from the participant to the multimedia device (615) and weighting the control input by the posture-based probability (620).

主权项

1. A method for multimodal human-machine interaction comprising:
sensing a body posture of a participant using a camera; evaluating the body posture to determine a posture-based probability that certain modalities of communication will be presented by the participant based on the body posture, the posture-based probability being calculated using a processor in a multimedia device; wherein the modalities of communication comprise gestural and vocal; precluding, as input o the processor, a gestural modality of communication based on a determination that the calculated posture-based probability of the gestural modality of communication is below a predetermined level while receiving as input the vocal modality of communication; detecting control input presented via a modality of communication by the participant using a sensing device in the multimedia device; weighting the control input by the posture-based probability to determine a command to associate with the control input; and wherein calculating the posture-based probability of the gestural modality of communication is accomplished according to the following formula:P⁡(M)=∑K,A⁢P⁡(M|K)⁢P⁡(K|A)⁢P⁡(A) where
P(M) is the probability of a modality M;P(M|K) is the probability of a modality given a posture K;P(K|A) is the probability of a posture given an application A; andP(A) is the probability of an application for a multimedia device.