Combined lip reading and voice recognition multimodal interface system

摘要

The present invention provides a combined lip reading and voice recognition multimodal interface system, which can issue a navigation operation instruction only by voice and lip movements, thus allowing a driver to look ahead during a navigation operation and reducing vehicle accidents related to navigation operations during driving. The combined lip reading and voice recognition multimodal interface system in accordance with the present invention includes: an audio voice input unit; a voice recognition unit; a voice recognition instruction and estimated probability output unit; a lip video image input unit; a lip reading unit; a lip reading recognition instruction output unit; and a voice recognition and lip reading recognition result combining unit that outputs the voice recognition instruction