Noise reduction based on mouth area movement recognition

摘要

A computing device can capture video data of at least a portion of a mouth area (e.g., mouth, lips, tongue, chin, jaw) of a user of the device. The computing device can also capture sound data including a voice of the user as well as noise (e.g. background noise). The video data can be processed to detect a movement of the portion of the mouth area. The movement of the portion of the mouth area can be analyzed and compared with mouth area movement models characteristic of oral communication (e.g., speech, song). If the movement of the portion of the mouth area corresponds to at least one model characteristic of oral communication, then the movement indicates that the user is likely engaging in oral communication. Noise reduction can be applied and/or increased on the captured sound data to reduce noise and in turn enhance the user's voice.

主权项

1. A computer-implemented method, comprising:
capturing video information using a camera of a computing device, the video information showing at least a portion of a mouth area of a user of the computing device; capturing audio information using a microphone of the computing device, the audio information including voice data generated by the user and an amount of noise; processing the video information to determine a movement of the portion of the mouth area of the user; applying noise reduction to the audio information to generate modified audio information that corresponds to a reduction of at least a portion of the noise; transmitting, over a communication network, the modified audio information; determining that the movement of the portion of the mouth area does not correspond to user speech; and causing at least one of capturing the audio information, applying the noise reduction, or transmitting the modified audio information to cease being performed for at least a period of time.