METHOD FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL AND APPARATUS FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL

摘要

Audio inpainting is a technique for recovering missing audio samples. Known inpainting methods are not suitable if the missing speech samples are very different from the remaining available speech signal, or if a gap covers an entire word or a sequence of words. An improved method for speech inpainting comprises synthesizing speech or using natural speech recordings for a gap that occurs in a speech signal by using a transcript of the speech signal, converting the synthesized or natural speech by voice conversion according to the original speech signal, and blending the synthesized or natural converted speech into the original speech signal to fill the gap. The disclosed speech audio inpainting technique plausibly recovers lost speech parts with the help of synthetic speech generated from the text transcript or natural speech recordings of the missing part. The synthesized speech is modified by voice conversion to fit with the original speaker's voice.