Bootstrapping language models for spoken dialog systems using the world wide web

摘要

A system, method and computer readable medium that generates a language model from data from a web domain is disclosed. The method may include filtering web data to remove unwanted data from the web domain data, extracting predicate/argument pairs from the filtered web data, generating conversational utterances by merging the extracted predicate/argument pairs into conversational templates, and generating a web data language model using the generated conversational utterances.

主权项

1. A method comprising:
identifying, via a processor communicating with Internet resources, common task independent web-sentences based on frequently occurring phrases across multiple websites from a web domain stored in a data store; selectively removing the common task independent web-sentences from the web domain data, to yield filtered web domain data comprising domain-specific data; identifying, via the processor, predicate/argument pairs from the filtered web domain data; replacing, via the processor, the predicate/argument pairs with predicate/argument tokens; generating, via the processor, conversational utterances by merging the predicate/argument tokens with manually written conversational templates while preserving a relative frequency of the manually written conversational templates, to yield generated conversational utterances; and generating, via the processor, a web data language model using the generated conversational utterances, and providing it as an initial language model for deployment of an automated speech recognition system.