Answering questions using environmental context

摘要

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving audio data encoding an utterance and environmental data, obtaining a transcription of the utterance, identifying an entity using the environmental data, submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the entity, and obtaining one or more results of the query.

主权项

1. A computer-implemented method comprising:
generating, by a mobile device, an audio recording of (i) a spoken, natural language question that has been asked about an item of media content and that does not name, or request the name of, the item of media content, and (ii) ambient sounds that are associated with the playback of the item of media content and that are recorded contemporaneously with the question being asked; forwarding the audio recording to a coordination engine server; receiving from the coordination engine server an answer to the question, the answer based on processing of the question by a query processing engine server and on processing of the ambient sounds by a content identification engine server; and in response to the question, providing, by the mobile device, an answer to the question that has been asked about the item of media content.