Interactive text recognition by a head-mounted device

摘要

A user is assisted in a real-world environment. An assistance engine receives at least one context from the user. The assistance engine also receives a video stream of the real-world environment. The assistance engine performs an optical character recognition process on the video stream based upon the at least one context. The assistance engine generates a response for the user. A microphone on a head-mounted device receives the context from the user. A camera on the head-mounted device captures the video stream of the real-world environment. A speaker on the head-mounted device communicates the response to the user. The user may move in the real-world environment based upon the response to improve the optical character recognition process.

主权项

1. A system for assisting a user in a real-world environment, the system comprising:
an assistance engine configured to:
receive, from a network transceiver of a head-mounted device, at least one context,receive, from the network transceiver, a video stream that comprises a first set of images,perform, based on the at least one context, an optical character recognition (OCR) process on the video stream, andtransmit, to the network transceiver, user information in response to performing the OCR process; and the head-mounted device comprising:
a microphone configured to receive the at least one context from the user,a camera configured to capture the video stream from the real-world environment,a speaker configured to communicate the user information to the user, andthe network transceiver configured to transmit the video stream and the at least one context to the assistance engine, the network transceiver further configured to receive the user information from the assistance engine.