Systems and methods for determining and displaying multi-line foreign language translations in real time on mobile devices

摘要

The present invention is related to systems and methods for translating language text on a mobile camera device offline without access to the Internet. More specifically, the present invention relates to systems and methods for displaying text of a first language and a translation of the first language text into a second language text which is displayed in real time in augmented reality on the mobile device. The processing can use a single line or a multiline algorithm designed with a plurality of processing innovations to insure accurate real-time translations without motion jitter. The invention may be used to help travelers in a foreign country with difficulties in reading and understanding text written in the local language of that country. The present invention may be utilized with wearable computers or glasses, producing seamless augmented reality foreign language translations. Some embodiments are particularly useful in translations from Asian languages to English. Some embodiments are especially optimized to translate multiple lines of foreign language text.

主权项

1. A method for providing one or more translations in a real-time video feed of a first language into a second language, comprising:
cropping a frame of the real-time video feed of one or more words of the first language to fit inside a bounding box to produce a cropped frame; performing character segment detection on the cropped frame to produce a plurality character segments; performing character merging on the character segments to produce a plurality of merged character segments while determining at least a shape score for at least one merged character segment; performing character recognition on the merged character segments by utilizing at least the shape score of the at least one merged character segment to produce a plurality of recognized characters with high scores; performing one or more translations on the recognized characters of the first language into one or more translated words of the second language; and displaying the translated words of the second language.