Methods and systems for image or audio recognition processing

摘要

Many of the detailed technologies are useful in enabling a smart phone to respond to a user's environment, e.g., so it can serve as an intuitive hearing and seeing device. A few of the detailed arrangements involve optimizing division of shared processing tasks between the phone and remote devices; using a phone GPU for exhaustive speculative execution and machine vision purposes (including facial recognition); novel device architectures involving abstraction layers that facilitate substitution of different local and remote services; interactions with private networks as they relate to audio/image processing; adapting the orders in which operations are executed, and the types of data that are exchanged with remote servers, in accordance with current context; reconfiguring networks based on sensed social affiliations among users and in accordance with predictive models of user behavior; etc. A great variety of other features and arrangements are also detailed.

主权项

1. In a distributed processing method that includes performing an operation on stimuli captured by a camera or microphone sensor of a user's mobile device, by using a combined system that includes both processing hardware in the mobile device and processing hardware remote from the mobile device, an improvement wherein:
the operation comprises an image or audio recognition operation, and the method includes: identifying, using a hardware processor, a set of component operations that should be executed by said combined system to perform said recognition operation; and determining a sequence in which said component operations should be performed based on one or more circumstance or context factors selected from the group consisting of: (a) information about mobile device power availability or usage; (b) information about a needed function response time; (c) information about a routing constraint; (d) information about a state of hardware resources within the mobile device; (e) information about mobile device connectivity; (f) information about a geographical consideration; (g) information about a pipeline stall risk; (h) information about turnaround time or cost associated with the remote processor; and (i) information about a user preference regarding remote processing; and wherein, at a first time, said component operations are performed in a first sequence, and at a second time, said component operations are performed in a second, different, sequence, due to a difference in one or more of said factors between the first and second times.