Sensor-based mobile search, related methods and systems

摘要

A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

主权项

1. A method comprising:
receiving first data corresponding to imagery captured by a camera of a smartphone, the imagery depicting a visual subject; deriving recognition features from the imagery, said deriving being performed by a processing system in the smartphone configured to perform such act; receiving second data corresponding to non-image stimulus captured by a sensor of the smartphone, said non-image stimulus comprising at least one stimulus selected from the group consisting of: audio, temperature, magnetic field, smell, or chemical presence; from a set of reference recognition features associated with a first set of visual subjects, identifying a smaller subset of recognition features associated with a second, smaller set of visual subjects, said smaller set of visual subjects including first and second visual subjects; using the non-image stimulus, classifying an environment of the smartphone by assigning a first probability value that the smartphone environment is a first environment and assigning a second probability value that the smartphone environment is a second environment, both of said probability values being more than 0% and less than 100%; obtaining two values respectively indicating likelihoods of encountering the first visual subject in said first and second environments, and obtaining two other values respectively indicating likelihoods of encountering the second visual subject in said first and second environments; and combining said probability and likelihood values together in assessing that the visual subject is more likely to be the first visual subject than the second visual subject: wherein the visual subject is identified from among said second set of subjects, by correspondence between the derived recognition features and recognition features in said subset, and by use of said probability and likelihood values.