Determining networked mobile device position and orientation for augmented-reality window shopping

摘要

A consumer user can perform a number of augmented reality shopping operations using a local client shopping application executable on a mobile device. Features related to implementations of the current subject matter can include, but are not limited to, the ability to view an augmented reality depiction of a shopping display using the mobile device and to execute various “e-commerce” functions accessible via the augmented reality depiction. Visual image processing based on one or more inputs from sensors on the mobile device can be used to compute and efficiently provide the two dimensional projection of the three dimensional model to be presented on the screen of the mobile device. Related methods, systems, articles of manufacture, and the like are described.

主权项

1. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
receiving, at a processor from a mobile device executing a local client shopping application, sensor data generated by one or more sensors associated with the mobile device; narrowing, at the processor using the sensor data generated by the one or more sensors, a number of available degrees of freedom relating to a current view of a shopping display being sensed by a camera of the mobile device; estimating, at the processor based on the narrowed available degrees of freedom and using a computer vision algorithm, a pose and position of the camera relative to a three dimensional model of a shopping display; computing, at the processor based on the pose and position of the camera, a two dimensional projection of the three dimensional model; augmenting, at the processor based on the two dimensional projection, a two dimensional picture comprising an augmented reality marker associated with a physical item in the shopping display; and transmitting the two dimensional picture to a mobile device for presentation on a screen of the mobile device.