Three-dimensional scanning using existing sensors on portable electronic devices

摘要

The disclosed embodiments provide a method and system for operating a portable electronic device. The portable electronic device includes a camera that obtains a set of images and one or more inertial sensors that obtain inertial data associated with the portable electronic device. The portable electronic device also includes an analysis apparatus that uses the set of images to obtain a set of features in proximity to the portable electronic device. Next, the analysis apparatus updates a set of locations of the features based on the set of images and the inertial data. Finally, the analysis apparatus uses the set of features and the set of locations to provide a model of an environment around the portable electronic device without requiring the use of specialized hardware to track the features and the locations.

主权项

1. A method for operating a portable electronic device, comprising:
using a set of images from a camera on the portable electronic device to obtain a set of features in proximity to the portable electronic device; updating, by the portable electronic device, a set of locations of the features based on the set of images and inertial data from one or more inertial sensors on the portable electronic device, wherein updating the set of locations of the features based on the set of images and the inertial data comprises:
providing the inertial data as input to a three-dimensional (3D) model;using the 3D model to predict a two-dimensional (2D) location of a feature from the set of features in an image from the set of images;obtaining a measurement of the 2D location from the image; andusing a residual between the predicted 2D location and the measurement to estimate a 3D location of the feature; and using the set of features and the set of locations to provide, on the portable electronic device, the 3D model of an environment around the portable electronic device.