Scanning and tracking dynamic objects with depth cameras

摘要

A video conference server receives a plurality of video frames including a current frame and at least one previous frame. Each of the video frames includes a corresponding image and a corresponding depth map. The server produces a directional distance function (DDF) field that represents an area surrounding a target surface of the object captured in the current frame. A forward transformation is generated that modifies the reference surface to align with the target surface. Using at least a portion of the forward transformation, a backward transformation is calculated that modifies the target surface of the current frame to align with the reference surface. The backward transformation is then applied to the DDF to generate a transformed DDF. The server updates the reference model with the transformed DDF and transmits data for the updated reference model to enable a representation of the object to be produced at a remote location.

主权项

1. A method comprising:
receiving a plurality of video frames comprising a current frame and at least one previous frame, wherein each of the plurality of video frames includes data for a corresponding image and data for a corresponding depth map; receiving a reference model comprising data from the at least one previous frame, the reference model including data representing a reference surface corresponding to at least one dynamic object captured in the plurality of video frames; processing the data representing the depth map of the current frame to produce data for a directional distance function (DDF) field that represents an area surrounding a target surface of the at least one dynamic object captured in the current frame; generating a forward transformation that modifies the data representing the reference surface to align with data representing the target surface and generate data for a modified reference surface; calculating a backward transformation that modifies data representing the target surface of the current frame to align with the reference surface, wherein the backward transformation uses at least a portion of the forward transformation; generating a transformed DDF by applying the backward transformation to the DDF; fusing the transformed DDF into a multi-mode distance function (MDDF) comprising multiple distance values and direction vectors at each of a plurality of voxels; and updating the reference model based on the MDDF to produce data for an updated reference model.