Spatially registered augmented video

摘要

A source video stream is processed to extract a desired object from the remainder of video stream to produce a segmented video of the object. Additional relevant information, such as the orientation of the source camera for each frame in the resulting segmented video of the object, is also determined and stored. During replay, the segmented video of the object, as well as the source camera orientation are obtained. Using the source camera orientation for each frame of the segmented video of the object, as well as target camera orientation for each frame of a target video stream, a transformation for the segmented video of the object may be produced. The segmented video of the object may be displayed over the target video stream, which may be a live video stream of a scene, using the transformation to spatially register the segmented video to the target video stream.

主权项

1. A method comprising:
obtaining a plurality of segmented image frames of an object, a first panoramic image associated with the plurality of segmented image frames of the object, and an orientation of a source camera with respect to the first panoramic image for each frame in the plurality of segmented image frames, the plurality of segmented image frames of the object captured with the source camera; causing a plurality of target image frames to be captured with a target camera; generating a second panoramic image from the plurality of target image frames; determining an orientation of the target camera with respect to the second panoramic image for each frame of the plurality of target image frames; matching the first panoramic image to the second panoramic image to align the first panoramic image to the second panoramic image; calculating a transformation for each frame of the plurality of segmented image frames of the object with the first panoramic image aligned to the second panoramic image using the orientation of the source camera with respect to the first panoramic image for each frame in the plurality of segmented image frames and the orientation of the target camera with respect to the second panoramic image for each respective frame in the plurality of target image frames; and causing the plurality of segmented image frames to be displayed over the plurality of target image frames using the transformation for each frame.