Enhancing a spatio-temporal resolution of a depth data stream

摘要

What is disclosed is a system and method for enhancing a spatio-temporal resolution of a depth data stream. In one embodiment, time-sequential reflectance frames and time-sequential depth frames of a scene are received. If a temporal resolution of the reflectance frames is greater than the depth frames then a new depth frame is generated based on correlations determined between motion patterns in the sequence of reflectance frames and the sequence of depth frames. The new depth frame is inserted into the sequence of depth frames at a selected time point. If a spatial resolution of the reflectance frames is greater than the depth frames then the spatial resolution of a selected depth frame is enhanced by generating new pixel depth values which are added to the selected depth frame. The spatially enhanced depth frame is then inserted back into the sequence of depth frames.

主权项

1. A method for enhancing a spatio-temporal resolution of time-sequential depth frames of a scene, the method comprising:
receiving time-sequential reflectance frames of pixel intensity values acquired of a scene by at least one reflectance sensing device; receiving time-sequential depth frames of pixel depth values of said scene acquired by at least one depth sensing device; and performing at least one of:
(A) in response to a temporal resolution of said reflectance frames being greater than a temporal resolution of said depth frames,
selecting a time point where a depth frame does not exist in said sequence of depth frames;constructing at least one new depth frame based on correlations determined between motion patterns in said sequence of reflectance frames and said sequence of depth frames; andinserting said new depth frame into said sequence of depth frames at said selected time point; and(B) in response to a spatial resolution of said reflectance frames being greater than a spatial resolution of said depth frames,
selecting a time point where a depth frame does exist in said sequence of depth frames; andenhancing a spatial resolution of said existing depth frame at said selected time point based on correlations determined between a distribution of pixel intensity values of said reflectance frames and a distribution of pixel depth values of said associated depth frames.