PREDICTIVE VIRTUAL REALITY DISPLAY SYSTEM WITH POST RENDERING CORRECTION

摘要

A virtual reality display system that generates display images in two phases: the first phase renders images based on a predicted pose at the time the display will be updated; the second phase re-predicts the pose using recent sensor data, and corrects the images based on changes since the initial prediction. The second phase may be delayed so that it occurs just in time for a display update cycle, to ensure that sensor data is as accurate as possible for the revised pose prediction. Pose prediction may extrapolate sensor data by integrating differential equations of motion. It may incorporate biomechanical models of the user, which may be learned by prompting the user to perform specific movements. Pose prediction may take into account a user's tendency to look towards regions of interest. Multiple parallel pose predictions may be made to reflect uncertainty in the user's movement.

主权项

1. A predictive virtual reality display system with post rendering correction, comprising at least one display viewable by a user;
at least one sensor that generates sensor data that measures one or more aspects of a pose of one or more body parts of said user; a pose predictor coupled to said at least one sensor, wherein said pose predictor is configured to calculate a predicted pose of said one or more body parts of said user at a future point in time, based on said sensor data generated by said at least one sensor and based on said future point in time; a 3D model of a scene; a scene renderer coupled to said at least one display, said pose predictor, and said 3D model, wherein said scene renderer is configured to
obtain an initial predicted pose at a future display update point in time from said pose predictor;calculate one or more 2D projections of said 3D model, based on said initial predicted pose; an image warper coupled to said at least one display, said scene renderer, and said pose predictor, wherein said image warper is configured to
receive said initial predicted pose at said future display update point in time from said scene renderer;receive said one or more 2D projections from said scene renderer;obtain a revised predicted pose at said future display update point in time from said pose predictor;calculate a change in pose between said initial predicted pose and said revised predicted pose;generate a rerendering approximation of said one or more 2D projections based on said change in pose;modify said one or more 2D projections based on said rerendering approximation;transmit said one or more 2D projections to said at least one display.