Image processing based on depth information and color data of a scene

摘要

Systems, apparatuses, methods, and computer program products perform image processing in an environment in which depth information and color data of a scene including a player and a background are received from a capture device, and in which an image of the player combined with video data is output.

主权项

1. A method for performing image processing based on depth information and color data of a scene, the method comprising:
performing, by a processor, the steps of:
receiving the depth information of the scene and the color data of the scene from a capture device, wherein the scene includes a player and a background;applying a noise reduction filter to the color data;applying an image enhancing color transform to the color data;generating a player mask from the depth information;blurring the edges of the player mask;constructing a background model of the scene by performing, for each pixel, the steps of:
storing the position p of the pixel and color components of the pixel,initializing a model pixel at the same position p with a color component distribution,initializing a variable as an update weight,updating values of the model pixel according to a predetermined process based on the variable and the color components of the pixel, anddetermining the pixel to be a background pixel or a foreground pixel based on a predetermined rule, the color components of the pixel, and the updated values of the model pixel,wherein if the pixel is determined to be a background pixel, then the color data from the capture device is used to update the background model, and if the pixel is determined to be a foreground pixel, then the color data from the capture device is not used to update the background model; andoutputting an image of the player using the color data and depth information, wherein the image of the player is combined with video data, and wherein the image of the player is composited with the video data using the blurred mask as an alpha channel.