Background modification in video conferencing

摘要

Methods and systems for real-time video processing can be used in video conferencing to modify image quality of background. One example method includes the steps of receiving a video including a sequence of images, identifying at least one object of interest (e.g., a face) in one or more of the images, detecting feature reference points of the at least one object of interest, and tracking the at least one object of interest in the video. The tracking may comprise aligning a virtual face mesh to the at least one object of interest in one or more of the images. Further, a background is identified in the images by separating the at least one object of interest from each image based on the virtual face mesh. The background is then modified in each of the images by blurring, changing a resolution, colors, or other parameters.

主权项

1. A computer-implemented method for real-time video processing, the method comprising:
receiving a video including a sequence of images; identifying at least one object of interest in one or more of the images; detecting at least one shape unit, at least one action unit, and a position vector of the at least one object of interest, the at least one shape unit representing a parameter of a face of the at least one object of interest, the at least one action unit representing a facial mimic, and the position vector corresponding to a rotation around three axes and a translation along the axes; generating a virtual face mesh from the at least one shape unit, the at least one action unit, and the position vector, the at least one shape unit controlling a shape of the virtual face mesh, the at least one action unit contributing to the shape of the virtual face mesh; tracking the at least one object of interest in the video, wherein the tracking comprises aligning the virtual face mesh to the at least one object of interest in one or more of the images based at least in part on one or more of the at least one shape unit, the at least one action unit, and the position vector; identifying a background in each of the images by separating the at least one object of interest from each image based at least in part on the virtual face mesh; modifying the background in each of the images, thereby generating a modified background; and generating a modified video which includes the at least one object of interest and the modified background.