PERCEPTUAL OPTIMIZATION FOR MODEL-BASED VIDEO ENCODING

摘要

Perceptual statistics may be used to compute importance maps that indicate which regions of a video frame are important to the human visual system. Importance maps may be applied to the video encoding process to enhance the quality of encoded bitstreams. The temporal contrast sensitivity function (TCSF) may be computed from the encoder's motion vectors. Motion vector quality metrics may be used to construct a true motion vector map (TMVM) that can be used to refine the TCSF. Spatial complexity maps (SCMs) can be calculated from metrics such as block variance, block luminance, SSIM, and edge strength, and the SCMs can be combined with the TCSF to obtain a unified importance map. Importance maps may be used to improve encoding by modifying the criterion for selecting optimum encoding solutions or by modifying the quantization for each target block to be encoded.

主权项

1. A method of encoding a plurality of video frames having non-overlapping target blocks, the method comprising:
encoding, via an encoder, the plurality of video frames using importance maps, such that the importance maps modify quantization affecting the encoding quality of each target block to be encoded in each video frame, the importance maps being formed by:
configuring the importance maps using temporal information and spatial information; andcomputationally causing the importance maps indicate which parts of a video frame in the plurality of video frames which are most noticeable to human perception, such that: (i) in blocks where the importance maps take on high values, the block quantization parameter (QP) is reduced relative to the frame quantization parameter QPframe, resulting in higher quality for those blocks, and (ii) in target blocks where the importance maps take on low values, the block quantization parameter is increased relative to the frame quantization parameter QPframe, resulting in lower quality for those blocks.