A Pixel-Level Segmentation-Synthesis Framework for Dynamic Texture Video Compression

Dynamic textures are time-varying motion patterns that exhibit certain temporal stationarity. The motion of such patterns is usually anisotropic, which is a great challenge for video coding. In this paper, we firstly analyze the predictive and temporal characteristics of dynamic textures and, based on the analysis, a pixel-level segmentation-synthesis framework for dynamic texture compression is proposed to improve predictive-coding efficiency. This framework consists of three sub-modules: dynamic texture segmentation, dynamic texture synthesis and fusion process. A deep-learning-based segmentation method is introduced to obtain a pixel-level mask. Subsequently, optical flow fields are used to generate flowlines and help analyze the motion periodicity of a given sequence. The flowlines and period values are utilized as inputs for flow-based dynamic texture synthesis. The segmented dynamic texture area is then reconstructed using the synthesized results at the decoder side at pixel-level, such that the overhead for transmitting these contents could be saved. The segmented mask is transmitted through an All-Zero-Run-Length coding algorithm and an effective fusion module is further proposed to reduce edge artifacts that occur at the boundary of dynamic texture areas. Coding blocks have been classified into three types and a weighting reconstruction scheme is performed. The proposed framework has been integrated into VTM-10.0. Experimental results on a collected dynamic texture test set demonstrate that 15.1% and 16.1% MOS BD-rate savings can be achieved on average for LDB and LDP configurations, respectively.