Characterizing Generalized Rate-Distortion Performance of Videos

Rate-distortion (RD) analysis is at the heart of lossy data compression. Here we extend the idea to generalized RD (GRD) functions of compressed videos that characterize the visual quality of a video and its encoding profile, which includes not only bit rate but also other attributes such as video resolution. We first define the theoretical functional space of the GRD function by analyzing its mathematical properties. We show that the GRD function space is a convex set in a Hilbert space, inspiring a computational model of the GRD function, based on which a general framework is proposed for GRD function reconstruction from known samples. We collect a large-scale database of GRD functions generated from diverse video contents and encoders. Using the database, we demonstrate that real-world GRD functions are clustered in a low-dimensional subspace in the theoretical space of all possible GRD functions. Combining the GRD reconstruction framework and the learned low-dimensional space, we create a low-parameter eigen GRD (eGRD) method to accurately estimate the GRD function of a source video content from only a few queries. Experimental results show that the proposed algorithm significantly outperforms state-of-the-art empirical RD estimation methods in accuracy and efficiency. Finally, we demonstrate the usefulness of the proposed eGRD model in a practical application: video codec comparison.