LGGD+: Image Retargeting Quality Assessment by Measuring Local and Global Geometric Distortions

Numerous image retargeting algorithms have been proposed to achieve adaptive image resizing during the past years. To compare different image retargeting algorithms, reliable objective image retargeting quality assessment (IRQA) metrics are highly desired. Given that image retargeting usually introduces geometric distortions, this paper presents an objective IRQA metric by measuring both local and global geometric distortions (LGGD). Since human visual system perception is highly dependent on edges and the geometric distortions caused by image retargeting usually cause edge deformation, a sketch token-based local edge descriptor (ST-LED) is introduced to represent geometric-aware features in LGGD. First, ST-LED is first applied on both source and retargeted images for edge pattern representation. Second, pixel-level backward registration is conducted to enable estimating local geometric distortion (LGD) and a spatial pyramid-improved Bag-of-Token (BoT) model is built to enable estimating global geometric distortion (GGD). Since the proposed LGGD metric only focuses on geometric distortion while image retargeting quality is related with more aspects, we further fuse LGGD and an existing (EXT) IRQA metric to build a final version called LGGD+ for IRQA. Experiments on two benchmark databases demonstrate the superiority of LGGD+ and the excellent compatibility of our proposed LGGD for further improving a wide range of existing IRQA metrics (including both geometric distortion and non-geometric distortion metrics). In addition, the effectiveness of our LGGD metric is also demonstrated in another relevant task, i.e., quality evaluation of depth-image-based rendering (DIBR)-synthesized images, which also calls for accurate estimation of geometric distortion.