Hierarchical visual comfort assessment for stereoscopic image retargeting

Abstract Stereoscopic Image Retargeting (SIR) has made it possible for the popularity of 3D application. Meanwhile, the adjustments brought to images may affect the visual comfort when enjoying 3D service. While for SIR, previous Visual Comfort Assessment (VCA) methods often cannot perform well, because they only analyze the influence of disparity on discomfort and do not take into account the effects from the unique and complex distortions of SIR. In this paper, we propose a Hierarchical Visual Comfort Assessment (Hi-VCA) scheme for SIR, considering hybrid distortions including structure, information, semantic distortions usually occurring in retargeting, and binocular incongruity existing in stereoscopic multimedia. Specifically, we first propose valid Local-SSIM and Dual Natural Scene Statistics (D-NSS) features to measure structural distortion and information loss. Considering disparity adjustments may brought by SIR, we design the binocular incongruity measurement by analyzing various binocular anomaly perception mechanisms of HVS. Finally, CNN-based feature is utilized to ensure the correct delivery of semantic information. Each measurement is complementary in describing visual comfort degradation and they are further aggregated. Extensive experiment results on published SIR database SIRD and two ordinary databases IEEE-SA and NBU 3D-VCA, demonstrate Hi-VCA has superior performance by better handling hybrid distortions compared to state-of-the-art schemes.