Focus Mismatches in Multiview Systems and Efficient Adaptive Reference Filtering for Multiview Video Coding

In this paper, we analyze focus mismatches among cameras utilized in a multiview system, and propose techniques to efficiently apply our previously proposed adaptive reference filtering (ARF) scheme to inter-view prediction in multiview video coding (MVC). We show that, with heterogeneous focus setting, the differences exhibit in images captured by different cameras can be represented in terms of the focus setting mismatches (view-dependency) and the depths of objects (depth-dependency). We then analyze the performance of the previously proposed ARF in MVC inter-view prediction. The gains in coding efficiency show a strong view-wise variation. Furthermore, the estimated filter coefficients demonstrate strong correlation when the depths of objects in the scene remain similar. By exploiting the properties derived from the theoretical and performance analysis, we propose two techniques to achieve efficient ARF coding scheme: i) view-wise ARF adaptation based on RD-cost prediction, which determines whether ARF is beneficial for a given view, and ii) filter updating based on depth-composition change, in which the same set of filters will be used (i.e., no new filters will be designed) until there is significant change in the depth-composition within the scene. Simulation results show that significant complexity savings are possible (e.g., the complete ARF encoding process needs to be applied to only 20% ∼35% of the frames) with negligible quality degradation (e.g., around 0.05 dB loss).