LF-MAGNet: Learning Mutual Attention Guidance of Sub-Aperture Images for Light Field Image Super-Resolution

Many light field image super-resolution networks are proposed to directly aggregate the features of different low-resolution sub-aperture images (SAIs) to reconstruct high-resolution sub-aperture images. However, most of them ignore aligning different SAI’s features before aggregation, which will generate sub-optimal light field image super-resolution results. To handle this limitation, we design a mutual attention mechanism to align the SAI’s features and propose a Light Field Mutual Attention Guidance Network (LF-MAGNet) constructed by multiple Mutual Attention Guidance blocks (MAGs) in a cascade manner. MAG achieves the mutual attention mechanism between center SAI and any surrounding SAI with two modules: the center attention guidance module (CAG) and the surrounding attention guidance module (SAG). Specifically, CAG first aligns the center-SAI features and any surrounding SAI features with the attention mechanism and then guides the surrounding SAI feature to learn from the center-SAI features, generating refined-surrounding SAI features. SAG aligns the refined-surrounding SAI feature and the original surrounding SAI feature and guides the refined surrounding SAI feature to learn from the original surrounding SAI features, generating the final outputs of MAG. With the help of MAG, LF-MAGNet can efficiently utilize different SAI features and generate high-quality light field image super-resolution results. Experiments are performed on commonly-used light field image super-resolution benchmarks. Qualitative and quantitative results prove the effectiveness of our LF-MAGNet.