SCSA-Net: Presentation of two-view reliable correspondence learning via spatial-channel self-attention

Abstract Seeking reliable correspondences between pairwise images is non-trivial in feature matching. In this paper, we propose a novel network, called the Spatial-Channel Self-Attention Network (SCSA-Net), to capture abundant contextual information of correspondences for obtaining reliable correspondences and estimating accurate camera pose of the matching images. In our proposed SCSA-Net, we introduce two types of attention modules, i.e., the spatial attention module and the channel attention module. The two types of attention modules are able to capture complex global context of the feature maps by selectively aggregating mutual information in the spatial dimension and channel dimension, respectively. Meanwhile, we combine the outputs of two modules to generate rich global context and obtain feature maps with strong representative ability. Our SCSA-Net is able to effectively remove outliers, and simultaneously estimate accurate camera pose between pairwise images. These reliable correspondences and camera pose are vital for many computer vision tasks, such as SfM, SLAM and stereo matching. The tremendous experiments on outlier removal and pose estimate tasks have shown the better performance improvements of our SCSA-Net over current state-of-the-art methods on both outdoor and indoor datasets. Especially, our SCSA-Net outperforms the recent state-of-the-art OANet++ by 5.55% mAP5°on unknown outdoor datasets. Code is available at https://github.com/x-gb/SCSA-Net.