DVT-SLAM: Deep-Learning Based Visible and Thermal Fusion SLAM

The problem of visual odometry (VO) and localization in extreme illumination conditions is widely concerned. In this paper, we propose a novel SLAM algorithm namely DVT-SLAM (Deep-learning based Visible-Thermal SLAM). It focuses on the fusion of thermal infrared image and visible image which have complementary advantages in characteristics. With the contrastive learning and the measurement of mutual information between multi-modal images, the first part of DVT-SLAM is the DVT-GAN network to fuse visible-thermal images and generate pseudo visible images at night. Given the generated images, visual odometry is applied for pose estimation base. Extensive evaluations are performed on the Brno Urban Dataset, a multi-modal dataset containing different time and weather conditions in diverse scenarios. Series of experiments show that DVT-SLAM is a robustness and suitability solution for single visible camera failures, which can reduce positioning error by half and achieve superior SLAM performance.