Multimodal Convolutional Neural Networks with Cross-Channel Reconstruction

With the ever-growing availability of remote sensing (RS) data from either satellite or airborne sensors, simultaneous processing and analysis of multimodal data have been paid more and more attention by researchers in various RS-related applications. In this paper, we propose a multimodal convolutional neural network with an advanced cross-channel reconstruction module, called CCR-Net. As the name suggests, CCR-Net enables a more compact fusion of different RS data sources by the means of the reconstruction strategy across modalities that can mutually exchange information in a more effective way. Experiment are conducted on a widely-used dataset, including hyperspectral and Light Detection and Ranging (LiDAR) data, i.e., Houston2013, to verify the effectiveness and superiority of the proposed CCR - N et in comparison with several state-of-the-art baseline methods.