Learning hyperspectral images from RGB images via a coarse-to-fine CNN

Hyperspectral remote sensing is well-known for its extraordinary spectral distinguishability to discriminate different materials. However, the cost of hyperspectral image (HSI) acquisition is much higher compared to traditional RGB imaging. In addition, spatial and temporal resolutions are sacrificed to obtain very high spectral resolution owing to the limitations of sensor technologies. Therefore, in this paper, HSIs are reconstructed using easily acquired RGB images and a convolutional neural network (CNN). As a result, high spatial and temporal resolution RGB images can be inherited to HSIs. Specifically, a two-stage CNN, referred to as the spectral super-resolution network (SSR-Net), is designed to learn the transformation model between RGB images and HSIs from training data, including a band prediction network (BP-Net) to estimate hyperspectral bands from RGB images and a refinement network (RF-Net) to further reduce spectral distortion in the band prediction step. As a result, the learned joint features in the proposed SSR-Net can directly predict HSIs from their corresponding scenes in RGB images without prior knowledge. Experimental results obtained on several benchmark datasets demonstrate that the proposed SSR-Net outperforms several state-of-the-art methods by ensuring higher quality in HSI reconstruction, and significantly improves the performance of traditional RGB images in classification.