Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

With the increasing availability and resolution of satellite sensor data, multispectral (MS) and panchromatic (PAN) images are the most popular data that are used in remote sensing among applications. This paper proposes a novel cross-resolution hidden layer features fusion (CRHFF) approach for joint classification of multi-resolution MS and PAN images. In particular, shallow spectral and spatial features at a global scale are firstly extracted from a MS image. Then deep cross-resolution hidden layer features extracted from MS and PAN are fused from patches at a local scale according to an Autoencoder (AE) like deep network. Finally, the selected multi-resolution hidden layer features are classified in a supervised manner. By taking advantage of integrated shallow-to-deep and global-to-local features from the high-resolution MS and PAN images, the cross-resolution latent information can be extracted and fused in order to better model imaged objects from the multi-model representation, and finally increase the classification accuracy. Experimental results obtained on three real multiresolution data sets covering complex urban scenarios confirm the effectiveness of the proposed approach in terms of higher accuracy and robustness with respect to literature methods.