Multilevel Cross-Aware RGBD Semantic Segmentation of Indoor Environments

Semantic segmentation is the main step towards scene understanding which is one of the most important tasks of computer vision. As the depth and color information are independent, the combination of depth and RGB images can improve the quality of semantic labeling. In this paper, we proposed a multilevel cross-aware network (MCA-Net) for RGBD semantic segmentation to jointly reason about 2D appearance and depth geometric information. Our MCA-Net utilizes basic residual structure to encode texture information and depth geometric information respectively. Multilevel cross-aware fusion modules are designed to fuse multi-scale complementary features extracted from RGB and depth images. The proposed network produces high quality segmentation results of RGBD images particularly in indoor environments. The experiments conducted on Scannet dataset demonstrate the effectiveness of apperceiving and fusing multilevel features and that proposed MCA-Net outperforms state-of-the-art methods.