Mask-Height R-CNN: An End-to-End Network for 3D Building Reconstruction from Monocular Remote Sensing Imagery

3D building reconstruction from monocular remote sensing imagery is a promising and economical way to generate 3D city models at a large scale, yet the task is rarely touched. The paper tackles the problem via an end-to-end network. The goal is achieved by a modified network, named Mask-Height R-CNN, based on Mask R-CNN, with an additional height prediction head in the Region Proposal Network (RPN). Unlike most deep learning based methods, the height estimation is done on the instance level instead of pixel level, which does not require the assembly of the height maps and building masks. The proposed network gains good performances on ISPRS datasets, with 3D F1 scores of over 0.8.