Building outline delineation: From aerial images to polygons with an improved end-to-end learning framework

Abstract Deep learning methods based upon convolutional neural networks (CNNs) have demonstrated impressive performance in the task of building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. In this paper, we introduce an improved method that is able to predict regularized building outline in a vector format within an end-to-end deep learning framework. The main idea of our framework is to learn to predict the location of key vertices of the buildings and connect them in sequence. The proposed method is based on PolyMapper. We upgrade the feature extraction by introducing global context and boundary refinement blocks and add channel and spatial attention modules to improve the effectiveness of the detection module. In addition, we introduce stacked conv-GRU to further preserve the geometric relationship between vertices and accelerate inference. We tested our method on two large-scale VHR-RS building extraction dataset. The results on both COCO and PoLiS metrics demonstrate better performance compared with Mask R-CNN and PolyMapper. Specifically, we achieve 4.2 mask mean average precision (mAP) and 3.7 mean average recall (mAR) absolute improvements compared to PolyMapper. Also, the qualitative comparison shows that our method significantly improves the instance segmentation of buildings of various shapes.