Utilization of Semantic Planes: Improved Localization and Dense Semantic Map for Monocular SLAM in Urban Environment

In this letter, we propose a novel semantic direct monocular simultaneous localization and mapping (SLAM) system that fuses the semantic information obtained by an advanced deep neural network (DNN) into direct sparse odometry with loop closure(LDSO), with the purpose of improving the localization accuracy and building a dense semantic map of the urban environment. For localization, we apply a point reselection strategy based on coarse semantic plane (CSP) constraints to discard static points inconsistent with the nearby co-plane points of the same semantic class and dynamic points beyond the visible range. Moreover, a point group movement consistency (PGMC) check is utilized to decrease the impact of moving dynamic objects. For the dense semantic map, we model numerous small semantic planes from well-estimated points to measure the depth of each static pixel, rather than conduct stereo matching. Experimental results show that our method is more accurate than LDSO and comparable with ORB-SLAM in terms of localization. Moreover, it is capable of building a dense semantic map of the urban environment for better scene understanding.