Soft-Boundary Label Relaxation with class placement constraints for semantic segmentation of the railway environment

Abstract In this paper, we focus on the challenging task of the semantic segmentation of train front-view images. Managing trackside facilities can be done by using detailed and precise information about the surrounding railway environment. Semantic segmentation enables us to understand the 2D environment, but there is no adequate large-scale dataset available for training a CNN for this purpose. Some attempts have been made to generate pseudo-data from unlabeled sequential frames to compensate for the lack of volume in training data, but the moving speed of trains makes it difficult to apply them directly. We aim to solve this problem by proposing the Soft Boundary Label Relaxation (Soft-BLR) method, which considers label boundaries extending over multiple pixels to cope with more severely distorted pseudo-data and to better train the CNN in the initial training stage. Furthermore, we modify the loss function to penalize inference results based on the distance from the label boundary to solve the misalignment problems of border pixels. Through experimental evaluation, we report that the proposed method outperforms previous methods on not only the semantic segmentation of challenging railway images, but also that of general street-view images.