A hierarchical learning approach for railway fastener detection using imbalanced samples

Abstract Fastener needs to be detected periodically to maintain the railway safety. However, the detection performance of the existing methods is insufficient in the case of imbalanced fastener samples. To tackle this problem, a hierarchical learning approach which consists of fastener localization and fastener detection is proposed in this paper. Firstly, a multi-scale features-based deep detection network (MSF-DDN) is proposed to locate the fastener regions from railway images. Then, a region classification network is constructed to recognize the type of key sub-regions obtained from the located fastener region images. Finally, fastener detection is achieved by analyzing the recognition results of key sub-regions through the constructed decision tree. A large number of experiments are conducted on the collected real railway images. The experimental results indicate that the hierarchical learning approach achieves an average precision of 96.4% and recall of 96.3% on the detection of imbalanced fasteners, which outperforms state-of-the-art methods.