A two-stage real-time YOLOv2-based road marking detector with lightweight spatial transformation-invariant classification

Abstract In recent years, Autonomous Driving Systems (ADS) become more and more popular and reliable. Road markings are important for drivers and advanced driver assistance systems by better understanding the road environment. While the detection of road markings may suffer a lot from various illumination, weather conditions and angles of view, most traditional road marking detection methods use fixed threshold to detect road markings, which is not robust enough to handle various situations in the real world. To deal with this problem, some deep learning-based real-time detection frameworks such as Single Shot Detector (SSD) and You Only Look Once (YOLO) are suitable for this task. However, these deep learning-based methods are data-driven even while there is no public road marking dataset. Besides, these detection frameworks usually struggle with distorted road markings and balancing between the precision and recall. We propose a two-stage YOLOv2-based network to tackle distorted road marking detection as well as to balance precision and recall. The proposed spatial transformer layer is able to handle the distorted road markings in the second stage, so as to achieve the improvement of precision. Our network is able to run at 58 FPS in a single GTX 1070 under diverse circumstances. Furthermore, we present a dataset for the public use of road marking detection tasks, which consists of 11,800 high-resolution images captured under different weather conditions. Specifically, the images are manually annotated into 13 classes with bounding boxes. We empirically demonstrate both mean average precision (mAP) and detection speed of our system over several baseline models.