Longitudinal and Lateral Coupling Model Based End-to-End Learning for Lane Keeping of Self-driving Cars.

Longitudinal and lateral coupling control is essential for high-precision lane keeping of self-driving cars. Recently, the end-to-end based method, which can learn high dimensional features from driving dataset, has been a new way for lane keeping of self-driving cars. However, state-of-the-art end-to-end based solutions mostly focus on the steering prediction and neglect the coupling effects of longitudinal and lateral control. This paper proposes a 3DCNN-LSTM (3D Convolutional Neural Networks-Long Short-Term Memory) model based end-to-end learning method for longitudinal and lateral coupling lane keeping of self-driving cars. A coupling controller will be designed in TORCS for lane keeping data collection. Then, the raw dataset will be augmented for better model training. Next, a 3DCNN-LSTM model with combined steering and speed loss function is proposed for model training and validation. At last, both the offline model testing and the online driving in TORCS are designed for the end-to-end model evaluation. The experiments show the proposed end-to-end coupling model achieves higher-precision lane keeping compared with the traditional end-to-end model with only lateral prediction.