Understanding pedestrian behaviour with pose estimation and recurrent networks

Early detection of pedestrians, quick understanding of their behavior and prediction of their intention are of extreme importance in the quest to the 5th level of autonomous driving. The development of deep learning techniques for pose estimation, both in terms of quality and speed, advertise pose estimation as a highly informative representation for human figures present in an image. With the help of pose keypoints we aim to classify short sequences of images in different types of pedestrian behaviors such as standing or walking, crossing, speeding up, waving. For this we propose a set of features based on pose estimations and two recurrent architectures, one with single output, and one with multiple output. We train and evaluate our models on different subsets of JAAD dataset in order to deal with the imbalanced classes.