Modeling Driver’s Visual Fixation Behavior Using White-Box Representations

Driving is a highly dynamic task requiring extensive and fast processing of visuospatial information in brief time intervals, while experienced drivers can efficiently perform this task. Therefore, investigating the human-like visual fixation behavior shows much potential in developing the advanced driving assistance systems and efficient autonomous vehicles. Literature shows that driver’s visual fixation behavior is guided by a combination of top-down and bottom-up mechanisms. Currently, existing excellent computational frameworks rely heavily on the deep neural network’s strong feature extraction ability, which is inherently task-unaware. These black-box models are limited in strengthening the guidance of task-aware top-down factors in governing the fixation behavior. Furthermore, using the black-box model to fit the training data directly decreases its interpretability. In this paper, we encourage to adopt the white-box method. By researching the findings in the intersection of neuroscience and psychology, we propose to identify two white-box representations for driving separately: the information selection representation including salience, effort, expectancy, and value, and the driving task-aware representation highlighting sub-task and speed. We present a novel computational framework to incorporate all the above factors to predict driver’s visual fixation systematically. On the BDDA dataset, we have conducted the extensive quantitative evaluation and qualitative analysis to validate the effectiveness. Its performance surpasses the previous state-of-the-art by +11.3% in the Pearson’s Correlation Coefficient, especially by a large margin of +24.5% and +28.8% in the turning operations. The ablation study further demonstrates the influence of each component in our framework. Moreover, the trained model has been deployed on the DR(eye)VE dataset to verify the generalization ability, demonstrating the potential of domain adaptation.