A generalized trajectories-based evaluation approach for pedestrian evacuation models

Abstract The fundamental diagram and self-organized phenomena in crowds are widely used to test the applicability of evacuation models. These benchmarks are good indicators for the validity of a model, whereas they are insufficient descriptors for the realistic microscopic behaviors of pedestrians. In recent years, the rapid increase of the trajectory datasets which benefits from the development of recognition technologies open the door to new possibilities for an extensive quantitative validation of the models. In this work, a trajectories-based analysis approach which contains types of indexes is proposed. The indexes are a mix of macroscopic type (fundamental diagram index, speed choice index, and direction choice index) and microscopic type (trajectories pattern index), distribution type (route length distribution index, travel time distribution index) and time-series type (starting position distance time-series index, destination position distance time-series index. Moreover, the Kolmogorov-Smirnov (K-S) test as well as the dynamic time warping (DTW) method are introduced to quantify the similarities of results on different types of indexes. In brief, by comparing experimental and simulation trajectories, we can measure a set of performance scores in different perspectives. Here, the Social Force Model (SFM) and Heuristics Model (HM) are respectively introduced and evaluated. According to the proposed evaluation approach, we show that the HM performs better than the SFM. Our analysis approach is model agnostic and is defined in a general way, such that it can be applied for trajectory sets from different experiment settings. This work can help to improve the accuracy of simulation models, and the pedestrian safety in crowd activities and autonomous vehicle navigation will be benefited.