Anomaly Detection and Localization: A Novel Two-Phase Framework Based on Trajectory-Level Characteristics

Detecting and locating anomalies defined as unusual and irregular behaviors are important for public security in surveillance videos. In this paper, we propose a novel feature called Point Trajectory-based Histogram of Optical Flow (PT-HOF) to better capture the fine-grained spatial and temporal information along the point trajectory in crowd scenes. By encoding the extracted features through an unsupervised autoencoder network, the high-level representation features are used to build a Gaussian Mixture Model for estimating the anomaly likelihood of each trajectory. Furthermore, the consistency motion object (CMO) is constructed by clustering similar point trajectories in a local region to analyze the spatial structure of trajectories, which can improve the accuracy of anomaly localization. Experiments on two benchmark datasets demonstrate the advantage of the proposed algorithm by comparing with state-of-the-art methods.