Random Walk Erasing with Attention Calibration for Action Recognition

Action recognition in videos has attracted growing research interests because of the explosive surveillance data in social security applications. In this process, due to the distraction and deviation of the network caused by occlusions, human action features usually suffer different degrees of performance degradation. Considering the occlusion scene in the wild, we find that the occluded objects usually move unpredictably but continuously. Thus, we propose a random walk erasing with attention calibration (RWEAC) for action recognition. Specifically, we introduce the random walk erasing (RWE) module to simulate the unknown occluded real conditions in frame sequence, expanding the diversity of data samples. In the case of erasing (or occlusion), the attention area is sparse. We leverage the attention calibration (AC) module to force the attention to stay stable in other regions of interest. In short, our novel RWEAC network enhances the ability to learn comprehensive features in a complex environment and make the feature representation robust. Experiments are conducted on the challenging video action recognition UCF101 and HMDB51 datasets. The extensive comparison results and ablation studies demonstrate the effectiveness and strength of the proposed method.