Single trunk multi-scale network for micro-expression recognition

Abstract Micro-expressions are the external manifestations of human psychological activities. Therefore, micro-expression recognition has important research and application value in many fields such as public services, criminal investigations, and clinical diagnosis. However, the particular characteristics (e.g., short duration and subtle changes) of micro-expressions bring great challenges to micro-expression recognition. In this paper, we explore the differences in the direction of facial muscle movement when people make different expressions to recognize micro-expressions. We first use optical flow to capture the subtle changes in the facial movement when a micro-expression occurs. Next, we extract facial movement information to an aniso-weighted optical flow image based on anisotropically weighting the horizontal and vertical components of the optical flow. Finally, we feed the aniso-weighted optical flow image into the proposed Single Trunk Multi-scale Network for micro-expression recognition. In particular, the designed multi-scale feature catcher in the network can capture features of micro-expressions with different intensities. We conduct extensive experiments on four spontaneous micro-expression datasets, and the experiment results show that our proposed method is competitive and effective.