Compressed Video Action Recognition Using Motion Vector Representation

Action recognition is an important task for video understanding. Due to expensive time consumption, the conventional approaches employing the optical flow are difficult to be used for real-time purpose. Recently, the Motion Vector (MV), which can be directly extracted from the compressed video, has been introduced for action recognition. In this paper, we propose a novel approach by utilizing motion vector representation for action recognition. On the one hand, we use the motion vector information to select key information sequences for recognition. On the other hand, we further use the motion vector to formulate the representation of the selected sequences. We evaluate the proposed approach on UCF101 and HMDB51 datasets. The experimental results demonstrate that the proposed approach is able to achieve competitive recognition performance, and is able to maintain a 461.5 fps end-to-end processing rate at the same time.