Spatial-Temporal Skeleton Feature: An Unit-Level Feature for Temporal Action Proposal Generation

Temporal Action Proposal generation is an important issue in the field of action detection. In this study, we introduce a novel unit-level feature called Spatial-Temporal Skeleton Feature (STSF). It combines skeleton features and optical flow features for extracting the temporal action proposal. Compared with RGB information, the skeleton information is more clear and concise in action representation and less sensitive to the changes of human appearance. The optical flow features extracted by the temporal CNN provide more temporal information. The combined features contain temporal-spatial information, which is essential for generating temporal action proposal. STSF can be embedded in any existing architecture related to temporal action proposal. The proposed feature embedded into TURN architecture was evaluated on THUMOS-14 dataset. The stateof-the-art results are achieved. We further test TURN with STSF as a proposal generation method in an existing action detector. Significant improvements are demonstrated in all aspects.