Action Category and Phase Consistency Regularization for High-Quality Temporal Action Proposal Generation

Temporal action detection is a fundamental yet challenging task in video content analysis. The performance of existing methods still remains far from satisfactory as the mAP reduces dramatically at high tIoU threshold. With the goal of predicting the starting and ending points more precisely, this work first introduces the action category label into the temporal proposal generation stage of the training process. Specifically, with the category information, we proposed two extra constrains, i.e, action based constraint and action-class agnostic constraints. The former aims at minimizing the discrepancy inside the same action category while the latter forces the feature of the samples aggregates in the same phase. Comprehensive experiments are conducted on the THUMOS’14 benchmark. A remarkable improvement of average recall is attained especially when the number of proposals is small. And our approach achieves 29.0% mAP at a strict tIoU@0.7.