Perceiving informative key-points: A self-attention approach for person search

Abstract Though person re-identification has witnessed significant progress, person search, as a more practical task considering the unavailability of annotations of pedestrian bounding boxes, has progressed much slower because of less discriminative feature representation. To this end, we propose a novel self-attention based person search approach by perceiving informative implicit key-points with weak supervision. Firstly, we design the Self-attention Slice Part (SSP) module, to implicitly localize informative key-points by only taking a pre-defined number of points as supervision. Concretely, this module utilizes both channel-attention and spatial attention with weak supervision on partitioned pedestrian slices to get the most discriminative key-points. After that, we strengthen the self-attention weight for these cardinal key-points, and then, more robust feature representations for conducting person search can be obtained from these self-mined key-points. Meanwhile, the SSP also provides semantic alignment with the horizontally partitioned slices. Besides, to focus more on reducing the inner-class margin rather than enlarging inter-class distance, the Random Label Smooth (RLS) loss is defined for more robust classification. The RLS loss not only provides a larger margin hyperplane but also enhances the training efficiency. Therefore, all in all: (1) We propose an end-to-end person search framework by fully exploiting current object detection and person re-identification techniques jointly with weak supervision. (2) Our proposed weakly-supervised self-attention module is generic and can be plugged into any related tasks to improve the performance. (3) We conduct extensive experiments on popular benchmarks, including the dataset of CUHK-SYSU and PRW, and our approach outperforms most current state-of-the-art methods according to mAP and top-1 evaluation metrics.