MeMu: Metric Correlation Siamese Network and Multi-class Negative Sampling for Visual Tracking

Abstract Despite the great success in the computer vision field, visual tracking is still a challenging task. The main obstacle is that the target object often suffers from interference, such as occlusion. As most Siamese network-based trackers mainly sample image patches of target objects for training, the tracking algorithm lacks sufficient information about the surrounding environment. Besides, many Siamese network-based tracking algorithms build a regression only with the target object samples without considering the relationship between target and background, which may deteriorate the performance of trackers. In this paper, we propose a metric correlation Siamese network and multi-class negative sampling tracking method. For the first time, we explore a sampling approach that includes three different kinds of negative samples: virtual negative samples for pre-learning the potential occlusion situation, boundary negative samples to cope with potential tracking drift, and context negative samples to cope with potential incorrect positioning. With the three kinds of negative samples, we also propose a metric correlation method to train a correlation filter that contains metric information for better discrimination. Furthermore, we design a Siamese network-based architecture to embed the metric correlation filter module mentioned above in order to benefit from the powerful representation ability of deep learning. Extensive experiments on challenging OTB100 and VOT2017 datasets demonstrate the competitive performance of the proposed algorithm performs favorably compared with state-of-the-art approaches.