Light regression memory and multi-perspective object special proposals for abrupt motion tracking

Abstract Visual tracking is challenging as the tracked target often undergo drastic appearance changes, fast motion, out-of-view and other abrupt motion in real world. Also, how to monitor the tracking failure in real-time and how to recover after tracking failure detected are very attractive research problems. In this paper, we propose the abrupt motion tracking method via the light regression memory network and multi-perspective object special proposals to resolve these problems. Firstly, we propose a highly accurate multi-perspective object special proposal method to recover tracking when tracking failure happened. Commonsense information is used as a priori knowledge to refine region proposals. Meanwhile, semantic target-aware estimation and spatial structure estimation are designed to rank and choose region proposals from two complementary viewpoints separately. By a comprehensive decision, the final proposals can fully cover the target’s states even under abrupt motion. Secondly, we learn the memory network on a single-convolutional layer network by convolution linear regression. Without training on large offline datasets, it only need to fine-tune a fixed number of network parameters online by reliable target frame information to reinforce the memory of the changes in the target’s appearance. Without size increasing and complex update strategy, the memory network can keep the good pattern of the target to reliably determine whether tracking fails, which ensures the effectivity of model updating and locating the target well. Finally, we integrate the memory network and region proposals into Siamese matching framework to achieve accurate abrupt motion tracking. Numerous experiments over multiple tracking benchmarks prove that the proposed tracker achieves excellent performance.