Learning to Track by Bi-Directional Long Short-Term Memory Networks

Invisualtracking, deeplearningbasedtrackershave demonstrated strong competitiveness under many challenging conditions such as background clutters and occlusions. Several recent studies have devoted to defeating such conditions by using Recurrent Neural Networks (RNNs) to explore temporal smoothness and stable deep representation. In this paper, we propose a tracker based on Bi-directional Long Short-Term Memory network (Bi-LSTM) under the tracking-by-detection paradigm. In particular, the proposed tracker adapts the YOLO algorithm to detect the target in single frame, and combines both detected target and tracked target in Bi-LSTM. Since BiLSTM learns the representation of current frame from both historical frames and subsequent frames, the proposed tracker can simultaneously incorporate forward plays and backward plays. This brings two advancements: 1) Bi-LSTM extends visual features into temporal domain and controls motions of the target, and2)Bi-LSTMpredictsreliabletargetlocationbyincorporating both past and future motions. We evaluated the proposed tracker on the visual benchmark OTB challenge videos and confirmed its effectiveness by comparing with the representative trackers to illustrate the feasibility of managing trace memory.