Improved SiamRPN++ with Clustering-Based Frame Differencing for Object Tracking of Remote Sensing Videos

Deep learning (DL) based object tracking methods have achieved encouraging results on natural videos. However, directly applying these DL-based methods to the vehicle tracking of optical remote sensing videos (ORSV) still faces many challenges. Different with the vehicles in nature videos, most vehicles in ORSV are blurry, small in size, and highly similar to other vehicles. Furthermore, blurred vehicles easily blend into the background and are difficult to distinguish. To solve these problems, an improved Siamrpn++ with clustering-based frame differencing (CFD-SiamRPN++) is proposed. In CFD-SiamRPN++, a clustering method is used to divide the differencing map among adjacent frames into two clusters. By using the statistics of the clustering results, each cluster is judged to the target or not. According to the judgment of each cluster, the differencing map is refined by reducing the background noises and retaining moving information of vehicles. Then, the refined differencing map is fused with the original frame as the input of the tracking network to enhance the discriminative ability of small-sized and blurred vehicles. In the tracking phase, SiamRPN++ is selected as the tracking network due to its feature extraction capability with multi-scale feature fusion and efficient tracking performance. Experiment results based on Jilin-1 ORSV dataset show that the proposed method provides a competitive tracking performance over state-of art deep learning methods.