Facilitating 3D Object Tracking in Point Clouds with Image Semantics and Geometry

Recent works have shown remarkable success in 3D object tracking in point clouds. However, these methods may fail when tracking distant objects or objects interfered by similar geometries in point clouds. We aim to use high-resolution images with rich textures to help point cloud based tracking to deal with the above-mentioned failures. In this paper, we propose an end-to-end framework, which effectively uses both image semantic features and geometric features to facilitate tracking in point clouds. Specifically, we design a fusion module to establish the correspondence between image and point cloud features in a point-to-point manner and use attention-weighted image features to enhance point features. In addition, we utilize geometric transformation to convert 2D image geometric features inferred by deep layer aggregation network (DLA) to 3D as extra tracking clues for 3D voting. Quantitative and qualitative comparisons on the KITTI tracking dataset demonstrate the advantages of our model.