Real-Time 3D Motion Tracking and Reconstruction System Using Camera and IMU Sensors

Jointly using the data of visual and inertial sensors to achieve higher accuracy and robustness is a problem of high computational complexity. In this work, this paper presents a 3D motion tracking and reconstruction system on a mobile device using camera and Inertial Measurement Unit (IMU) sensors. The mobile device has very small active infrared projection depth sensors with high-performance IMU and wide field of view cameras. We utilize Visual-Inertial Odometry (VIO) to integrate visual and IMU data. However, traditional VIO method is too complex to apply to a mobile device with limited computing resources in real time. We employ Sliding Window Filter (SWF) for the proposed system to achieve accurate 3D motion tracking and high-quality reconstruction models in real time based on delayed state marginalization. Moreover, we also extensively evaluate in the dataset and real world. Many experiments show the accurate trajectories and high quality 3D reconstruction models by the proposed system. To the end, the qualitative and quantitative experimental results indicate the proposed system based on SWF has much better performance than existing methods in motion tracking and 3D reconstruction models.