Point cloud and visual feature-based tracking method for an augmented reality-aided mechanical assembly system

To improve the applicability and robustness of the three-dimensional tracking method of an augmented reality-aided assembly guiding system for mechanical products, a tracking method based on the combination of point cloud and visual feature is proposed. First, the tracking benchmark coordinate system is defined using a reference model point cloud to determine the position of the virtual assembly guiding information. Then a camera tracking algorithm combining visual feature matching and point cloud alignment is implemented. To obtain enough matching points of visual features in a textureless assembly environment, a novel ORB feature-matching strategy based on the consistency of direction vectors is presented. The experimental results show that the proposed method has good robust stability and tracking accuracy in an assembly environment that lacks both visual and depth features, and it can also achieve good real-time results. Its comprehensive performance is better than the point cloud-based KinectFusion tracking method.