Part defect recognition based on 2D and 3D feature combination

Surface defect recognition is used to test product’s quality. The current way of recognition is traditional 2D imagebased method. But 2D image lacks 3D information which results in false inspection and missed inspection, which has become a bottleneck of current classification model. Because of the recent rapid development of 3D measurement technology, we can apply 3D data information in surface defect detection to improve the recognition ability of defects. We propose a new convolutional network model to identify surface defects, and realize the feature depth fusion of 3D point cloud and 2D image in the model. In this work, we introduce an attention network to extract features from a 3D point cloud to generate a 2D attention mask. The high quality feature map is produced by combining the 2D attention mask with a 2D image. We further merge the attention network and the classification network into a single network. The attention network is used to analyze which part of the image should be more concerned by the classification network. Therefore, mutual learning of 2D data and 3D data is realized in the training process, which reduces the dependence on the number of samples and enhances the generalization performance of the model. Experiments on the defect dataset verify that our method can improve the classification effect of the model.