Confidence Distance Matrix for outlier identification: A new method to improve the characterizations of surfaces measured by confocal microscopy

Abstract This paper proposes a statistical method for outlier identification for surface measurement data obtained by confocal microscopy. The implemented statistical method is Confidence Distance Matrix (CDM) which were widely used in statistics and many engineering areas, such as signal processing, sensor data fusion, information problems, etc. However, no investigations on identifying outliers in measured surface data using CDM have been found. This paper introduces and simplifies the mathematical model of CDM method. Algorithms for identifying random outliers using Monte Carlo method for uncertainty evaluation and for identifying outliers in a unique measured surface are developed and validated. For validation of the algorithms, a synthetic data SG_3-3 provided by National Institute of Standards and Technology and a data of artificial stochastic surface generated by our own algorithms are implemented. The difference of Sq of the data with outliers is 2.3342% and after deletion of outliers is 0.0037% with reference to the certified value. A type C1 spacing standard with dust dropped is measured and processed using CDM. The difference of Sa decreases from 29.65% to 3.52% after processing outliers with reference to the certified value Ra. A steel plate is measured and processed. Surface slopes and curvatures of the data in the two validations and two experiments are compared. All those parameters, the surface reconstructions, histogram of heights, and QQ plot of the measured surface data versus the data after deletion of outliers indicate our proposed method working well.