Prior Mask R-CNN Based on Graph Cuts Loss and Size Input for Precipitation Measurement

Fusing prior knowledge with data-driven deep learning for measurement is interesting and challenging. For the detection of metallographic precipitations, the measurements of size and shape of precipitations are roughly predictable in advance through a transmission electron microscope (TEM). In this article, we proposed a novel instance segmentation network named prior mask R-CNN by fusing prior knowledge for automatic precipitation detection. On the basis of the typical mask R-CNN framework, we made the following improvements. First, at the bounding box classification stage, in order to restore area information, we input the size information besides only uniform dimension features after the region of interest align (RoIAlign). Second, at the mask segmentation stage, we proposed a new loss function based on normalized graph cuts. It is category-sensitive by setting different weight strategies for different categories based on their prior shapes. In addition, from the point of view of practicality, we designed an effective measurement extraction module to get specific measurements, such as the length of precipitations, from the final prediction results of our network. In a variety of experiments, our method achieves the highest mean average precision (mAP) of 0.475 and 0.298 among different famous methods for bounding box detection and mask segmentation tasks, respectively, which proves the effectiveness of our method.