Ada-CCFNet: Classification of multimodal direct immunofluorescence images for membranous nephropathy via adaptive weighted confidence calibration fusion network

In the pathological diagnosis of early, late and non-membranous nephropathy, direct immunofluorescence is highly likely to present potentially specific lesions, while it is often overlooked due to the difficulty of screening with naked eyes. With the advanced progress of deep learning, they have shown powerful abilities in detecting potential lesions. In this paper, we propose an adaptive weighted confidence calibration fusion framework (Ada-CCFNet) consisting of a preprocessing module, an adaptive weighted confidence calibration fusion (Ada-CCF) module and a classification module for diagnosis of membranous nephropathy by classifying the multimodal direct immunofluorescence images. In the preprocessing module, we use the well-known U-Net to segment individual glomeruli and standardize their luminance appearance by the average luminance difference method, allowing the subsequent modules to focus more on the diseased glomerular region. Subsequently, in the Ada-CCF module, six confidence calibration methods are utilized for two main direct immunofluorescence images, IgG and C3, and the comprehensive calibration scores are obtained based on the adaptive weighted fusion of six confidence calibration methods to obtain more reliable confidence level, in which the adaptive weights are related with expected calibration error reductions. For the classification module, the weighted probability scores of IgG and C3 are jointly fed into the module to achieve the classification by random forest. Experimental results showed that Ada-CCFNet achieves the classification accuracy of 73.52%, surpassing the methods of using single IgG or C3 images and positive grade indicator with 8.24%, 8.94% and 22.76%, and outperforming the compared methods in the classification of membranous nephropathy.