Automatic diagnosis for aggressive posterior retinopathy of prematurity via deep attentive convolutional neural network

Abstract Retinopathy of Prematurity (ROP) is one of the most common childhood blindness in premature infants, and Aggressive Posterior Retinopathy of Prematurity (AP-ROP) is a special type of ROP, which will rapidly evolve into the fifth stage of ROP, even blindness if not detecting and treating in time. However, the incidence of AP-ROP is low and the symptoms are rare, which can easily cause misdiagnosis. Early diagnosis of AP-ROP using computer-assisted methods can help clinician make decision accurately. This paper proposes an automated AP-ROP diagnosis system, which divides a complex task into two tasks. Specifically, the two independent networks are utilized to automatically diagnose AP-ROP in an end-to-end way. Network 1 is responsible for identifying whether there is ROP in fundus images, and Network 2 is devised to divide ROP fundus images into AP-ROP and Regular ROP. A channel attention module is added to the two networks to improve the feature representation. In addition, a bilinear pooling module is leveraged to obtain complementary information between layers. The transfer learning mechanism is used to transfer the parameters of Network 1 to Network 2 to improve the classification performance of Network 2. The network is trained with 6867 fundus images that are collected from 2012 to 2018, in which 1709 fundus images are used as a validation set to assist training the network model, and the system performance is evaluated with another 3654 images. The accuracy, recall and specificity of Network 1 to identify the presence of ROP are 98.88%, 98.42% and 99.28%, respectively. The accuracy, sensitivity and specificity for diagnosing AP-ROP using Network 2 are 93.09%, 87.70% and 95.40%, respectively. After combining Network 1 and Network 2, the joint diagnosis accuracy of the test set is 95.81%, which is significantly better than using single three-class networks (95.16% for ResNet50 and 94.53% for ResNet101). We use a further 100 fundus images in 2019 for clinical evaluation of the automated AP-ROP diagnosis system. In the clinical environment, the diagnosis accuracy of the system is 100%, which is better than 90.00%, 85.00%, and 70.00% from the three ophthalmologists, respectively. Our system in this study achieves good performance, and the combined method is better than the three-classification method. Through the final performance evaluation of the devised framework, our classification results are close to those of prevalence ophthalmologists, which indicate that our method can effectively assist ophthalmologists to implement the diagnosis of AP-ROP.