Early Neoplasia Identification in Barrett’s Esophagus via Attentive Hierarchical Aggregation and Self-Distillation

Abstract Automatic surveillance of early neoplasia in Barrett’s esophagus (BE) is of great significance for improving the survival rate of esophageal cancer. It remains, however, a challenging task due to (1) the large variation of early neoplasia, (2) the existence of hard mimics, (3) the complicated anatomical and lighting environment in endoscopic images, and (4) the intrinsic real-time requirement of this application. We propose a novel end-to-end network equipped with an attentive hierarchical aggregation module and a self-distillation mechanism to comprehensively address these challenges. The hierarchical aggregation module is proposed to capture the complementariness of adjacent layers and hence strengthen the representation capability of each aggregated feature. Meanwhile, an attention mask is developed to selectively integrate the logits of each feature, which not only improves the prediction accuracy but also enhances the prediction interpretability. Furthermore, an efficient self-distillation mechanism is implemented based on a teacher-student architecture, where the student aims at capturing abstract high-level features while the teacher is applied to bring more low-level semantic details to calibrate the classification results. The proposed techniques are effective yet lightweight, improving the classification performance without sacrificing time performance, and thus achieving real-time inference. We extensively evaluate the proposed method on the MICCAI EndoVis Challenge Dataset. Experimental results demonstrate the proposed method can achieve competitive accuracy with a much faster speed than state-of-the-arts.