Class-Incremental Learning Network for Small Objects Enhancing of Semantic Segmentation in Aerial Imagery

Due to the differences in the feature distribution between classes, when the model learns in a continuous data stream, it will encounter catastrophic forgetting. The incremental learning methods have shown great potential to solve this problem. However, most existing methods based on task-incremental learning are difficult to adapt to characteristics of remote sensing scenes with few differences in appearance but large differences in features, which is not conducive to artificially distinguish task-ID. Thus, we propose a class-incremental learning network for small objects enhancing of semantic segmentation in aerial imagery. Specifically, considering the superior accurate of the binary classifier, we propose a Twin-Auxiliary model that adds an auxiliary binary classification task. Then, for expansion and contraction at the edge and small object confusion problems, we introduce a diversity distillation loss, using the results of binary-classifier to constrain the multi-class segmentation results, and strengthen the attention to the locations of the segmentation results that have changed. Finally, we design a conflict reduction mechanism for multi-head classifier to achieve single-head prediction for class-incremental learning. Experiments demonstrate that our method has good performance on the Vaihingen and Potsdam data sets by ISPRS, outperforming state-of-the-art incremental learning methods. The code will be available soon.