Concept-based Explanation for Fine-grained Images and Its Application in Infectious Keratitis Classification

Interpretability has become an essential topic as deep learning is widely applied in professional fields (e.g., medical image processing)where high level of accountability is required. Existing methods for explanation mainly focus on computing the importance of low level pixels or segments, rather than the high-level concepts. Concepts are of paramount importance for human to understand and make decisions, especially for those fine-grained tasks. In this paper, we focus on the real application problem of classification of infectious keratitis and propose a visual concept mining (VCM) method to explain the fine-grained infectious keratitis images. Based on our discovered explainable visual concepts, we further propose a visual concept enhanced framework for infectious keratitis classification. Extensive empirical experiments demonstrate that (i) our discovered visual concepts are highly coherent with the physicians? understanding and interpretation, and (ii) our visual concept enhanced model achieves significant improvement on the performance of infectious keratitis classification.