Category Boundary Re-decision by Component Labels to Improve Generation of Class Activation Map

Abstract Class Activation Maps (CAMs) can visualize the contribution of each pixel to a certain category and thus, can localize objects. However, in existing generation methods, CAMs are only highly responsive to small local features of objects and the highlighted region in CAMs are much smaller than the actual area of objects, which limits further usage of CAMs in downstream researches. For this problem, we think the reason lies in the image-level labels: the category boundary determined by the image-level label is rough and loose in high-dimensional feature space so that the supervision of the target classification is insufficient. Based on this conjecture, we propose to use a set of lower-level labels, called component labels, to refine the classification boundary and provide more supervision to force the classification network to learn more local features of the target. These component labels are designed based on WordNet hierarchy, which provides word relation and inclusion. In this way, one image-level label is transformed to multiple lower-level component labels and the category boundary is re-decided and enlarged by component boundaries. Besides, We adopt a graph convolution network (GCN) as the classifier, which construct a relation graph based on label conditional probability to improve the classification accuracy. After getting component features, we design a feature fusion module to merge local component features into the global feature of the target, and finally, generated CAM can be expanded and highlight the whole target. The experiment section shows the effectiveness of our component labels and we get better subjective and objective results than existing CAM generation methods. Besides, we show that our component label has good generalization on new categories.