Weakly supervised image classification and pointwise localization with graph convolutional networks

Abstract In computer vision, the research community has been looking to how to benefit from weakly supervised learning that utilizes easily obtained image-level labels to train neural network models. The existing deep convolutional neural networks for weakly supervised learning, however, generally do not fully exploit the label dependencies in an image. To make full use of this information, in this paper, we propose a new framework for weakly supervised learning of deep convolutional neural networks, introducing graph convolutional networks to capture the semantic label co-occurrence in an image. Moreover, we propose a novel initialization method for label embedding in graph convolutional networks, which enables a smoother optimization for interrelationships learning. Extensive experiments and comparisons on four public benchmark datasets (PASCAL VOC 2007, PASCAL VOC 2012, Microsoft COCO, and NUS-WIDE) show the superior performance of our approach in both image classification and weakly supervised pointwise object localization. These results lead us to conclude that the label dependencies in the input image can provide valuable evidence for learning strongly localized features.