Gradient-Aligned convolution neural network

Abstract Although Convolution Neural Networks (CNN) have achieved great success in many applications of computer vision in recent years, rotation invariance is still a difficult problem for CNN. Especially for some images, the content can appear in the image at any angle of rotation, such as medical images, microscopic images, remote sensing images and astronomical images. In this paper, we propose a novel convolution operation, called Gradient-Aligned Convolution (GAConv), which can help CNN achieve rotation invariance by replacing vanilla convolutions in CNN. GAConv is implemented with a prior pixel-level gradient alignment operation before regular convolution. With GAConv, Gradient-Aligned CNN (GACNN) can achieve rotation invariance without any data augmentation, feature-map augmentation, and filter enrichment. In GACNN, rotation invariance does not learn from the training set, but bases on the network model. Different from the vanilla CNN, GACNN will output invariant results for all rotated versions of an object, no matter whether the network is trained or not. This means that we only need to train the network with one canonical version of the object and all other rotated versions of this object should be recognized with the same accuracy. Classification experiments have been conducted to evaluate GACNN compared with some rotation invariant approaches. GACNN achieved the best results on the 360 ∘ rotated test set of MNIST-rotation, Plankton-sub-rotation, and Galaxy Zoo 2.