Sequence-dropout Block for Reducing Overfitting Problem in Image Classification

Overfitting is a common problem for computer vision applications It is a problem that when training convolution neural networks and is caused by lack of training data or network complexity. The novel sequence-dropout (SD) method is proposed in this paper to alleviate the problem of overfitting when training networks. The SD method works by dropping out units (channels of feature) from the network in a sequence, replacing the traditional operation of random omitting. Sophisticated aggregation strategies are used to obtain the global information of feature channels, and channel-wise weights are produced by gating mechanism. The SD method then selectively drops out the feature channels according to the channel-wise weights that represent the importance degree of each channel. The proposed SD block can be plugged into state-of-the-art backbone CNN models such as VGGNet and ResNet. The SD block is then evaluated on these models, demonstrating consistent performance gains over the baseline model on widely-used benchmark image classification datasets including MNIST, CIFAR-10, CIFAR-100, and ImageNet2012. Experimental results demonstrate that the superior performance of the SD block compared to other modern methods.