A Short Text Classification Model Based on Cross-Layer Connected Gated Recurrent Unit Capsule Network

Text classification is an important task in natural language processing. In the past few years, some prominent methods have achieved great results in text classification, but there is still a lot of room for development in enhancing the expressive capabilities of text features. In addition, although deeper networks can better extract features, they are easy to produce gradient disappearance or gradient explosion problems. To solve these problems, this paper proposed a hybrid model based on cross-layer connected gated recurrent unit capsule network (CBiGRU_CapsNet). Firstly, the model fused word vectors trained by Word2Vec and GloVe to form the network input layer. In the stage of high-level semantic modeling of text, a cross-layer connected gated recurrent unit has been proposed, which can solve the problem of gradient disappearance or explosion, and strengthen the transfer between features of each layer. Furthermore, capsule network is applied to obtain the rich spatial position information in the deep high-level semantic representation, and the weight value of important features has been increasing through the core dynamic routing algorithm of the capsule network to improve the expression ability of the features, which is then inputted to the softmax layer to complete text classification task. The experimental results on several public datasets shown that CBiGRU_CapsNet outperforms the state-of-the-art models in terms of the accuracy for text classification tasks.