Semi-supervised Training for Bottle-neck Feature based DNN-HMM Hybrid Systems

In this paper, we investigate semi-supervised training (SST) method in various state-of-the-art acoustic modeling techniques, using bottle-neck and corresponding tandem features. These techniques include subspace GMM, tanh-neuron deep neural network (DNN), and a generalized soft-maxout (p-norm) DNN. We demonstrate that SST may lead up to 2% Word Error Rate (WER) reduction using all these techniques in each case, and the best one comes from tandem feature based p-norm DNN system. In addition to recognition performance, effectiveness of the SST on keyword search performance is also investigated. Results on Actual Term Weighted Value (ATWV) are reported, with an analysis on lattice density. It is shown that SST may not necessarily increase ATWV due to the shrink of lattices size. Index Terms: Semi-supervised training, Bottle-neck features, Maxout Networks, keyword search, Speech recognition