Research on Visual Speech Recognition Based on Local Binary Pattern and Stacked Sparse Autoencoder

Lip feature extraction from human mouth image plays an essential role in visual speech recognition applications. This paper presents a lip feature extraction algorithm based on Local Binary Patterns (LBP) and Stacked Sparse Autoencoders (SSAE). First, LBP texture features are extracted from lip images. Then SSAE uses greedy unsupervised learning to extract high-level features. At last, we improve the performance of overall system by fine-tuning and input the extracted features into the Softmax classifier. Compared with traditional methods, the model proposed in this paper has higher classification accuracy and more applicability.