Towards enhancing stacked extreme learning machine with sparse autoencoder by correntropy

Abstract The stacked extreme learning machine (S-ELM) is an advanced framework of deep learning. It passes the ‘reduced’ outputs of the previous layer to the current layer, instead of directly propagating the previous outputs to the next layer in traditional deep learning. The S-ELM could address some large and complex data problems with a high accuracy and a relatively low requirement for memory. However, there is still room for improvement of the time complexity as well as robustness while using S-ELM. In this article, we propose an enhanced S-ELM by replacing the original principle component analysis (PCA) technique used in this algorithm with the correntropy-optimized temporal PCA (CTPCA), which is robust for outliers rejection and significantly improves the training speed. Then, the CTPCA-based S-ELM performs better than S-ELM in both accuracy and learning speed, when dealing with dataset disturbed by outliers. Furthermore, after integrating the extreme learning machine (ELM) sparse autoencoder (AE) method into the CTPCA-based S-ELM, the learning accuracy is further improved while spending a little more training time. Meanwhile, the sparser and more compact feature information are available by using the ELM sparse AE with more computational efforts. The simulation results on some benchmark datasets verify the effectiveness of our proposed methods.