R-ELMNet: Regularized extreme learning machine network

Abstract Principal component analysis network (PCANet), as an unsupervised shallow network, demonstrates noticeable effectiveness on datasets of various volumes. It carries a two-layer convolution with PCA as filter learning method, followed by a block-wise histogram post-processing stage. Following the structure of PCANet, extreme learning machine auto-encoder (ELM-AE) variants are employed to replace the PCA’s role, which come from extreme learning machine network (ELMNet) and hierarchical ELMNet. ELMNet emphasizes the importance of orthogonal projection while overlooking non-linearity. The latter introduces complex pre-processing to overcome drawback of non-linear ELM-AE. In this paper, we analyze intrinsic characteristics of ELM-AE variants and accordingly propose a regularized ELM-AE, which combines non-linearity learning capability and approximately orthogonal projection. Experiments on image classification show the effectiveness compared to supervised convolutional neural networks and related shallow networks on unsupervised feature learning.