A Novel Deep Learning Approach: Stacked Evolutionary Auto-encoder

Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification prob- lems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.