Sparse Multilayer Autoencoders Based on the Small World Network

Unlike most of manifold learning algorithms, multilayer autoencoders construct a bi-directional mapping between the original data and the low dimensional representation. However, the time complexity of multilayer autoencoders is much higher since the neurons between the adjacent layers are fully connected in multilayer autoencoders. According to the small world network model, we build sparse multilayer autoencoders. To obtain a sparse network, the connections between the nodes in some layer and the nodes in their adjacent upper layer are firstly removed with a probability p. Then the connections are established between the nodes deleted the connections and the nodes in higher layer. The higher layer connected is selected with the method called limited random numbers. Experiments demonstrate that the reconstruction quality and the recognition accuracy from the sparse network are very near to those from the original network. However, the executing efficiency of the sparse network is improved.