A Priori Estimates of the Generalization Error for Autoencoders

Autoencoder is a machine learning model which aims for dimensionality reduction, by reconstructing its input through a bottleneck with lower dimension than the input. It is among the most popular models used in unsupervised learning and semi-supervised learning. In this paper, we build theoretical understanding about autoencoders. Specifically, assuming the existence of the underlying groundtruth encoder and decoder, we establish a priori estimates of the generalization error for autoencoders when an appropriately chosen regularization term is applied. The estimate is a priori in the sense that it only depend on some norms of the groundtruth encoder and decoder, but not the model parameters. The bound acheives nearly optimal rates with respect to the number of data and parameters. To our knowledge, this is the first try to build a priori estimates to unsupervised learning models. Numerical experiments show the tightness of the bounds.