Adaptive learning rate and limited error signal for multilayer perceptrons with n-th order cross-entropy error

Although an n-th order cross-entropy (nCE) error function resolves the incorrect saturation problem of conventional error backpropagation algorithm, the performance of multilayer perceptrons (MLPs) using the nCE function depends heavily on the order of nCE. In this paper, we propose an adaptive learning rate to make the MLP performance insensitive to the order of nCE. Additionally, we propose a method to limit error signal values at the output nodes for stable learning with an adaptive learning rate. The effectiveness of the proposed method is demonstrated in a handwritten digit recognition task.