Deep belief network-based AR model for nonlinear time series forecasting

Abstract For a class of nonlinear time series whose dynamic behavior smoothly changes with the system state, a state-dependent auto-regressive (SD-AR) model is proposed to characterize the nonlinear time series. A set of deep belief networks (DBNs) is used to build the state-dependent functional coefficients of the SD-AR model, and the proposed model is called DBN-AR model, which combines the advantage of DBN in function approximation and the merit of SD-AR model in nonlinear dynamics description. The DBN-AR model is driven by the state signal changing with time. Based on the least squares solution with minimum norm and the pseudo inverse matrix approach, the initial target values of the DBNs are determined in pre-training stage. In fine tuning stage, all parameters of DBN-AR model is finally tuned by the back propagation (BP) algorithm designed for fine-tuning of DBN-AR model. Through experiment and comparative study on the sunspot data, the electricity load demand data sets from Australian Energy Market Operator (AEMO), the weekly British Pound/US dollar (GBP/USD) exchange rate data and the daily electricity generation data of the Three Gorges dam right bank power station, it is shown that the DBN-AR model is superior to some existing models or methods in prediction accuracy.