A New Regularized Recursive Dynamic Factor Analysis With Variable Forgetting Factor and Subspace Dimension for Wireless Sensor Networks With Missing Data

The dynamic factor analysis (DFA) is an effective method for reducing the dimension of multivariate time series measurements in wireless sensor networks (WSNs) for prediction, monitoring, and anomaly detection. In large-scale systems, it is crucial to be able to track the time-varying loadings (or subspace) and the underlying factor signals, identify their dimensions, and impute missing or outlier samples, which can arise from transmission loss, hardware failure, and other factors. This article proposes a new regularized recursive DFA algorithm with a variable forgetting factor (FF) and subspace dimension, which is computationally efficient and capable of missing data imputation. A novel multiple deflation (MD) and rank-one-modification-based approach is proposed to recursively estimate the eigenvalues and associated eigenvectors from the progressively extracted subspace via MD. This allows us to update efficiently the subspace dimension and loading (eigen) vectors. By modeling the factor signals as independent univariate autoregressive (AR) processes, one can utilize the forecast value for outlier detection and missing samples’ imputation. Furthermore, variable FF and regularization schemes are proposed to improve the tracking capability and reduce the variance of the proposed recursive DFA algorithm. Experimental results using real WSN datasets show that the proposed algorithm achieves better tracking performance and accuracy than other conventional approaches, which may serve as a useful tool for prediction, monitoring, and anomaly detection in WSNs.