A Framework for Robust Assimilation of Potentially Malign Third-Party Data, and its Statistical Meaning

This article presents a model-based method for fusing data from multiple sensors with a hypothesis-test-based component for rejecting potentially faulty or otherwise malign data. Our framework is based on an extension of the classic particle filter algorithm for real-time state estimation of uncertain systems with nonlinear dynamics with partial and noisy observations. This extension, based on classical statistical theories, utilizes statistical tests against the system's observation model. We discuss the application of the two major statistical testing frameworks, Fisher significance testing and Neyman?Pearson hypothesis testing, to the Monte Carlo and sensor fusion settings. The Monte Carlo Neyman?Pearson test we develop is useful when one has a reliable model of faulty data, while the Fisher method is applicable when one does not have a model of faults, which may occur when dealing with third-party data, such as the Global Navigation Satellite System (GNSS) data of transportation system users. These statistical tests can be combined with a particle filter to obtain a Monte Carlo state estimation scheme that is robust to faulty or outlier data. We present a synthetic freeway-traffic state estimation problem where the filters are able to reject simulated faulty GNSS measurements. The fault-model-free Fisher filter underperforms the Neyman?Pearson filter when the latter has an accurate fault model but outperforms it when the assumed fault model is incorrect.