Multimodal Kalman filtering

A difficult aspect of multimodal estimation is the possible discrepancy between the sampling rates and/or the noise levels of the considered data. Many algorithms cope with these dissimilarities empirically. In this paper, we propose a conceptual analysis of multimodality where we try to find the "optimal" way of combining modalities. More specifically, we consider a simple Kalman filtering framework where several noisy sensors with different sampling frequences and noise variances regularly observe a hidden state. We experimentally underline some relationships between the sampling grids and the asymptotic variance of the maximum a posteriori (MAP) estimator. However, the explicit study of the asymptotic variance seems intractable even in the simplest cases. We describe a promising idea to circumvent this difficulty: exploiting a stochastic measurement model for which one can more easily study the average asymptotic behavior.