How Much Information is Needed for Accurate Kalman Filtering?

Kalman filtering provides a principled inference framework for linear Gaussian hidden Markov models, but it leaves open a complementary question of representation: what is the minimum amount of information about the observation history that must be retained to support accurate filtering? We show that this question leads naturally to an indirect rate-distortion problem, in which an encoder observes the history of noisy measurements while distortion is evaluated with respect to the latent state. Directly solving this problem is challenging, since the optimization ranges over arbitrary stochastic kernels from a time-varying observation history to an output representation, resulting in an infinite-dimensional formulation. We overcome this difficulty by proving that any feasible encoder can be replaced by a stochastic encoder acting only on the Kalman posterior mean, achieving the same mean-square error with no larger mutual information. The reduced problem admits a closed-form solution in the form of a linear Gaussian blurring mechanism determined by the spectral decomposition of the Riccati solution. This yields an explicit expression for the rate distortion function $R_t(\epsilon)$, showing that any representation with distortion at most $\epsilon$ must retain at least $R_t(\epsilon)$ nats of information. We further use this characterization to derive information-theoretic lower bounds on the population risk of sequential learning and on the prediction error of temporal GP. Numerical experiments corroborate the theoretical predictions.