Control Under the Wrong Model Is Better Than Under the Correct One

Building an optimal controller requires solving two coupled problems: (1) computing beliefs about the state by filtering observations, and (2) designing a control signal based on those beliefs to minimize a cost function. Ideally, these two problems can be solved independently: first, a filter is computed to estimate the state; then, a controller is built on top of that estimate. When exact inference is tractable, this approach is optimal and applies to the prominent Linear-Quadratic-Gaussian stochastic control problem. However, perfect inference is generally intractable, and filtering and control are closely intertwined problems. The question, then, is whether the commonly used first-filter-then-control approach remains optimal, and whether optimal control strategies should generally rely on internal representations that mirror the dynamics of the external world. We show that this is not the case: even in a simple setting with linear dynamics, multiplicative noise, and internal noise, the optimal linear controller is characterized by internal forward dynamics that do not match the forward dynamics of the external state. Instead, the optimal controller relies on internal representations that mix estimation and control, and this mismatch becomes more pronounced as internal noise increases. Our work challenges standard approaches that prioritize external world modeling over control.