Improving optimal control and estimation for realistic noise models of the sensorimotor system

Sustaining perception-action loops is a fundamental brain computation, which can be effectively described by stochastic optimal control theory through optimality principles. When accounting for a realistic noise model of the sensorimotor system, including multiplicative noise in feedback and motor output as well as internal noise in estimation, the mathematical complexity of the problem increases significantly. The standard algorithm in use is the one introduced in the seminal study in (Todorov, 2005). We identify a limitation in the original derivation stemming from the assumption of unbiased estimation and propose an efficient gradient descent optimization that minimizes the cost-to-go, enforcing only the linearity of the control law. To achieve the optimal solution, we propagate sufficient statistics in closed form to evaluate the expected cost, then minimize this cost with respect to the filter and control gains. Our results demonstrate that this approach achieves a lower overall cost than state-of-the-art solutions when internal noise is considered. Deriving the optimal control law in these cases is essential for addressing problems like inverse control inference.