Inverse reinforcement learning for naturalistic behavior with evolving objectives

Naturalistic behavior is becoming increasingly central to neuroscience, and new tools can now measure it at scale. These tools describe an animal's movements, but not the objectives that organize its actions over time. Inverse reinforcement learning (IRL) recovers such objectives as reward functions that make observed behavior likely. However, existing approaches for animal behavior are limited to small, discrete state spaces or task-specific reward forms. We present a maximum-entropy IRL approach spanning continuous-state, freely moving behavior, and structured tasks with discrete states. An evolving internal variable, either latent or constructed from experience, modulates the reward, allowing the objectives to flexibly vary over time. Fit to mice hunting crickets, the model recovers three switching goals: search, pursuit, and home. Fit to mice learning a labyrinth task, it recovers how their internal reward converges to the task objective. Across both settings, the model predicts behavior better than movement-sequence baselines, and generates extended trajectories resembling observed behavior.