Outcome-Irrelevant and State-Independent Learning Mechanisms in Human Reinforcement Learning

Humans are adept at associating actions with rewards while considering their state in the environment. However, recent evidence suggests that humans also tend to learn in a state-independent manner, resulting in outcome-irrelevant learning. This study explores this phenomenon, where individuals form associations between actions and outcomes, even when these associations are known with high certainty to be due to random noise. Using a multi-armed bandit task, we demonstrate that humans tend to rely on a reward-dependent preference for spatial action features in making their decisions, despite knowing these are not predictive of outcomes. Through computational modeling and simulations, we show that this behavior, though sub-optimal in stable environments, may offer adaptive advantages in situations involving unexpected changes. The findings have implications for both understanding human cognition and improving machine learning algorithms by incorporating flexibility in state representations. Our results suggest that humans' predisposition for state-independent learning may reflect an evolved strategy to anticipate environmental variability.