Freshness-Gated Imagination: Step-Level Trust for Latent World Models

Model-based reinforcement learning agents optimize policies by backpropagating through imagined latent trajectories. However, model reliability can vary substantially within a single rollout: some imagined steps produce useful gradients while others introduce harmful bias. Existing adaptive methods address this at either the rollout level (adaptive horizons, priority replay) or the per-step level (gradient truncation); their interaction remains unexplored. In this paper, Freshness-Gated Imagination (FGI), a lightweight trust layer for latent world models that combines rollout-level adaptation with per-step entropy-based soft gating at zero additional forward-pass cost, is proposed. Through systematic factorial ablation on Crafter (10 seeds), a superadditive synergy is uncovered: per-step gating alone harms performance (−6.2%), rollout-level adaptation alone is ineffective (+0.6%), while their combination achieves +14.8% improvement in terms of the Crafter geometric-mean score (Wilcoxon signed-rank p=0.032, Cohen's d=0.74). A bias-variance analysis shows that this synergy arises because rollout-level priority replay improves the calibration of the entropy signal on which the per-step gate depends. On DMC continuous control tasks, FGI preserves baseline performance, confirming no-harm in well-modeled environments. Code is available at https://anonymous.4open.science/r/FGI-89D9.