Neural Bayesian Networks as Continual World Models for Reinforcement Learning

World models are central to sample-efficient Reinforcement Learning (RL), yet most are monolithic black boxes: trained once, frozen for deployment, and unable to answer the question "which part of the world changed?" typical in continual learning. Bayesian networks answer exactly that, splitting the dynamics into local mechanisms, one per variable, often arranged on a Directed Acyclic Graph (DAG). However, their classical formulation is a tabular one-value-at-a-time, and this has kept them out of modern world modeling: query computation is slow and continuous variables are poorly supported. We propose Neural Bayesian Networks (NBNs), whose mechanisms are differentiable functions handling discrete and continuous variables through GPU-batched modules and operations. NBNs make Bayesian network architectures finally usable for continual RL. In our preliminary experiments they re-track a drifting environment ${\sim}3\times$ more closely than retraining from scratch, and answer queries up to ${\sim}4000\times$ faster than a classical inference engine.