Temporal Behavior Trees for Reinforcement Learning: Specifications as Rewards, Curricula, and Diagnostics

Reinforcement learning from formal specifications often separates task definition, reward shaping, curriculum design, and policy diagnostics into different mathematical objects. This fragmentation introduces additional conversion steps and discards structure already present in the specification, such as sub-task boundaries, ordering constraints, and quantitative margins. We introduce TBTCL, a framework that uses Temporal Behaviour Trees (TBTs) as a unified interface for specification-guided reinforcement learning. A single TBT specifies the task, induces dense robustness-based rewards, defines a curriculum, and supports better understanding of learnt policies - grounded in the formal semantics of the specification language. By adapting TBT syntax and semantics for specification guided reinforcement learning, we provide a formalism that is more expressive than popular specification languages such LTL and inherently hierarchical. We present algorithms to automatically extract a curriculum DAG and derive dense, robustness-based rewards directly from TBT semantics, requiring no additional user inputs like region predicates or demonstrations. Furthermore, we utilize TBT trace segmentation to provide build understandable competence profiles and automatically adapt the reward toward bottlenecks. We evaluate TBTCL on six specifications across discrete and continuous domains. The results show that TBTCL matches dense STL shaping on simpler tasks and improves over baselines on reach-avoid specifications, while also exposing interpretable sub-task failures through the same formal object used for training. We show that a Temporal Behaviour Tree act as a specification, and a reward, and a curriculum and a diagnostic.