Riemannian Admissibility Flow for Offline-to-Online Safe Reinforcement Learning

Offline-to-Online (O2O) reinforcement learning (RL) is essential for policy adaptation but frequently suffers from initialization collapse due to evaluation and improvement mismatches across the learning transition. In safety-critical domains, safety-constrained model-free O2O RL without offline data retention remains significantly under-explored. Existing O2O fine-tuning methods, typically relying on policy regularization or value recalibration within Euclidean spaces, lack the structural capacity to strictly sequester policies from hazardous regions. To facilitate a geometric paradigm for safe policy fine-tuning, we propose Riemannian Admissibility Flow (RAF), which lifts safety constraints into a geometric framework by reformulating policy optimization as probability transport on anisotropic Riemannian manifolds. By fusing safety and uncertainty into the metric tensor, RAF transforms extrinsic scalar penalties into intrinsic topological barriers with directional gating, enabling target-free geodesic flow matching for hazard circumvention without offline data retention. Evaluation on representative safe O2O benchmarks shows that RAF achieves competitive performance in constraint satisfaction and task execution while effectively mitigating initialization collapse.