Generative Flow Sparse Networks

Generative Flow Networks (GFlowNets) were introduced as generative models for sampling from discrete objects. Recent works have established a connection between GFlowNets and KL-regularized reinforcement learning (RL), showing the two frameworks are equivalent under appropriate formulations. In this work, we ask whether Tsallis-regularized RL admits an analogous interpretation and can likewise serve as a generative model for discrete objects. We first formulate a general framework of regularized Markov decision processes (MDPs) on directed acyclic graphs (DAGs) and show that GFlowNets arise as a particular instance of this framework. We then specialize the regularizer to Tsallis divergence and derive a new generative model based on the new formulation. We study the theoretical properties of the resulting model and compare its empirical performance with trajectory-balance GFlowNets on HyperGrid and Bag environments.