Skip to yearly menu bar Skip to main content

Workshop: Deep Reinforcement Learning Workshop

Distributional deep Q-learning with CVaR regression

Mastane Achab · REDA ALAMI · YASSER ABDELAZIZ DAHOU DJILALI · Kirill Fedyanin · Eric Moulines · Maxim Panov


Reinforcement learning (RL) allows an agent interacting sequentially with an environment to maximize its long-term return, in expectation. In distributional RL (DRL), the agent is also interested in the probability distribution of the return, not just its expected value. This so-called distributional perspective of RL has led to new algorithms with improved empirical performance. In this paper, we recall the atomic DRL (ADRL) framework based on atomic distributions projected via the Wasserstein-2 metric. Then, we derive two new deep ADRL algorithms, namely SAD-Q-learning and MAD-Q-learning (both for the control task). Numerical experiments on various environments compare our approach against existing deep (distributional) RL methods.

Chat is not available.