Timezone: »

 
Poster
Anytime-Competitive Reinforcement Learning with Policy Prior
Jianyi Yang · Pengfei Li · Tongxin Li · Adam Wierman · Shaolei Ren

Wed Dec 13 08:45 AM -- 10:45 AM (PST) @ Great Hall & Hall B1+B2 #2012
in Poster Session 3 »

This paper studies the problem of Anytime-Competitive Markov Decision Process (A-CMDP). Existing works on Constrained Markov Decision Processes (CMDPs) aim to optimize the expected reward while constraining the expected cost over random dynamics, but the cost in a specific episode can still be unsatisfactorily high. In contrast, the goal of A-CMDP is to optimize the expected reward while guaranteeing a bounded cost in each round of any episode against a policy prior. We propose a new algorithm, called Anytime-Competitive Reinforcement Learning (ACRL), which provably guarantees the anytime cost constraints. The regret analysis shows the policy asymptotically matches the optimal reward achievable under the anytime competitive constraints. Experiments on the application of carbon-intelligent computing verify the reward performance and cost constraint guarantee of ACRL.

Author Information

Jianyi Yang (University of California, Riverside)
Pengfei Li
Tongxin Li (The Chinese University of Hong Kong (Shenzhen))
Adam Wierman (Caltech)
Shaolei Ren (University of California, Riverside)

More from the Same Authors