Timezone: »

 
Poster
Multi-Game Decision Transformers
Kuang-Huei Lee · Ofir Nachum · Mengjiao (Sherry) Yang · Lisa Lee · Daniel Freeman · Sergio Guadarrama · Ian Fischer · Winnie Xu · Eric Jang · Henryk Michalewski · Igor Mordatch

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #107
in Poster Session 5 »

A longstanding goal of the field of AI is a method for learning a highly capable, generalist agent from diverse experience. In the subfields of vision and language, this was largely achieved by scaling up transformer-based models and training them on large, diverse datasets. Motivated by this progress, we investigate whether the same strategy can be used to produce generalist reinforcement learning agents. Specifically, we show that a single transformer-based model – with a single set of weights – trained purely offline can play a suite of up to 46 Atari games simultaneously at close-to-human performance. When trained and evaluated appropriately, we find that the same trends observed in language and vision hold, including scaling of performance with model size and rapid adaptation to new games via fine-tuning. We compare several approaches in this multi-game setting, such as online and offline RL methods and behavioral cloning, and find that our Multi-Game Decision Transformer models offer the best scalability and performance. We release the pre-trained models and code to encourage further research in this direction.

Keywords: Reinforcement Learning · Multi-Environment RL · Generalist Agent · Upside-Down RL · Decision Transformers
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Kuang-Huei Lee (Google Research)
Ofir Nachum (Google Brain)
Mengjiao (Sherry) Yang (Google Brain)
Lisa Lee (Google Brain)
Daniel Freeman (Google Research)
Sergio Guadarrama (Google Research)
Ian Fischer (Google)
Winnie Xu (University of Toronto / Stanford University)
Eric Jang (Google)
Henryk Michalewski (Google)
Igor Mordatch (Google)

More from the Same Authors