Timezone: »

 
Poster
Emergent Discrete Communication in Semantic Spaces
Mycal Tucker · Huao Li · Siddharth Agrawal · Dana Hughes · Katia Sycara · Michael Lewis · Julie A Shah

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @
in Poster Session 4 »

Neural agents trained in reinforcement learning settings can learn to communicate among themselves via discrete tokens, accomplishing as a team what agents would be unable to do alone. However, the current standard of using one-hot vectors as discrete communication tokens prevents agents from acquiring more desirable aspects of communication such as zero-shot understanding. Inspired by word embedding techniques from natural language processing, we propose neural agent architectures that enables them to communicate via discrete tokens derived from a learned, continuous space. We show in a decision theoretic framework that our technique optimizes communication over a wide range of scenarios, whereas one-hot tokens are only optimal under restrictive assumptions. In self-play experiments, we validate that our trained agents learn to cluster tokens in semantically-meaningful ways, allowing them communicate in noisy environments where other techniques fail. Lastly, we demonstrate both that agents using our method can effectively respond to novel human communication and that humans can understand unlabeled emergent agent communication, outperforming the use of one-hot communication.

Keywords: Reinforcement Learning and Planning · Language

Author Information

Mycal Tucker (Massachusetts Institute of Technology)
Huao Li (University of Pittsburgh)
Siddharth Agrawal (Carnegie Mellon University)
Dana Hughes (Carnegie Mellon University)
Katia Sycara
Michael Lewis (University of Pittsburgh)
Julie A Shah (MIT)

More from the Same Authors