Timezone: »

Talking to Strangers: Zero-Shot Emergent Communication
Marie Ossenkopf · Angelos Filos · Abhinav Gupta · Michael Noukhovitch · Angeliki Lazaridou · Jakob Foerster · Kalesha Bullard · Rahma Chaabouni · Eugene Kharitonov · Roberto Dessì

Sat Dec 12 07:00 AM -- 02:10 PM (PST) @ None
Event URL: https://sites.google.com/view/emecom2020/home »

Communication is one of the most impressive human abilities but historically it has been studied in machine learning mainly on confined datasets of natural language. Thanks to deep RL, emergent communication can now be studied in complex multi-agent scenarios.

Three previous successful workshops (2017-2019) have gathered the community to discuss how, when, and to what end communication emerges, producing research later published at top ML venues (e.g., ICLR, ICML, AAAI). However, many approaches to studying emergent communication rely on extensive amounts of shared training time. Our question is: Can we do that faster?

Humans interact with strangers on a daily basis. They possess a basic shared protocol, but a huge partition is nevertheless defined by the context. Humans are capable of adapting their shared protocol to ever new situations and general AI would need this capability too.

We want to explore the possibilities for artificial agents of evolving ad hoc communication spontaneously, by interacting with strangers. Since humans excel on this task, we want to start by having the participants of the workshop take the role of their agents and develop their own bots for an interactive game. This will illuminate the necessities of zero-shot communication learning in a practical way and form a base of understanding to build algorithms upon. The participants will be split into groups and will have one hour to develop their bots. Then, a round-robin tournament will follow, where bots will play an iterated zero-shot communication game with other teams’ bots.

This interactive approach is especially aimed at the defined NeurIPS workshop goals to clarify questions for a subfield or application area and to crystallize common problems. It condenses our experience from former workshops on how workshop design can facilitate cooperation and progress in the field. We also believe that this will maximize the interactions and exchange of ideas between our community.

Sat 7:00 a.m. - 7:08 a.m.

Welcome! The whole workshop will be held in gather.town. Please drop in any time.

Sat 7:08 a.m. - 7:10 a.m.
Intro to Ruth Byrne (Intro)
Sat 7:10 a.m. - 7:40 a.m.

I consider the sorts of models people construct to reason about other people’s thoughts based on several strands of evidence from cognitive science experiments. The first is from studies of how people think about decisions to cooperate or not with another person in various sorts of social interactions in which they must weigh their own self-interest against the common interest. I discuss results from well-known games such as the Prisoner’s dilemma, such as the finding that people who took part in the game imagine the outcome would have been different if a different decision had been made by the other player, not themselves. The second strand of evidence comes from studies of how people think about other people’s false beliefs. I discuss reasoning in change-of-intentions tasks, in which an observer who witnesses an actor carrying out an action forms a false belief about the reason. People appear to develop the skills to make inferences about other people’s false beliefs by creating counterfactual alternatives to reality about how things would have been. I consider how people construct models of other people’s thoughts, and consider the implications for how AI agents could construct models of other AI agents.

Ruth Byrne
Sat 7:40 a.m. - 7:50 a.m.

Explanation of the Game Rules for the live coding session

Sat 7:50 a.m. - 8:00 a.m.

Find your group

Sat 8:00 a.m. - 9:00 a.m.

Time for your team to solve the game. Session will be held in gather.town

Sat 9:00 a.m. - 9:45 a.m.

Will be held in gather.town

Sat 9:45 a.m. - 10:45 a.m.

The matches between the teams will be shown live during the lunch break. Optional attendance.

Sat 10:45 a.m. - 11:00 a.m.

Short presentation of which strategies seemed to work well. No thorough analysis yet.

Sat 11:00 a.m. - 11:02 a.m.
Intro to Michael Bowling (Intro)
Sat 11:02 a.m. - 11:37 a.m.

I will look at some of the often unstated principles common in multiagent learning research (and emergent communication work too), suggesting that they may be responsible for holding us back. In response, I will offer an alternative set of principles, which leads to the view of hindsight rationality, with connections to online learning and correlated equilibria. I will then describe some recent technical work understanding how we can build increasingly more powerful algorithms for hindsight rationality in sequential decision-making settings.

Speaker's Bio: Michael Bowling is a professor at the University of Alberta, a Fellow of the Alberta Machine Intelligence Institute, and a senior scientist in DeepMind. Michael led the Computer Poker Research Group, which built some of the best poker playing artificial intelligence programs in the world, including being the first to beat professional players at both limit and no-limit variants of the game. He also was behind the use of Atari 2600 games to evaluate the general competency of reinforcement learning algorithms and popularized research in Hanabi, a game that illustrates emergent communication and theory of mind.

Michael Bowling
Sat 11:37 a.m. - 11:40 a.m.
Intro to Richard Futrell (Intro)
Sat 11:40 a.m. - 12:25 p.m.

I claim that human languages can be modeled as information-theoretic codes, that is, systems that maximize information transfer under certain constraints. I argue that the relevant constraints for human language are those involving the cognitive resources used during language production and comprehension and in particular working memory resources. Viewing human language in this way, it is possible to derive and test new quantitative predictions about the statistical, syntactic, and morphemic structure of human languages. I start by reviewing some of the many ways that natural languages differ from optimal codes as studied in information theory. I argue that one distinguishing characteristic of human languages, as opposed to other natural and artificial codes, is a property I call information locality: information about particular aspects of meaning is localized in time within a linguistic utterance. I give evidence for information locality at multiple levels of linguistic structure, including the structure of words and the order of words in sentences. Next, I state a theorem showing that information locality is an inevitable property of any communication system where the encoder and/or decoder are operating under memory constraints. The theorem yields a new, fully formal, and quantifiable definition of information locality, which leads to new predictions about word order and the structure of words across languages. I test these predictions in broad corpus studies of word order in over 50 languages, and in case studies of the order of morphemes within words in two languages.

Richard Futrell
Sat 12:15 p.m. - 1:00 p.m.

Will be held in gather.town

Sat 1:00 p.m. - 1:15 p.m.

We will chat about the game and our insights into the problem of zero-shot coordination after playing it.

Sat 1:15 p.m. - 2:00 p.m.

In this panel, we want to discuss the lessons we learned in this workshop about the possibilities and necessities of learning to cooperate and communicate with strangers.

Guests: Angeliki Lazaridou Jakob Foerster Ruth Byrne Michael Bowling Richard Futrell

Sat 2:00 p.m. - 2:10 p.m.
Closing Remarks (Outro)
Sat 2:10 p.m. - 3:00 p.m.

Come together to discuss the workshop in our cozy gather.town bar

Author Information

Marie Ossenkopf (University of Kassel)

Marie Ossenkopf (Uni Kassel) is a PhD student at the University of Kassel in the Distributed Systems Group supervised by Kurt Geihs. She is currently writing her thesis on architectural necessities of emergent communication, especially for multilateral agreements. She received her MSc in Automation Engineering from RWTH Aachen University in 2016 and organizes international youth exchange workshops since 2017. She was a co-organizer of the Emergent Communication workshop at NeurIPS 2019. When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning. Ossenkopf, Marie; Jorgensen, Mackenzie; Geihs, Kurt. In: Cybernetics and Systems vol. 50, Taylor & Francis (2019), Nr. 8, pp. 672-692 Hierarchical Multi-Agent Deep Reinforcement Learning to Develop Long-Term Coordination. Ossenkopf, Marie, Mackenzie Jorgensen, and Kurt Geihs. SAC 2019.

Angelos Filos (University of Oxford)
Abhinav Gupta (Mila)
Michael Noukhovitch (Mila (Université de Montréal))

Master's student at MILA supervised by Aaron Courville and co-supervised by Yoshua Bengio

Angeliki Lazaridou (DeepMind)
Jakob Foerster (Facebook AI Research)

Jakob Foerster received a CIFAR AI chair in 2019 and is starting as an Assistant Professor at the University of Toronto and the Vector Institute in the academic year 20/21. During his PhD at the University of Oxford, he helped bring deep multi-agent reinforcement learning to the forefront of AI research and interned at Google Brain, OpenAI, and DeepMind. He has since been working as a research scientist at Facebook AI Research in California, where he will continue advancing the field up to his move to Toronto. He was the lead organizer of the first Emergent Communication (EmeCom) workshop at NeurIPS in 2017, which he has helped organize ever since.

Kalesha Bullard (Facebook AI Research)
Rahma Chaabouni (FAIR/ENS)
Eugene Kharitonov (Facebook AI Research)
Roberto Dessì (Facebook AI / Universitat Pompeu Fabra)

More from the Same Authors