Timezone: »

 
Poster
PatchGame: Learning to Signal Mid-level Patches in Referential Games
Kamal Gupta · Gowthami Somepalli · Anubhav Gupta · Vinoj Yasanga Jayasundara Magalle Hewa · Matthias Zwicker · Abhinav Shrivastava

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

We study a referential game (a type of signaling game) where two agents communicate with each other via a discrete bottleneck to achieve a common goal. In our referential game, the goal of the speaker is to compose a message or a symbolic representation of "important" image patches, while the task for the listener is to match the speaker's message to a different view of the same image. We show that it is indeed possible for the two agents to develop a communication protocol without explicit or implicit supervision. We further investigate the developed protocol and show the applications in speeding up recent Vision Transformers by using only important patches, and as pre-training for downstream recognition tasks (e.g., classification).

Keywords: Machine Learning · Self-Supervised Learning · Transformers

Author Information

Kamal Gupta (University of Maryland College Park)
Gowthami Somepalli (University of Maryland, College Park)
Anubhav Gupta (University of Maryland, College Park)
Vinoj Yasanga Jayasundara Magalle Hewa (University of Maryland)
Matthias Zwicker (University of Maryland)
Abhinav Shrivastava (University of Maryland, College Park)

More from the Same Authors