Timezone: »

 
Poster
Human-Adversarial Visual Question Answering
Sasha Sheng · Amanpreet Singh · Vedanuj Goswami · Jose Magana · Tristan Thrush · Wojciech Galuba · Devi Parikh · Douwe Kiela

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual
in Poster Session 1 »

Performance on the most commonly used Visual Question Answering dataset (VQA v2) is starting to approach human accuracy. However, in interacting with state-of-the-art VQA models, it is clear that the problem is far from being solved. In order to stress test VQA models, we benchmark them against human-adversarial examples. Human subjects interact with a state-of-the-art VQA model, and for each image in the dataset, attempt to find a question where the model’s predicted answer is incorrect. We find that a wide range of state-of-the-art models perform poorly when evaluated on these examples. We conduct an extensive analysis of the collected adversarial examples and provide guidance on future research directions. We hope that this Adversarial VQA (AdVQA) benchmark can help drive progress in the field and advance the state of the art.

Keywords: Adversarial Robustness and Security · Vision

Author Information

Sasha Sheng
Amanpreet Singh (Facebook)
Vedanuj Goswami (Facebook)

Research engineer in computer vision and machine learning.

Jose Magana (Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM))
Tristan Thrush (Facebook)
Wojciech Galuba (Meta AI)
Devi Parikh (Georgia Tech / Facebook AI Research (FAIR))
Douwe Kiela (Facebook AI Research)

More from the Same Authors