Skip to yearly menu bar Skip to main content


Poster

Saliency-based Sequential Image Attention with Multiset Prediction

Sean Welleck · Jialin Mao · Kyunghyun Cho · Zheng Zhang

Pacific Ballroom #129

Keywords: [ Recurrent Networks ] [ Deep Learning ] [ Attention Models ] [ Computer Vision ] [ Perception ] [ Hierarchical RL ]


Abstract:

Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.

Live content is unavailable. Log in and register to view live content