Timezone: »
Recurrent neural networks (RNNs) such as long short-term memory and gated recurrent units are pivotal building blocks across a broad spectrum of sequence modeling problems. This paper proposes a recurrently controlled recurrent network (RCRN) for expressive and powerful sequence encoding. More concretely, the key idea behind our approach is to learn the recurrent gating functions using recurrent networks. Our architecture is split into two components - a controller cell and a listener cell whereby the recurrent controller actively influences the compositionality of the listener cell. We conduct extensive experiments on a myriad of tasks in the NLP domain such as sentiment analysis (SST, IMDb, Amazon reviews, etc.), question classification (TREC), entailment classification (SNLI, SciTail), answer selection (WikiQA, TrecQA) and reading comprehension (NarrativeQA). Across all 26 datasets, our results demonstrate that RCRN not only consistently outperforms BiLSTMs but also stacked BiLSTMs, suggesting that our controller architecture might be a suitable replacement for the widely adopted stacked architecture. Additionally, RCRN achieves state-of-the-art results on several well-established datasets.
Author Information
Yi Tay (Nanyang Technological University)
Anh Tuan Luu (Institute for Infocomm Research)
Siu Cheung Hui (Nanyang Technological University)
More from the Same Authors
-
2021 Poster: Self-Instantiated Recurrent Units with Dynamic Soft Recursion »
Aston Zhang · Yi Tay · Yikang Shen · Alvin Chan · SHUAI ZHANG -
2019 Poster: Compositional De-Attention Networks »
Yi Tay · Anh Tuan Luu · Aston Zhang · Shuohang Wang · Siu Cheung Hui -
2019 Poster: Quaternion Knowledge Graph Embeddings »
SHUAI ZHANG · Yi Tay · Lina Yao · Qi Liu -
2018 Poster: Densely Connected Attention Propagation for Reading Comprehension »
Yi Tay · Anh Tuan Luu · Siu Cheung Hui · Jian Su