Timezone: »

A Multiplicative Model for Learning Distributed Text-Based Attribute Representations
Jamie Kiros · Richard Zemel · Russ Salakhutdinov

Tue Dec 09 04:00 PM -- 08:59 PM (PST) @ Level 2, room 210D #None

In this paper we propose a general framework for learning distributed representations of attributes: characteristics of text whose representations can be jointly learned with word embeddings. Attributes can correspond to a wide variety of concepts, such as document indicators (to learn sentence vectors), language indicators (to learn distributed language representations), meta-data and side information (such as the age, gender and industry of a blogger) or representations of authors. We describe a third-order model where word context and attribute vectors interact multiplicatively to predict the next word in a sequence. This leads to the notion of conditional word similarity: how meanings of words change when conditioned on different attributes. We perform several experimental tasks including sentiment classification, cross-lingual document classification, and blog authorship attribution. We also qualitatively evaluate conditional word neighbours and attribute-conditioned text generation.

Author Information

Jamie Kiros (Google Brain)
Richard Zemel (Vector Institute/University of Toronto)
Russ Salakhutdinov (Carnegie Mellon University)

More from the Same Authors