Timezone: »
We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observed data using a feature-rich conditional random field (CRF). Then a reconstruction of the input is (re)generated, conditional on the latent structure, using a generative model which factorizes similarly to the CRF. The autoencoder formulation enables efficient exact inference without resorting to unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. Finally, we show competitive results with instantiations of the framework for two canonical tasks in natural language processing: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines.
Author Information
Waleed Ammar (CMU)
Chris Dyer (DeepMind)
Noah A Smith (Carnegie Mellon University)
Related Events (a corresponding poster, oral, or spotlight)
-
2014 Oral: Conditional Random Field Autoencoders for Unsupervised Structured Prediction »
Tue. Dec 9th 09:20 -- 09:40 PM Room Level 2, room 210
More from the Same Authors
-
2021 Poster: End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering »
Devendra Singh · Siva Reddy · Will Hamilton · Chris Dyer · Dani Yogatama -
2017 Poster: On-the-fly Operation Batching in Dynamic Computation Graphs »
Graham Neubig · Yoav Goldberg · Chris Dyer -
2014 Workshop: Representation and Learning Methods for Complex Outputs »
Richard Zemel · Dale Schuurmans · Kilian Q Weinberger · Yuhong Guo · Jia Deng · Francesco Dinuzzo · Hal Daumé III · Honglak Lee · Noah A Smith · Richard Sutton · Jiaqian YU · Vitaly Kuznetsov · Luke Vilnis · Hanchen Xiong · Calvin Murdock · Thomas Unterthiner · Jean-Francis Roy · Martin Renqiang Min · Hichem SAHBI · Fabio Massimo Zanzotto -
2014 Workshop: Modern Machine Learning and Natural Language Processing »
Ankur P Parikh · Avneesh Saluja · Chris Dyer · Eric Xing -
2010 Poster: Empirical Risk Minimization with Approximations of Probabilistic Grammars »
Shay Cohen · Noah A Smith -
2008 Poster: Unsupervised Bayesian Parameter Estimation for Probabilistic Grammars »
Shay Cohen · Kevin Gimpel · Noah A Smith