Timezone: »
A common problem in knowledge representation and related fields is reasoning over a large joint knowledge graph, represented as triples of a relation between two entities. The goal of this paper is to develop a more powerful neural network model suitable for inference over these relationships. Previous models suffer from weak interaction between entities or simple linear projection of the vector space. We address these problems by introducing a neural tensor network (NTN) model which allow the entities and relations to interact multiplicatively. Additionally, we observe that such knowledge base models can be further improved by representing each entity as the average of vectors for the words in the entity name, giving an additional dimension of similarity by which entities can share statistical strength. We assess the model by considering the problem of predicting additional true relations between entities given a partial knowledge base. Our model outperforms previous models and can classify unseen relationships in WordNet and FreeBase with an accuracy of 86.2% and 90.0%, respectively.
Author Information
Richard Socher (MetaMind)
Danqi Chen (Stanford University)
Christopher D Manning (Stanford University)
Andrew Y Ng (DeepLearning.AI)
More from the Same Authors
-
2021 : Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning »
Kaylee Burns · Christopher D Manning · Li Fei-Fei -
2022 Poster: Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset »
Peter Henderson · Mark Krass · Lucia Zheng · Neel Guha · Christopher D Manning · Dan Jurafsky · Daniel Ho -
2023 : Teaching language models with canonical examples »
John Hewitt · Sarah Chen · Percy Liang · Christopher D Manning -
2023 : Language Model Detectors Are Easily Optimized Against »
Charlotte Nicks · Eric Mitchell · Rafael Rafailov · Archit Sharma · Christopher D Manning · Chelsea Finn · Stefano Ermon -
2023 : An Emulator for Fine-tuning Large Language Models using Small Language Models »
Eric Mitchell · Rafael Rafailov · Archit Sharma · Chelsea Finn · Christopher D Manning -
2023 : Fine-tuning Language Models for Factuality »
Katherine Tian · Eric Mitchell · Huaxiu Yao · Christopher D Manning · Chelsea Finn -
2023 : Teaching language models with canonical examples »
John Hewitt · Sarah Chen · Percy Liang · Christopher D Manning -
2023 Poster: Direct Preference Optimization: Your Language Model is Secretly a Reward Model »
Rafael Rafailov · Archit Sharma · Eric Mitchell · Christopher D Manning · Stefano Ermon · Chelsea Finn -
2023 Poster: An NLP Benchmark Dataset for Assessing Corporate Climate Policy Engagement »
Gaku Morio · Christopher D Manning -
2023 Oral: Direct Preference Optimization: Your Language Model is Secretly a Reward Model »
Rafael Rafailov · Archit Sharma · Eric Mitchell · Christopher D Manning · Stefano Ermon · Chelsea Finn -
2022 Poster: Deep Bidirectional Language-Knowledge Graph Pretraining »
Michihiro Yasunaga · Antoine Bosselut · Hongyu Ren · Xikun Zhang · Christopher D Manning · Percy Liang · Jure Leskovec -
2021 Poster: Evaluating State-of-the-Art Classification Models Against Bayes Optimality »
Ryan Theisen · Huan Wang · Lav Varshney · Caiming Xiong · Richard Socher -
2020 : Contributed Talk - ProGen: Language Modeling for Protein Generation »
Ali Madani · Bryan McCann · Nikhil Naik · · Possu Huang · Richard Socher -
2017 Poster: Learned in Translation: Contextualized Word Vectors »
Bryan McCann · James Bradbury · Caiming Xiong · Richard Socher -
2016 : Richard Socher - Tackling the Limits of Deep Learning for NLP »
Richard Socher -
2015 Symposium: Deep Learning Symposium »
Yoshua Bengio · Marc'Aurelio Ranzato · Honglak Lee · Max Welling · Andrew Y Ng -
2014 Workshop: Deep Learning and Representation Learning »
Andrew Y Ng · Yoshua Bengio · Adam Coates · Roland Memisevic · Sharanyan Chetlur · Geoffrey E Hinton · Shamim Nemati · Bryan Catanzaro · Surya Ganguli · Herbert Jaeger · Phil Blunsom · Leon Bottou · Volodymyr Mnih · Chen-Yu Lee · Rich M Schwartz -
2014 Poster: Learning Distributed Representations for Structured Output Prediction »
Vivek Srikumar · Christopher D Manning -
2014 Spotlight: Learning Distributed Representations for Structured Output Prediction »
Vivek Srikumar · Christopher D Manning -
2014 Poster: Global Belief Recursive Neural Networks »
Romain Paulus · Richard Socher · Christopher Manning -
2014 Poster: Simple MAP Inference via Low-Rank Relaxations »
Roy Frostig · Sida Wang · Percy Liang · Christopher D Manning -
2013 Demonstration: Easy Text Classification with Machine Learning »
Richard Socher · Romain Paulus · Bryan McCann · Andrew Y Ng -
2013 Poster: Zero-Shot Learning Through Cross-Modal Transfer »
Richard Socher · Milind Ganjoo · Christopher D Manning · Andrew Y Ng -
2012 Poster: Recursive Deep Learning on 3D Point Clouds »
Richard Socher · Bharath Bath · Brody Huval · Christopher D Manning · Andrew Y Ng -
2012 Poster: Deep Learning of invariant features via tracked video sequences »
Will Y Zou · Andrew Y Ng · Shenghuo Zhu · Kai Yu -
2012 Poster: Large Scale Distributed Deep Networks »
Jeff Dean · Greg Corrado · Rajat Monga · Kai Chen · Matthieu Devin · Quoc V Le · Mark Mao · Marc'Aurelio Ranzato · Andrew Senior · Paul Tucker · Ke Yang · Andrew Y Ng -
2012 Poster: Emergence of Object-Selective Features in Unsupervised Feature Learning »
Adam Coates · Andrej Karpathy · Andrew Y Ng -
2011 Workshop: Challenges in Learning Hierarchical Models: Transfer Learning and Optimization »
Quoc V. Le · Marc'Aurelio Ranzato · Russ Salakhutdinov · Josh Tenenbaum · Andrew Y Ng -
2011 Workshop: Deep Learning and Unsupervised Feature Learning »
Yoshua Bengio · Adam Coates · Yann LeCun · Nicolas Le Roux · Andrew Y Ng -
2011 Poster: ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning »
Quoc V. Le · Alexandre Karpenko · Jiquan Ngiam · Andrew Y Ng -
2011 Poster: Unfolding Recursive Autoencoders for Paraphrase Detection »
Richard Socher · Eric H Huang · Jeffrey Pennin · Andrew Y Ng · Christopher D Manning -
2011 Poster: Sparse Filtering »
Jiquan Ngiam · Pang Wei Koh · Zhenghao Chen · Sonia A Bhaskar · Andrew Y Ng -
2011 Spotlight: Sparse Filtering »
Jiquan Ngiam · Pang Wei Koh · Zhenghao Chen · Sonia A Bhaskar · Andrew Y Ng -
2011 Demonstration: Haptic Belt with Pedestrian Detection »
Jean Feng · Marc Rasi · Andrew Y Ng · Quoc V. Le · Morgan Quigley · Justin K Chen · Tiffany Low · Will Y Zou -
2011 Poster: Selecting Receptive Fields in Deep Networks »
Adam Coates · Andrew Y Ng -
2011 Poster: Unsupervised learning models of primary cortical receptive fields and receptive field plasticity »
Andrew M Saxe · Maneesh Bhand · Ritvik Mudur · Bipin Suresh · Andrew Y Ng -
2010 Workshop: Deep Learning and Unsupervised Feature Learning »
Honglak Lee · Marc'Aurelio Ranzato · Yoshua Bengio · Geoffrey E Hinton · Yann LeCun · Andrew Y Ng -
2010 Poster: Tiled convolutional neural networks »
Quoc V. Le · Jiquan Ngiam · Zhenghao Chen · Daniel Jin hao Chia · Pang Wei Koh · Andrew Y Ng -
2010 Poster: Energy Disaggregation via Discriminative Sparse Coding »
J. Zico Kolter · Siddarth Batra · Andrew Y Ng -
2009 Mini Symposium: Machine Learning for Sustainability »
J. Zico Kolter · Thomas Dietterich · Andrew Y Ng -
2009 Poster: Measuring Invariances in Deep Networks »
Ian Goodfellow · Quoc V. Le · Andrew M Saxe · Andrew Y Ng -
2009 Poster: Unsupervised feature learning for audio classification using convolutional deep belief networks »
Honglak Lee · Peter Pham · Yan Largman · Andrew Y Ng -
2009 Poster: A Bayesian Analysis of Dynamics in Free Recall »
Richard Socher · Samuel J Gershman · Adler Perotte · Per Sederberg · David Blei · Kenneth Norman -
2008 Demonstration: High-Accuracy 3D Sensing for Mobile Manipulators »
Stephen Gould · Morgan Quigley · Siddarth Batra · Ellen Klingbiel · Quoc V Le · Andrew Y Ng -
2007 Poster: Sparse deep belief net model for visual area V2 »
Honglak Lee · Ekanadham Chaitanya · Andrew Y Ng -
2007 Demonstration: Holistic Scene Understanding from Visual and Range Data »
Stephen Gould · Morgan Quigley · Andrew Y Ng · Daphne Koller -
2007 Spotlight: Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion »
J. Zico Kolter · Pieter Abbeel · Andrew Y Ng -
2007 Poster: Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion »
J. Zico Kolter · Pieter Abbeel · Andrew Y Ng -
2007 Demonstration: Building a 3-D Model From a Single Still Image »
Ashutosh Saxena · min sun · Andrew Y Ng -
2007 Poster: Efficient multiple hyperparameter learning for log-linear models »
Chuong B Do · Chuan-Sheng Foo · Andrew Y Ng -
2006 Demonstration: Peripheral-Foveal Vision for Real-time Object Recognition »
Benjamin Sapp · Stephen Gould · Adrian Kaehler · Gary R Bradski · Andrew Y Ng -
2006 Poster: Robotic Grasping of Novel Objects »
Ashutosh Saxena · Justin Driemeyer · Justin Kearns · Andrew Y Ng -
2006 Poster: Map-Reduce for Machine Learning on Multicore »
Cheng-Tao Chu · Sang Kyun Kim · Yi-An Lin · YuanYuan Yu · Gary R Bradski · Andrew Y Ng · Kunle Olukotun -
2006 Poster: An Application of Reinforcement Learning to Aerobatic Helicopter Flight »
Pieter Abbeel · Adam P Coates · Andrew Y Ng · Morgan Quigley -
2006 Talk: Map-Reduce for Machine Learning on Multicore »
Cheng-Tao Chu · Sang Kyun Kim · Yi-An Lin · YuanYuan Yu · Gary R Bradski · Andrew Y Ng · Kunle Olukotun -
2006 Spotlight: Robotic Grasping of Novel Objects »
Ashutosh Saxena · Justin Driemeyer · Justin Kearns · Andrew Y Ng -
2006 Talk: An Application of Reinforcement Learning to Aerobatic Helicopter Flight »
Pieter Abbeel · Adam P Coates · Andrew Y Ng · Morgan Quigley -
2006 Poster: Efficient sparse coding algorithms, end-stopping and nCRF surround suppression »
Honglak Lee · Alexis Battle · Raina Rajat · Andrew Y Ng