Timezone: »
The human ability to acquire a visual concept from a few examples, and to recognize instances of that concept in the context of a complex scene, poses a central challenge to the fields of computer vision, cognitive science, and machine learning. Representing visual objects and scenes as the human mind does is likely to require structural sophistication, something akin to a grammar for image parsing, with multiple levels of hierarchy and abstraction, rather than the "flat" feature vectors which are standard in most statistical pattern recognition. Grammar-based approaches to vision have been slow to develop, largely due to the absence of effective methods for learning and inference under uncertainty. However, recent advances in machine learning and statistical models for natural language have inspired a renewed interest in structural representations of visual objects, categories, and scenes. The result is a new and emerging body of research in computational visual cognition that combines sophisticated probabilistic methods for learning and inference with classical grammar-based approaches to representation. The goal of our workshop is to explore these new directions, in the context of several interdisciplinary connections that converge distinctively at NIPS. We will focus on these challenges: How can we learn better probabilistic grammars for machine vision by drawing on state-of-the-art methods in statistical machine learning or natural language learning? What can probabilistic grammars for machine vision tell us about human visual cognition? How can human visual cognition inspire new developments in computational vision and machine learning?
Author Information
Virginia Savova (Massachusetts Institute of Technology)
Josh Tenenbaum (MIT)
Josh Tenenbaum is an Associate Professor of Computational Cognitive Science at MIT in the Department of Brain and Cognitive Sciences and the Computer Science and Artificial Intelligence Laboratory (CSAIL). He received his PhD from MIT in 1999, and was an Assistant Professor at Stanford University from 1999 to 2002. He studies learning and inference in humans and machines, with the twin goals of understanding human intelligence in computational terms and bringing computers closer to human capacities. He focuses on problems of inductive generalization from limited data -- learning concepts and word meanings, inferring causal relations or goals -- and learning abstract knowledge that supports these inductive leaps in the form of probabilistic generative models or 'intuitive theories'. He has also developed several novel machine learning methods inspired by human learning and perception, most notably Isomap, an approach to unsupervised learning of nonlinear manifolds in high-dimensional data. He has been Associate Editor for the journal Cognitive Science, has been active on program committees for the CogSci and NIPS conferences, and has co-organized a number of workshops, tutorials and summer schools in human and machine learning. Several of his papers have received outstanding paper awards or best student paper awards at the IEEE Computer Vision and Pattern Recognition (CVPR), NIPS, and Cognitive Science conferences. He is the recipient of the New Investigator Award from the Society for Mathematical Psychology (2005), the Early Investigator Award from the Society of Experimental Psychologists (2007), and the Distinguished Scientific Award for Early Career Contribution to Psychology (in the area of cognition and human learning) from the American Psychological Association (2008).
Leslie Kaelbling (MIT)
Alan L Yuille (UCLA)
More from the Same Authors
-
2020 Workshop: Workshop on Computer Assisted Programming (CAP) »
Augustus Odena · Charles Sutton · Nadia Polikarpova · Josh Tenenbaum · Armando Solar-Lezama · Isil Dillig -
2020 Workshop: KR2ML - Knowledge Representation and Reasoning Meets Machine Learning »
Veronika Thost · Kartik Talamadupula · Vivek Srikumar · Chenwei Zhang · Josh Tenenbaum -
2020 Workshop: Differentiable computer vision, graphics, and physics in machine learning »
Krishna Murthy Jatavallabhula · Kelsey Allen · Victoria Dean · Johanna Hansen · Shuran Song · Florian Shkurti · Liam Paull · Derek Nowrouzezahrai · Josh Tenenbaum -
2020 Poster: Online Bayesian Goal Inference for Boundedly Rational Planning Agents »
Tan Zhi-Xuan · Jordyn Mann · Tom Silver · Josh Tenenbaum · Vikash Mansinghka -
2020 Poster: Program Synthesis with Pragmatic Communication »
Yewen Pu · Kevin Ellis · Marta Kryven · Josh Tenenbaum · Armando Solar-Lezama -
2020 Poster: Learning Compositional Rules via Neural Program Synthesis »
Maxwell Nye · Armando Solar-Lezama · Josh Tenenbaum · Brenden Lake -
2020 Poster: Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical Models »
Adarsh K Jeewajee · Leslie Kaelbling -
2020 Poster: Learning abstract structure for drawing by efficient motor program induction »
Lucas Tian · Kevin Ellis · Marta Kryven · Josh Tenenbaum -
2020 Oral: Learning abstract structure for drawing by efficient motor program induction »
Lucas Tian · Kevin Ellis · Marta Kryven · Josh Tenenbaum -
2020 Poster: Multi-Plane Program Induction with 3D Box Priors »
Yikai Li · Jiayuan Mao · Xiuming Zhang · Bill Freeman · Josh Tenenbaum · Noah Snavely · Jiajun Wu -
2020 Poster: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2020 Oral: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2019 Poster: Write, Execute, Assess: Program Synthesis with a REPL »
Kevin Ellis · Maxwell Nye · Yewen Pu · Felix Sosa · Josh Tenenbaum · Armando Solar-Lezama -
2019 Poster: ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models »
Andrei Barbu · David Mayo · Julian Alverio · William Luo · Christopher Wang · Dan Gutfreund · Josh Tenenbaum · Boris Katz -
2019 Poster: Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations »
Kevin Smith · Lingjie Mei · Shunyu Yao · Jiajun Wu · Elizabeth Spelke · Josh Tenenbaum · Tomer Ullman -
2019 Poster: Visual Concept-Metaconcept Learning »
Chi Han · Jiayuan Mao · Chuang Gan · Josh Tenenbaum · Jiajun Wu -
2019 Poster: Neural Relational Inference with Fast Modular Meta-learning »
Ferran Alet · Erica Weng · Tomás Lozano-Pérez · Leslie Kaelbling -
2019 Poster: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2019 Spotlight: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2018 Workshop: Infer to Control: Probabilistic Reinforcement Learning and Structured Control »
Leslie Kaelbling · Martin Riedmiller · Marc Toussaint · Igor Mordatch · Roy Fox · Tuomas Haarnoja -
2018 Workshop: Modeling the Physical World: Learning, Perception, and Control »
Jiajun Wu · Kelsey Allen · Kevin Smith · Jessica Hamrick · Emmanuel Dupoux · Marc Toussaint · Josh Tenenbaum -
2018 Poster: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Poster: Learning to Infer Graphics Programs from Hand-Drawn Images »
Kevin Ellis · Daniel Ritchie · Armando Solar-Lezama · Josh Tenenbaum -
2018 Poster: Learning Libraries of Subroutines for Neurally–Guided Bayesian Program Induction »
Kevin Ellis · Lucas Morales · Mathias Sablé-Meyer · Armando Solar-Lezama · Josh Tenenbaum -
2018 Oral: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Spotlight: Learning to Infer Graphics Programs from Hand-Drawn Images »
Kevin Ellis · Daniel Ritchie · Armando Solar-Lezama · Josh Tenenbaum -
2018 Spotlight: Learning Libraries of Subroutines for Neurally–Guided Bayesian Program Induction »
Kevin Ellis · Lucas Morales · Mathias Sablé-Meyer · Armando Solar-Lezama · Josh Tenenbaum -
2018 Poster: Visual Object Networks: Image Generation with Disentangled 3D Representations »
Jun-Yan Zhu · Zhoutong Zhang · Chengkai Zhang · Jiajun Wu · Antonio Torralba · Josh Tenenbaum · Bill Freeman -
2018 Poster: Learning to Share and Hide Intentions using Information Regularization »
DJ Strouse · Max Kleiman-Weiner · Josh Tenenbaum · Matt Botvinick · David Schwab -
2018 Poster: Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior »
Zi Wang · Beomjoon Kim · Leslie Kaelbling -
2018 Spotlight: Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior »
Zi Wang · Beomjoon Kim · Leslie Kaelbling -
2018 Poster: Learning to Exploit Stability for 3D Scene Parsing »
Yilun Du · Zhijian Liu · Hector Basevi · Ales Leonardis · Bill Freeman · Josh Tenenbaum · Jiajun Wu -
2018 Poster: End-to-End Differentiable Physics for Learning and Control »
Filipe de Avila Belbute-Peres · Kevin Smith · Kelsey Allen · Josh Tenenbaum · J. Zico Kolter -
2018 Spotlight: End-to-End Differentiable Physics for Learning and Control »
Filipe de Avila Belbute-Peres · Kevin Smith · Kelsey Allen · Josh Tenenbaum · J. Zico Kolter -
2018 Poster: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Poster: 3D-Aware Scene Manipulation via Inverse Graphics »
Shunyu Yao · Tzu Ming Hsu · Jun-Yan Zhu · Jiajun Wu · Antonio Torralba · Bill Freeman · Josh Tenenbaum -
2018 Spotlight: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Poster: Flexible neural representation for physics prediction »
Damian Mrowca · Chengxu Zhuang · Elias Wang · Nick Haber · Li Fei-Fei · Josh Tenenbaum · Daniel Yamins -
2017 Workshop: Learning Disentangled Features: from Perception to Control »
Emily Denton · Siddharth Narayanaswamy · Tejas Kulkarni · Honglak Lee · Diane Bouchacourt · Josh Tenenbaum · David Pfau -
2017 Spotlight: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Spotlight: Scene Physics Acquisition via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Learning to See Physics via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Poster: MarrNet: 3D Shape Reconstruction via 2.5D Sketches »
Jiajun Wu · Yifan Wang · Tianfan Xue · Xingyuan Sun · Bill Freeman · Josh Tenenbaum -
2017 Poster: Label Distribution Learning Forests »
Wei Shen · KAI ZHAO · Yilu Guo · Alan Yuille -
2017 Poster: Self-Supervised Intrinsic Image Decomposition »
Michael Janner · Jiajun Wu · Tejas Kulkarni · Ilker Yildirim · Josh Tenenbaum -
2017 Tutorial: Engineering and Reverse-Engineering Intelligence Using Probabilistic Programs, Program Induction, and Deep Learning »
Josh Tenenbaum · Vikash Mansinghka -
2016 Workshop: Intuitive Physics »
Adam Lerer · Jiajun Wu · Josh Tenenbaum · Emmanuel Dupoux · Rob Fergus -
2016 Poster: Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation »
Tejas Kulkarni · Karthik Narasimhan · Ardavan Saeedi · Josh Tenenbaum -
2016 Poster: Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling »
Jiajun Wu · Chengkai Zhang · Tianfan Xue · Bill Freeman · Josh Tenenbaum -
2016 Poster: SURGE: Surface Regularized Geometry Estimation from a Single Image »
Peng Wang · Xiaohui Shen · Bryan Russell · Scott Cohen · Brian Price · Alan L Yuille -
2016 Poster: Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images »
Junhua Mao · Jiajing Xu · Kevin Jing · Alan L Yuille -
2016 Poster: Sampling for Bayesian Program Learning »
Kevin Ellis · Armando Solar-Lezama · Josh Tenenbaum -
2016 Poster: Probing the Compositionality of Intuitive Functions »
Eric Schulz · Josh Tenenbaum · David Duvenaud · Maarten Speekenbrink · Samuel J Gershman -
2015 Workshop: Black box learning and inference »
Josh Tenenbaum · Jan-Willem van de Meent · Tejas Kulkarni · S. M. Ali Eslami · Brooks Paige · Frank Wood · Zoubin Ghahramani -
2015 Poster: Softstar: Heuristic-Guided Probabilistic Inference »
Mathew Monfort · Brenden M Lake · Brenden Lake · Brian Ziebart · Patrick Lucey · Josh Tenenbaum -
2015 Poster: Deep Convolutional Inverse Graphics Network »
Tejas Kulkarni · William Whitney · Pushmeet Kohli · Josh Tenenbaum -
2015 Spotlight: Deep Convolutional Inverse Graphics Network »
Tejas Kulkarni · William Whitney · Pushmeet Kohli · Josh Tenenbaum -
2015 Poster: Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning »
Jiajun Wu · Ilker Yildirim · Joseph Lim · Bill Freeman · Josh Tenenbaum -
2015 Poster: Unsupervised Learning by Program Synthesis »
Kevin Ellis · Armando Solar-Lezama · Josh Tenenbaum -
2015 Poster: Bayesian Optimization with Exponential Convergence »
Kenji Kawaguchi · Leslie Kaelbling · Tomás Lozano-Pérez -
2014 Workshop: 3rd NIPS Workshop on Probabilistic Programming »
Daniel Roy · Josh Tenenbaum · Thomas Dietterich · Stuart J Russell · YI WU · Ulrik R Beierholm · Alp Kucukelbir · Zenna Tavares · Yura Perov · Daniel Lee · Brian Ruttenberg · Sameer Singh · Michael Hughes · Marco Gaboardi · Alexey Radul · Vikash Mansinghka · Frank Wood · Sebastian Riedel · Prakash Panangaden -
2014 Workshop: Modern Nonparametrics 3: Automating the Learning Pipeline »
Eric Xing · Mladen Kolar · Arthur Gretton · Samory Kpotufe · Han Liu · Zoltán Szabó · Alan L Yuille · Andrew G Wilson · Ryan Tibshirani · Sasha Rakhlin · Damian Kozbur · Bharath Sriperumbudur · David Lopez-Paz · Kirthevasan Kandasamy · Francesco Orabona · Andreas Damianou · Wacha Bounliphone · Yanshuai Cao · Arijit Das · Yingzhen Yang · Giulia DeSalvo · Dmitry Storcheus · Roberto Valerio -
2014 Poster: Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations »
Xianjie Chen · Alan L Yuille -
2014 Poster: Learning From Weakly Supervised Data by The Expectation Loss SVM (e-SVM) algorithm »
Jun Zhu · Junhua Mao · Alan L Yuille -
2013 Workshop: Deep Learning »
Yoshua Bengio · Hugo Larochelle · Russ Salakhutdinov · Tomas Mikolov · Matthew D Zeiler · David Mcallester · Nando de Freitas · Josh Tenenbaum · Jian Zhou · Volodymyr Mnih -
2013 Poster: One-shot learning by inverting a compositional causal process »
Brenden M Lake · Russ Salakhutdinov · Josh Tenenbaum -
2013 Poster: Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs »
Vikash Mansinghka · Tejas D Kulkarni · Yura N Perov · Josh Tenenbaum -
2013 Oral: Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs »
Vikash Mansinghka · Tejas D Kulkarni · Yura N Perov · Josh Tenenbaum -
2011 Workshop: Challenges in Learning Hierarchical Models: Transfer Learning and Optimization »
Quoc V. Le · Marc'Aurelio Ranzato · Russ Salakhutdinov · Josh Tenenbaum · Andrew Y Ng -
2011 Poster: Learning to Learn with Compound HD Models »
Russ Salakhutdinov · Josh Tenenbaum · Antonio Torralba -
2011 Spotlight: Learning to Learn with Compound HD Models »
Russ Salakhutdinov · Josh Tenenbaum · Antonio Torralba -
2010 Workshop: Transfer Learning Via Rich Generative Models. »
Russ Salakhutdinov · Ryan Adams · Josh Tenenbaum · Zoubin Ghahramani · Tom Griffiths -
2010 Invited Talk (Posner Lecture): How to Grow a Mind: Statistics, Structure and Abstraction »
Josh Tenenbaum -
2010 Poster: Dynamic Infinite Relational Model for Time-varying Relational Data Analysis »
Katsuhiko Ishiguro · Tomoharu Iwata · Naonori Ueda · Josh Tenenbaum -
2010 Poster: Gaussian sampling by local perturbations »
George Papandreou · Alan L Yuille -
2010 Poster: Functional form of motion priors in human motion perception »
HongJing Lu · Tungyou Lin · Alan L Lee · Luminita Vese · Alan L Yuille -
2010 Poster: Nonparametric Bayesian Policy Priors for Reinforcement Learning »
Finale P Doshi-Velez · David Wingate · Nicholas Roy · Josh Tenenbaum -
2010 Poster: A unified model of short-range and long-range motion perception »
Shuang Wu · Xuming He · HongJing Lu · Alan L Yuille -
2009 Workshop: Bounded-rational analyses of human cognition: Bayesian models, approximate inference, and the brain »
Noah Goodman · Edward Vul · Tom Griffiths · Josh Tenenbaum -
2009 Workshop: Analyzing Networks and Learning With Graphs »
Edo M Airoldi · Jure Leskovec · Jon Kleinberg · Josh Tenenbaum -
2009 Poster: Perceptual Multistability as Markov Chain Monte Carlo Inference »
Samuel J Gershman · Edward Vul · Josh Tenenbaum -
2009 Poster: Modeling the spacing effect in sequential category learning »
HongJing Lu · Matthew Weiden · Alan L Yuille -
2009 Poster: Help or Hinder: Bayesian Models of Social Goal Inference »
Tomer D Ullman · Chris L Baker · Owen Macindoe · Owain Evans · Noah Goodman · Josh Tenenbaum -
2009 Spotlight: Perceptual Multistability as Markov Chain Monte Carlo Inference »
Samuel J Gershman · Edward Vul · Josh Tenenbaum -
2009 Poster: Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model »
Edward Vul · Michael C Frank · George Alvarez · Josh Tenenbaum -
2009 Oral: Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model »
Edward Vul · Michael C Frank · George Alvarez · Josh Tenenbaum -
2009 Poster: Modelling Relational Data using Bayesian Clustered Tensor Factorization »
Ilya Sutskever · Russ Salakhutdinov · Josh Tenenbaum -
2008 Workshop: Probabilistic Programming: Universal Languages, Systems and Applications »
Daniel Roy · John Winn · David A McAllester · Vikash Mansinghka · Josh Tenenbaum -
2008 Workshop: Machine learning meets human learning »
Nathaniel D Daw · Tom Griffiths · Josh Tenenbaum · Jerry Zhu -
2008 Poster: A Hierarchical Image Model for Polynomial-Time 2D Parsing »
Long Zhu · Yuanhao Chen · Yuan Lin · Alan L Yuille -
2008 Poster: Model selection and velocity estimation using novel priors for motion patterns »
Alan L Yuille · Shuang Wu · HongJing Lu -
2008 Spotlight: A Hierarchical Image Model for Polynomial-Time 2D Parsing »
Long Zhu · Yuanhao Chen · Yuan Lin · Alan L Yuille -
2008 Oral: Model selection and velocity estimation using novel priors for motion patterns »
Alan L Yuille · Shuang Wu · HongJing Lu -
2008 Poster: Multi-Agent Filtering with Infinitely Nested Beliefs »
Luke Zettlemoyer · Brian Milch · Leslie Kaelbling -
2008 Spotlight: Multi-Agent Filtering with Infinitely Nested Beliefs »
Luke Zettlemoyer · Brian Milch · Leslie Kaelbling -
2007 Spotlight: A Bayesian Framework for Cross-Situational Word-Learning »
Michael C Frank · Noah Goodman · Josh Tenenbaum -
2007 Poster: A Bayesian Framework for Cross-Situational Word-Learning »
Michael C Frank · Noah Goodman · Josh Tenenbaum -
2007 Poster: A complexity measure for intuitive theories »
Charles Kemp · Noah Goodman · Josh Tenenbaum -
2007 Poster: The Noisy-Logical Distribution and its Application to Causal Inference »
Alan L Yuille · HongJing Lu -
2007 Poster: Rapid Inference on a novel AND/OR graph: Detection, Segmentation and Parsing of Articulated Deformable Objects in Cluttered Backgrounds »
Yuanhao Chen · Long Zhu · Chenxi Lin · Alan L Yuille · Hongjiang Zhang -
2006 Talk: Unsupervised Learning of a Probabilistic Grammar for Object Detection and Parsing »
Long Zhu · Yuanhao Chen · Alan L Yuille -
2006 Poster: Unsupervised Learning of a Probabilistic Grammar for Object Detection and Parsing »
Long Zhu · Yuanhao Chen · Alan L Yuille -
2006 Poster: Combining causal and similarity-based reasoning »
Charles Kemp · Patrick Shafto · Allison Berke · Josh Tenenbaum -
2006 Poster: Multiple timescales and uncertainty in motor adaptation »
Konrad P Kording · Josh Tenenbaum · Reza Shadmehr -
2006 Poster: Learning annotated hierarchies from relational data »
Daniel Roy · Charles Kemp · Vikash Mansinghka · Josh Tenenbaum -
2006 Talk: Learning annotated hierarchies from relational data »
Daniel Roy · Charles Kemp · Vikash Mansinghka · Josh Tenenbaum -
2006 Spotlight: Multiple timescales and uncertainty in motor adaptation »
Konrad P Kording · Josh Tenenbaum · Reza Shadmehr -
2006 Talk: Combining causal and similarity-based reasoning »
Charles Kemp · Patrick Shafto · Allison Berke · Josh Tenenbaum -
2006 Poster: Causal inference in sensorimotor integration »
Konrad P Kording · Josh Tenenbaum -
2006 Tutorial: Bayesian Models of Human Learning and Inference »
Josh Tenenbaum