Timezone: »
We present a nonparametric Bayesian method for texture learning and synthesis. A texture image is represented by a 2D-Hidden Markov Model (2D-HMM) where the hidden states correspond to the cluster labeling of textons and the transition matrix encodes their spatial layout (the compatibility between adjacent textons). 2D-HMM is coupled with the Hierarchical Dirichlet process (HDP) which allows the number of textons and the complexity of transition matrix grow as the input texture becomes irregular. The HDP makes use of Dirichlet process prior which favors regular textures by penalizing the model complexity. This framework (HDP-2D-HMM) learns the texton vocabulary and their spatial layout jointly and automatically. The HDP-2D-HMM results in a compact representation of textures which allows fast texture synthesis with comparable rendering quality over the state-of-the-art image-based rendering methods. We also show that HDP-2D-HMM can be applied to perform image segmentation and synthesis.
Author Information
Leo Zhu
Yuanhao Chen (University of California, Los Angeles)
Bill Freeman (MIT/Google)
Antonio Torralba (Massachusetts Institute of Technology)
More from the Same Authors
-
2021 : ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation »
Chuang Gan · Jeremy Schwartz · Seth Alter · Damian Mrowca · Martin Schrimpf · James Traer · Julian De Freitas · Jonas Kubilius · Abhishek Bhandwaldar · Nick Haber · Megumi Sano · Kuno Kim · Elias Wang · Michael Lingelbach · Aidan Curtis · Kevin Feigelis · Daniel Bear · Dan Gutfreund · David Cox · Antonio Torralba · James J DiCarlo · Josh Tenenbaum · Josh McDermott · Dan Yamins -
2021 Spotlight: Learning to Compose Visual Relations »
Nan Liu · Shuang Li · Yilun Du · Josh Tenenbaum · Antonio Torralba -
2021 Spotlight: Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering »
Vincent Sitzmann · Semon Rezchikov · Bill Freeman · Josh Tenenbaum · Fredo Durand -
2021 Spotlight: Learning to See by Looking at Noise »
Manel Baradad Jurjo · Jonas Wulff · Tongzhou Wang · Phillip Isola · Antonio Torralba -
2021 Spotlight: Measuring Generalization with Optimal Transport »
Ching-Yao Chuang · Youssef Mroueh · Kristjan Greenewald · Antonio Torralba · Stefanie Jegelka -
2021 : Finding Maximally Informative Patches in Images »
Howard Zhong · Guha Balakrishnan · Richard Bowen · Ramin Zabih · Bill Freeman -
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2021 : Finding Maximally Informative Patches in Images »
Howard Zhong · Guha Balakrishnan · Richard Bowen · Ramin Zabih · Bill Freeman -
2023 Poster: Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision »
Ayush Tewari · Tianwei Yin · George Cazenavette · Semon Rezchikov · Josh Tenenbaum · Fredo Durand · Bill Freeman · Vincent Sitzmann -
2022 Poster: Associating Objects and Their Effects in Video through Coordination Games »
Erika Lu · Forrester Cole · Weidi Xie · Tali Dekel · Bill Freeman · Andrew Zisserman · Michael Rubinstein -
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2021 Poster: Learning to Compose Visual Relations »
Nan Liu · Shuang Li · Yilun Du · Josh Tenenbaum · Antonio Torralba -
2021 Poster: EditGAN: High-Precision Semantic Image Editing »
Huan Ling · Karsten Kreis · Daiqing Li · Seung Wook Kim · Antonio Torralba · Sanja Fidler -
2021 Poster: Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering »
Vincent Sitzmann · Semon Rezchikov · Bill Freeman · Josh Tenenbaum · Fredo Durand -
2021 Poster: Learning to See by Looking at Noise »
Manel Baradad Jurjo · Jonas Wulff · Tongzhou Wang · Phillip Isola · Antonio Torralba -
2021 Poster: PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning »
Yining Hong · Li Yi · Josh Tenenbaum · Antonio Torralba · Chuang Gan -
2021 Poster: Editing a classifier by rewriting its prediction rules »
Shibani Santurkar · Dimitris Tsipras · Mahalaxmi Elango · David Bau · Antonio Torralba · Aleksander Madry -
2021 Poster: Measuring Generalization with Optimal Transport »
Ching-Yao Chuang · Youssef Mroueh · Kristjan Greenewald · Antonio Torralba · Stefanie Jegelka -
2021 : ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation »
Chuang Gan · Jeremy Schwartz · Seth Alter · Damian Mrowca · Martin Schrimpf · James Traer · Julian De Freitas · Jonas Kubilius · Abhishek Bhandwaldar · Nick Haber · Megumi Sano · Kuno Kim · Elias Wang · Michael Lingelbach · Aidan Curtis · Kevin Feigelis · Daniel Bear · Dan Gutfreund · David Cox · Antonio Torralba · James J DiCarlo · Josh Tenenbaum · Josh McDermott · Dan Yamins -
2020 Poster: Causal Discovery in Physical Systems from Videos »
Yunzhu Li · Antonio Torralba · Anima Anandkumar · Dieter Fox · Animesh Garg -
2020 Poster: Multi-Plane Program Induction with 3D Box Priors »
Yikai Li · Jiayuan Mao · Xiuming Zhang · Bill Freeman · Josh Tenenbaum · Noah Snavely · Jiajun Wu -
2020 Demonstration: MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval »
Mark Hamilton · Stephanie Fu · Mindren Lu · Johnny Bui · Margaret Wang · Felix Tran · Marina Rogers · Darius Bopp · Christopher Hoder · Lei Zhang · Bill Freeman -
2019 : Feathers, wings and the future of computer vision research »
Bill Freeman -
2019 Poster: Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization »
Miika Aittala · Prafull Sharma · Lukas Murmann · Adam Yedidia · Gregory Wornell · Bill Freeman · Fredo Durand -
2018 Poster: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Oral: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Poster: Visual Object Networks: Image Generation with Disentangled 3D Representations »
Jun-Yan Zhu · Zhoutong Zhang · Chengkai Zhang · Jiajun Wu · Antonio Torralba · Josh Tenenbaum · Bill Freeman -
2018 Poster: Learning to Exploit Stability for 3D Scene Parsing »
Yilun Du · Zhijian Liu · Hector Basevi · Ales Leonardis · Bill Freeman · Josh Tenenbaum · Jiajun Wu -
2018 Poster: 3D-Aware Scene Manipulation via Inverse Graphics »
Shunyu Yao · Tzu Ming Hsu · Jun-Yan Zhu · Jiajun Wu · Antonio Torralba · Bill Freeman · Josh Tenenbaum -
2018 Poster: Co-regularized Alignment for Unsupervised Domain Adaptation »
Abhishek Kumar · Prasanna Sattigeri · Kahini Wadhawan · Leonid Karlinsky · Rogerio Feris · Bill Freeman · Gregory Wornell -
2017 : Sight and sound »
Bill Freeman -
2017 Spotlight: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Spotlight: Scene Physics Acquisition via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Learning to See Physics via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Poster: MarrNet: 3D Shape Reconstruction via 2.5D Sketches »
Jiajun Wu · Yifan Wang · Tianfan Xue · Xingyuan Sun · Bill Freeman · Josh Tenenbaum -
2016 : Invited Talk - Learning to see objects by listening »
Antonio Torralba -
2016 : Bill Freeman »
Bill Freeman -
2016 Poster: Unsupervised Learning of Spoken Language with Visual Context »
David Harwath · Antonio Torralba · James Glass -
2016 Poster: Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling »
Jiajun Wu · Chengkai Zhang · Tianfan Xue · Bill Freeman · Josh Tenenbaum -
2016 Poster: Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks »
Tianfan Xue · Jiajun Wu · Katherine Bouman · Bill Freeman -
2016 Oral: Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks »
Tianfan Xue · Jiajun Wu · Katherine Bouman · Bill Freeman -
2015 Poster: Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning »
Jiajun Wu · Ilker Yildirim · Joseph Lim · Bill Freeman · Josh Tenenbaum -
2014 Poster: Shape and Illumination from Shading using the Generic Viewpoint Assumption »
Daniel Zoran · Dilip Krishnan · José Bento · Bill Freeman -
2014 Poster: Learning Deep Features for Scene Recognition using Places Database »
Bolei Zhou · Agata Lapedriza · Jianxiong Xiao · Antonio Torralba · Aude Oliva -
2014 Spotlight: Learning Deep Features for Scene Recognition using Places Database »
Bolei Zhou · Agata Lapedriza · Jianxiong Xiao · Antonio Torralba · Aude Oliva -
2012 Poster: Modeling the Forgetting Process using Image Regions »
Aditya Khosla · Jianxiong Xiao · Antonio Torralba · Aude Oliva -
2012 Poster: Localizing 3D cuboids in single-view images »
Jianxiong Xiao · Bryan C Russell · Antonio Torralba -
2011 Poster: Learning to Learn with Compound HD Models »
Russ Salakhutdinov · Josh Tenenbaum · Antonio Torralba -
2011 Poster: Understanding the Intrinsic Memorability of Images »
Phillip Isola · Devi Parikh · Antonio Torralba · Aude Oliva -
2011 Spotlight: Learning to Learn with Compound HD Models »
Russ Salakhutdinov · Josh Tenenbaum · Antonio Torralba -
2011 Poster: Transfer Learning by Borrowing Examples »
Joseph Lim · Russ Salakhutdinov · Antonio Torralba -
2010 Workshop: Machine Learning meets Computational Photography »
Stefan Harmeling · Michael Hirsch · Bill Freeman · Peyman Milanfar -
2009 Poster: Unsupervised Detection of Regions of Interest Using Iterative Link Analysis »
Gunhee Kim · Antonio Torralba -
2009 Poster: Segmenting Scenes by Matching Image Composites »
Bryan C Russell · Alexei A Efros · Josef Sivic · Bill Freeman · Andrew Zisserman -
2009 Session: Oral session 7: Vision and Inference »
Antonio Torralba -
2009 Poster: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Oral: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Tutorial: Understanding Visual Scenes »
Antonio Torralba -
2008 Mini Symposium: Computational Photography »
Bill Freeman · Bernhard Schölkopf -
2008 Poster: Spectral Hashing »
Yair Weiss · Antonio Torralba · Rob Fergus -
2007 Spotlight: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman -
2007 Poster: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman