Timezone: »
We propose a new method that uses deep learning techniques to accelerate the popular alternating direction method of multipliers (ADMM) solution for inverse problems. The ADMM updates consist of a proximity operator, a least squares regression that includes a big matrix inversion, and an explicit solution for updating the dual variables. Typically, inner loops are required to solve the first two sub-minimization problems due to the intractability of the prior and the matrix inversion. To avoid such drawbacks or limitations, we propose an inner-loop free update rule with two pre-trained deep convolutional architectures. More specifically, we learn a conditional denoising auto-encoder which imposes an implicit data-dependent prior/regularization on ground-truth in the first sub-minimization problem. This design follows an empirical Bayesian strategy, leading to so-called amortized inference. For matrix inversion in the second sub-problem, we learn a convolutional neural network to approximate the matrix inversion, i.e., the inverse mapping is learned by feeding the input through the learned forward network. Note that training this neural network does not require ground-truth or measurements, i.e., data-independent. Extensive experiments on both synthetic data and real datasets demonstrate the efficiency and accuracy of the proposed method compared with the conventional ADMM solution using inner loops for solving inverse problems.
Author Information
Kai Fan (Duke University)
Qi Wei (JPMorgan Chase)
Lawrence Carin (Duke University)
Katherine Heller (Duke)
More from the Same Authors
-
2021 Spotlight: Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer »
Junya Chen · Zidi Xiu · Benjamin Goldstein · Ricardo Henao · Lawrence Carin · Chenyang Tao -
2021 : Maintaining fairness across distribution shifts: do we have viable solutions for real-world applications? »
Jessica Schrouff · Natalie Harris · Sanmi Koyejo · Ibrahim Alabdulmohsin · Eva Schnider · Diana Mincu · Christina Chen · Awa Dieng · Yuan Liu · Vivek Natarajan · Katherine Heller · Alexander D'Amour -
2022 : CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks »
Sakshi Varshney · Vinay Verma · Srijith PK · Piyush Rai · Lawrence Carin -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Berk Ustun · Chirag Nagpal · Katherine Heller -
2023 Poster: Participatory Personalization in Classification »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 Poster: Diagnosing failures of fairness transfer across distribution shift in real-world medical settings »
Jessica Schrouff · Natalie Harris · Sanmi Koyejo · Ibrahim Alabdulmohsin · Eva Schnider · Krista Opsahl-Ong · Alexander Brown · Subhrajit Roy · Diana Mincu · Christina Chen · Awa Dieng · Yuan Liu · Vivek Natarajan · Alan Karthikesalingam · Katherine Heller · Silvia Chiappa · Alexander D'Amour -
2021 Poster: Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer »
Junya Chen · Zidi Xiu · Benjamin Goldstein · Ricardo Henao · Lawrence Carin · Chenyang Tao -
2021 Poster: CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks »
Sakshi Varshney · Vinay Kumar Verma · P. K. Srijith · Lawrence Carin · Piyush Rai -
2020 Workshop: Machine Learning for Mobile Health »
Joseph Futoma · Walter Dempsey · Katherine Heller · Yian Ma · Nicholas Foti · Marianne Njifon · Kelly Zhang · Jieru Shi -
2020 Poster: GAN Memory with No Forgetting »
Yulai Cong · Miaoyun Zhao · Jianqiao Li · Sijia Wang · Lawrence Carin -
2020 Symposium: COVID-19 Symposium Day 2 »
Andrew Beam · Tristan Naumann · Katherine Heller · Elaine Nsoesie -
2020 Poster: Reconsidering Generative Objectives For Counterfactual Reasoning »
Danni Lu · Chenyang Tao · Junya Chen · Fan Li · Feng Guo · Lawrence Carin -
2020 Poster: AutoSync: Learning to Synchronize for Data-Parallel Distributed Deep Learning »
Hao Zhang · Yuan Li · Zhijie Deng · Xiaodan Liang · Lawrence Carin · Eric Xing -
2020 Poster: Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability »
Nathan Inkawhich · Kevin J Liang · Binghui Wang · Matthew Inkawhich · Lawrence Carin · Yiran Chen -
2020 Symposium: COVID-19 Symposium Day 1 »
Andrew Beam · Tristan Naumann · Katherine Heller · Elaine Nsoesie -
2020 Poster: Calibrating CNNs for Lifelong Learning »
Pravendra Singh · Vinay Kumar Verma · Pratik Mazumder · Lawrence Carin · Piyush Rai -
2019 Poster: Improving Textual Network Learning with Variational Homophilic Embeddings »
Wenlin Wang · Chenyang Tao · Zhe Gan · Guoyin Wang · Liqun Chen · Xinyuan Zhang · Ruiyi Zhang · Qian Yang · Ricardo Henao · Lawrence Carin -
2019 Poster: Ouroboros: On Accelerating Training of Transformer-Based Language Models »
Qian Yang · Zhouyuan Huo · Wenlin Wang · Lawrence Carin -
2019 Poster: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Spotlight: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Poster: Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching »
Hongteng Xu · Dixin Luo · Lawrence Carin -
2019 Poster: Kernel-Based Approaches for Sequence Modeling: Connections to Neural Methods »
Kevin J Liang · Guoyin Wang · Yitong Li · Ricardo Henao · Lawrence Carin -
2019 Poster: Certified Adversarial Robustness with Additive Noise »
Bai Li · Changyou Chen · Wenlin Wang · Lawrence Carin -
2019 Poster: On Fenchel Mini-Max Learning »
Chenyang Tao · Liqun Chen · Shuyang Dai · Junya Chen · Ke Bai · Dong Wang · Jianfeng Feng · Wenlian Lu · Georgiy Bobashev · Lawrence Carin -
2018 : Katherine Heller »
Katherine Heller -
2018 Poster: Adversarial Text Generation via Feature-Mover's Distance »
Liqun Chen · Shuyang Dai · Chenyang Tao · Haichao Zhang · Zhe Gan · Dinghan Shen · Yizhe Zhang · Guoyin Wang · Dinghan Shen · Lawrence Carin -
2018 Poster: Distilled Wasserstein Learning for Word Embedding and Topic Modeling »
Hongteng Xu · Wenlin Wang · Wei Liu · Lawrence Carin -
2018 Poster: Diffusion Maps for Textual Network Embedding »
Xinyuan Zhang · Yitong Li · Dinghan Shen · Lawrence Carin -
2018 Spotlight: Diffusion Maps for Textual Network Embedding »
Xinyuan Zhang · Yitong Li · Dinghan Shen · Lawrence Carin -
2017 : Panel: On the Foundations and Future of Approximate Inference »
David Blei · Zoubin Ghahramani · Katherine Heller · Tim Salimans · Max Welling · Matthew D. Hoffman -
2017 Spotlight: Targeting EEG/LFP Synchrony with Neural Nets »
Yitong Li · michael Murias · samantha Major · geraldine Dawson · Kafui Dzirasa · Lawrence Carin · David Carlson -
2017 Poster: Targeting EEG/LFP Synchrony with Neural Nets »
Yitong Li · michael Murias · samantha Major · geraldine Dawson · Kafui Dzirasa · Lawrence Carin · David Carlson -
2017 Poster: Triangle Generative Adversarial Networks »
Zhe Gan · Liqun Chen · Weiyao Wang · Yuchen Pu · Yizhe Zhang · Hao Liu · Chunyuan Li · Lawrence Carin -
2017 Poster: ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching »
Chunyuan Li · Hao Liu · Changyou Chen · Yuchen Pu · Liqun Chen · Ricardo Henao · Lawrence Carin -
2017 Poster: VAE Learning via Stein Variational Gradient Descent »
Yuchen Pu · Zhe Gan · Ricardo Henao · Chunyuan Li · Shaobo Han · Lawrence Carin -
2017 Poster: Deconvolutional Paragraph Representation Learning »
Yizhe Zhang · Dinghan Shen · Guoyin Wang · Zhe Gan · Ricardo Henao · Lawrence Carin -
2017 Poster: Adversarial Symmetric Variational Autoencoder »
Yuchen Pu · Weiyao Wang · Ricardo Henao · Liqun Chen · Zhe Gan · Chunyuan Li · Lawrence Carin -
2017 Poster: A Probabilistic Framework for Nonlinearities in Stochastic Neural Networks »
Qinliang Su · xuejun Liao · Lawrence Carin -
2017 Poster: Scalable Model Selection for Belief Networks »
Zhao Song · Yusuke Muraoka · Ryohei Fujimaki · Lawrence Carin -
2017 Poster: Cross-Spectral Factor Analysis »
Neil Gallagher · Kyle Ulrich · Austin Talbot · Kafui Dzirasa · Lawrence Carin · David Carlson -
2016 Poster: Towards Unifying Hamiltonian Monte Carlo and Slice Sampling »
Yizhe Zhang · Xiangyu Wang · Changyou Chen · Ricardo Henao · Kai Fan · Lawrence Carin -
2016 Poster: Variational Autoencoder for Deep Learning of Images, Labels and Captions »
Yunchen Pu · Zhe Gan · Ricardo Henao · Xin Yuan · Chunyuan Li · Andrew Stevens · Lawrence Carin -
2016 Poster: Linear Feature Encoding for Reinforcement Learning »
Zhao Song · Ronald Parr · Xuejun Liao · Lawrence Carin -
2016 Poster: Stochastic Gradient MCMC with Stale Gradients »
Changyou Chen · Nan Ding · Chunyuan Li · Yizhe Zhang · Lawrence Carin -
2015 Poster: Parallelizing MCMC with Random Partition Trees »
Xiangyu Wang · Fangjian Guo · Katherine Heller · David B Dunson -
2015 Poster: GP Kernels for Cross-Spectrum Analysis »
Kyle R Ulrich · David Carlson · Kafui Dzirasa · Lawrence Carin -
2015 Poster: Deep Poisson Factor Modeling »
Ricardo Henao · Zhe Gan · James Lu · Lawrence Carin -
2015 Poster: Preconditioned Spectral Descent for Deep Learning »
David Carlson · Edo Collins · Ya-Ping Hsieh · Lawrence Carin · Volkan Cevher -
2015 Poster: Large-Scale Bayesian Multi-Label Learning via Topic-Based Label Embeddings »
Piyush Rai · Changwei Hu · Ricardo Henao · Lawrence Carin -
2015 Spotlight: Large-Scale Bayesian Multi-Label Learning via Topic-Based Label Embeddings »
Piyush Rai · Changwei Hu · Ricardo Henao · Lawrence Carin -
2015 Poster: Fast Second Order Stochastic Backpropagation for Variational Inference »
Kai Fan · Ziteng Wang · Jeff Beck · James Kwok · Katherine Heller -
2015 Poster: On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators »
Changyou Chen · Nan Ding · Lawrence Carin -
2015 Poster: Deep Temporal Sigmoid Belief Networks for Sequence Modeling »
Zhe Gan · Chunyuan Li · Ricardo Henao · David Carlson · Lawrence Carin -
2014 Poster: Analysis of Brain States from Multi-Region LFP Time-Series »
Kyle R Ulrich · David Carlson · Wenzhao Lian · Jana S Borg · Kafui Dzirasa · Lawrence Carin -
2014 Poster: Bayesian Nonlinear Support Vector Machines and Discriminative Factor Modeling »
Ricardo Henao · Xin Yuan · Lawrence Carin -
2014 Poster: Compressive Sensing of Signals from a GMM with Sparse Precision Matrices »
Jianbo Yang · Xuejun Liao · Minhua Chen · Lawrence Carin -
2014 Poster: On the relations of LFPs & Neural Spike Trains »
David Carlson · Jana Schaich Borg · Kafui Dzirasa · Lawrence Carin -
2014 Poster: Dynamic Rank Factor Model for Text Streams »
Shaobo Han · Lin Du · Esther Salazar · Lawrence Carin -
2013 Poster: Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture »
Trevor Campbell · Miao Liu · Brian Kulis · Jonathan How · Lawrence Carin -
2013 Poster: Designed Measurements for Vector Count Data »
Liming Wang · David Carlson · Miguel Rodrigues · David Wilcox · Robert Calderbank · Lawrence Carin -
2013 Poster: Integrated Non-Factorized Variational Inference »
Shaobo Han · Xuejun Liao · Lawrence Carin -
2013 Poster: Real-Time Inference for a Gamma Process Model of Neural Spiking »
David Carlson · Vinayak Rao · Joshua T Vogelstein · Lawrence Carin -
2013 Poster: Efficient Algorithm for Privately Releasing Smooth Queries »
Ziteng Wang · Kai Fan · Jiaqi Zhang · Liwei Wang -
2012 Workshop: Bayesian Nonparametric Models For Reliable Planning And Decision-Making Under Uncertainty »
Jonathan How · Lawrence Carin · John Fisher III · Michael Jordan · Alborz Geramifard -
2012 Poster: Joint Modeling of a Matrix with Associated Text via Latent Binary Features »
XianXing Zhang · Lawrence Carin -
2012 Poster: Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models »
Jeff Beck · Katherine Heller · Alexandre Pouget -
2012 Spotlight: Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models »
Jeff Beck · Katherine Heller · Alexandre Pouget -
2012 Poster: Augment-and-Conquer Negative Binomial Processes »
Mingyuan Zhou · Lawrence Carin -
2012 Poster: Modelling Reciprocating Relationships with Hawkes processes »
Charles Blundell · Katherine Heller · Jeff Beck -
2012 Spotlight: Augment-and-Conquer Negative Binomial Processes »
Mingyuan Zhou · Lawrence Carin -
2012 Session: Oral Session 7 »
Katherine Heller -
2012 Spotlight: Modelling Reciprocating Relationships with Hawkes processes »
Charles Blundell · Katherine Heller · Jeff Beck -
2011 Poster: On the Analysis of Multi-Channel Neural Spike Data »
Bo Chen · David Carlson · Lawrence Carin -
2011 Poster: Testing a Bayesian Measure of Representativeness Using a Large Image Database »
Joshua T Abbott · Katherine Heller · Zoubin Ghahramani · Tom Griffiths -
2011 Session: Oral Session 7 »
Katherine Heller -
2011 Poster: The Kernel Beta Process »
Lu Ren · Yingjian Wang · David B Dunson · Lawrence Carin -
2011 Spotlight: The Kernel Beta Process »
Lu Ren · Yingjian Wang · David B Dunson · Lawrence Carin -
2011 Poster: Hierarchical Topic Modeling for Analysis of Time-Evolving Personal Choices »
XianXing Zhang · David B Dunson · Lawrence Carin -
2010 Session: Spotlights Session 9 »
Katherine Heller -
2010 Session: Oral Session 11 »
Katherine Heller -
2010 Poster: Joint Analysis of Time-Evolving Binary Matrices and Associated Documents »
Eric X Wang · Dehong Liu · Jorge G Silva · David B Dunson · Lawrence Carin -
2009 Workshop: Applications for Topic Models: Text and Beyond »
David Blei · Jordan Boyd-Graber · Jonathan Chang · Katherine Heller · Hanna Wallach -
2009 Poster: A Bayesian Model for Simultaneous Image Clustering, Annotation and Object Segmentation »
Lan Du · Lu Ren · David B Dunson · Lawrence Carin -
2009 Poster: Hierarchical Learning of Dimensional Biases in Human Categorization »
Katherine Heller · Adam Sanborn · Nick Chater -
2009 Spotlight: Hierarchical Learning of Dimensional Biases in Human Categorization »
Katherine Heller · Adam Sanborn · Nick Chater -
2009 Poster: Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations »
Mingyuan Zhou · Haojun Chen · John Paisley · Lu Ren · Guillermo Sapiro · Lawrence Carin -
2009 Poster: Learning to Explore and Exploit in POMDPs »
Chenghui Cai · Xuejun Liao · Lawrence Carin -
2008 Workshop: Cost Sensitive Learning »
Balaji R Krishnapuram · Shipeng Yu · Oksana Yakhnenko · R. Bharat Rao · Lawrence Carin -
2008 Poster: Bayesian Exponential Family PCA »
Shakir Mohamed · Katherine Heller · Zoubin Ghahramani -
2008 Spotlight: Bayesian Exponential Family PCA »
Shakir Mohamed · Katherine Heller · Zoubin Ghahramani -
2007 Poster: Semi-Supervised Multitask Learning »
Qiuhua Liu · Xuejun Liao · Lawrence Carin -
2007 Spotlight: Semi-Supervised Multitask Learning »
Qiuhua Liu · Xuejun Liao · Lawrence Carin