Timezone: »
A deep learning approach to recover conditional independence graphs
Harsh Shrivastava · Urszula Chajewska · Robin Abraham · Xinshi Chen
Event URL: https://openreview.net/forum?id=kEwzoI3Am4c »
Probabilistic Graphical Models are generative models of complex systems. They rely on conditional independence assumptions between variables to learn sparse representations which can be visualized in a form of a graph. Such models are used for domain exploration and structure discovery in poorly understood domains. This work introduces a novel technique to perform sparse graph recovery by optimizing deep unrolled networks. Assuming that the input data $X\in\mathbb{R}^{M\times D}$ comes from an underlying multivariate Gaussian distribution, we apply a deep model on $X$ that outputs the precision matrix $\Theta$. Then, the partial correlation matrix `$\rho$' is calculated which can also be interpreted as the conditional independence graph. Our model, \texttt{uGLAD}, builds upon and extends the state-of-the-art model \texttt{GLAD} to the unsupervised setting. The key benefits of our model are (1) \texttt{uGLAD} automatically optimizes sparsity-related regularization parameters leading to better performance than existing algorithms. (2) We introduce multi-task learning based `consensus' strategy for robust handling of missing data in an unsupervised setting. We evaluate performance on synthetic Gaussian, non-Gaussian data generated from Gene Regulatory Networks, and present a case study in anaerobic digestion.
Probabilistic Graphical Models are generative models of complex systems. They rely on conditional independence assumptions between variables to learn sparse representations which can be visualized in a form of a graph. Such models are used for domain exploration and structure discovery in poorly understood domains. This work introduces a novel technique to perform sparse graph recovery by optimizing deep unrolled networks. Assuming that the input data $X\in\mathbb{R}^{M\times D}$ comes from an underlying multivariate Gaussian distribution, we apply a deep model on $X$ that outputs the precision matrix $\Theta$. Then, the partial correlation matrix `$\rho$' is calculated which can also be interpreted as the conditional independence graph. Our model, \texttt{uGLAD}, builds upon and extends the state-of-the-art model \texttt{GLAD} to the unsupervised setting. The key benefits of our model are (1) \texttt{uGLAD} automatically optimizes sparsity-related regularization parameters leading to better performance than existing algorithms. (2) We introduce multi-task learning based `consensus' strategy for robust handling of missing data in an unsupervised setting. We evaluate performance on synthetic Gaussian, non-Gaussian data generated from Gene Regulatory Networks, and present a case study in anaerobic digestion.
Author Information
Harsh Shrivastava (Microsoft Research, Redmond)
Harsh Shrivastava is a Senior Applied Scientist in the AI Architecture and Strategy team at [Microsoft Research](https://www.microsoft.com/en-us/research/people/hshrivastava/). His research interests include working with probabilistic graphical models, using inductive biases to design deep learning architectures & high performance computing. Harsh holds a PhD in Machine Learning (CSE dept.) from Georgia Institute of Technology. He has an undergraduate (dual degree) in Electronics & ECE with Masters in VLSI circuits from Indian Institute of Technology Kharagpur.
Urszula Chajewska (Microsoft)
Robin Abraham (Microsoft)
Xinshi Chen (Georgia Institution of Technology)
More from the Same Authors
-
2021 Poster: Multi-task Learning of Order-Consistent Causal Graphs »
Xinshi Chen · Haoran Sun · Caleb Ellington · Eric Xing · Le Song -
2020 Poster: Understanding Deep Architecture with Reasoning Layer »
Xinshi Chen · Yufei Zhang · Christoph Reisinger · Le Song