Graphical models are widely used in science to represent joint probability distributions with an underlying conditional dependence structure. The inverse problem of learning a discrete graphical model given i.i.d samples from its joint distribution can be solved with near-optimal sample complexity using a convex optimization method known as Generalized Regularized Interaction Screening Estimator (GRISE). But the computational cost of GRISE becomes prohibitive when the energy function of the true graphical model has higher order terms. We introduce NeurISE, a neural net based algorithm for graphical model learning, to tackle this limitation of GRISE. We use neural nets as function approximators in an Interaction Screening objective function. The optimization of this objective then produces a neural-net representation for the conditionals of the graphical model. NeurISE algorithm is seen to be a better alternative to GRISE when the energy function of the true model has a high order with a high degree of symmetry. In these cases NeurISE is able to find the correct parsimonious representation for the conditionals without being fed any prior information about the true model. NeurISE can also be used to learn the underlying structure of the true model with some simple modifications to its training procedure. In addition, we also show a variant of NeurISE that can be used to learn a neural net representation for the full energy function of the true model.
Abhijith Jayakumar (Indian Institute of Science)
Andrey Lokhov (Los Alamos National Laboratory)
Sidhant Misra (Los Alamos National Laboratory)
Marc Vuffray (Los Alamos National Laboratory)
I’m a staff research scientist in the Theoretical Division at the Los Alamos National Laboratory (LANL), New Mexico, where I am part of the Advanced Network Science Initiative (ANSI) as well as the Condensed Matter and Complex Systems Group (T-4). My background is in statistical physics and information theory. My current work focuses on the design of machine learning techniques for learning probabilistic networks and on the development of new methods to control and optimize energy networks under uncertainty.
More from the Same Authors
2020 Poster: Efficient Learning of Discrete Graphical Models »
Marc Vuffray · Sidhant Misra · Andrey Lokhov
2019 Demonstration: Realtime Modeling and Anomaly Detection in Multivariate Data Streams »
Christopher Hannon · Andrey Lokhov · Deep Deka
2016 Poster: Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models »
Marc Vuffray · Sidhant Misra · Andrey Lokhov · Michael Chertkov