Timezone: »
Normalizing flows have shown great success as general-purpose density estimators. However, many real-world applications require the use of domain-specific knowledge, which normalizing flows cannot readily incorporate. We propose embedded-model flows (EMF), which alternate general-purpose transformations with structured layers that embed domain-specific inductive biases. These layers are automatically constructed by converting user-specified differentiable probabilistic models into equivalent bijective transformations. We also introduce gated structured layers, which allow bypassing the parts of the models that fail to capture the statistics of the data. We demonstrate that EMFs can be used to induce desirable properties such as multimodality, hierarchical coupling and continuity. Furthermore, we show that EMFs enable a high-performance form of variational inference where the structure of the prior model is embedded in the variational architecture. In our experiments, we show that this approach outperforms state-of-the-art methods in common structured inference problems.
Author Information
Gianluigi Silvestri (OnePlanet Research Center)
Emily Fertig (Google Research)
Dave Moore (Google AI)
Luca Ambrogioni (Radboud University)
More from the Same Authors
-
2023 Poster: Spontaneous symmetry breaking in generative diffusion models »
Gabriel Raya · Luca Ambrogioni -
2020 Poster: GAIT-prop: A biologically plausible learning rule derived from backpropagation of error »
Nasir Ahmad · Marcel A. J. van Gerven · Luca Ambrogioni -
2020 Spotlight: GAIT-prop: A biologically plausible learning rule derived from backpropagation of error »
Nasir Ahmad · Marcel A. J. van Gerven · Luca Ambrogioni -
2019 Workshop: Program Transformations for ML »
Pascal Lamblin · Atilim Gunes Baydin · Alexander Wiltschko · Bart van MerriĆ«nboer · Emily Fertig · Barak Pearlmutter · David Duvenaud · Laurent Hascoet -
2019 Poster: Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift »
Jasper Snoek · Yaniv Ovadia · Emily Fertig · Balaji Lakshminarayanan · Sebastian Nowozin · D. Sculley · Joshua Dillon · Jie Ren · Zachary Nado -
2019 Poster: Likelihood Ratios for Out-of-Distribution Detection »
Jie Ren · Peter Liu · Emily Fertig · Jasper Snoek · Ryan Poplin · Mark Depristo · Joshua Dillon · Balaji Lakshminarayanan -
2015 Poster: Gaussian Process Random Fields »
Dave Moore · Stuart J Russell