Timezone: »
We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI, an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to hyperparameter choices and provides the uncertainty on the MI estimate due to the finite sample size. We demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation.
Author Information
Davide Piras (University of Geneva)

Postdoc in Geneva, working on machine learning applied to cosmology. Formerly at University College London.
Hiranya Peiris (University College London/Stockholm University)
Andrew Pontzen (University College London)
Luisa Lucie-Smith (Max Planck Institute for Astrophysics)
Brian Nord (Fermi National Accelerator Laboratory)
Ningyuan (Lillian) Guo (University College London)
More from the Same Authors
-
2021 : Robustness of deep learning algorithms in astronomy - galaxy morphology studies »
Aleksandra Ciprijanovic · Diana Kafkes · Gabriel Nathan Perdue · Sandeep Madireddy · Stefan Wild · Brian Nord -
2021 : DeepZipper: A Novel Deep Learning Architecture for Lensed Supernovae Identification »
Robert Morgan · Brian Nord -
2021 : Error Analysis of Kilonova Surrogate Models »
Kamile Lukosiute · Brian Nord -
2022 : Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection »
Aleksandra Ciprijanovic · Ashia Lewis · Kevin Pedro · Sandeep Madireddy · Brian Nord · Gabriel Nathan Perdue · Stefan Wild -
2022 : Neural Inference of Gaussian Processes for Time Series Data of Quasars »
Egor Danilov · Aleksandra Ciprijanovic · Brian Nord -
2022 : DIGS: Deep Inference of Galaxy Spectra with Neural Posterior Estimation »
Gourav Khullar · Brian Nord · Aleksandra Ciprijanovic · Jason Poh · Fei Xu · Ashwin Samudre -
2022 : Strong Lensing Parameter Estimation on Ground-Based Imaging Data Using Simulation-Based Inference »
Jason Poh · Ashwin Samudre · Aleksandra Ciprijanovic · Brian Nord · Joshua Frieman · Gourav Khullar -
2023 Workshop: NeurIPS 2023 Workshop: Machine Learning and the Physical Sciences »
Brian Nord · Atilim Gunes Baydin · Adji Bousso Dieng · Emine Kucukbenli · Siddharth Mishra-Sharma · Benjamin Nachman · Kyle Cranmer · Gilles Louppe · Savannah Thais -
2022 : Invited talk: Hiranya Peiris, "Prospects for understanding the physics of the Universe" »
Hiranya Peiris · Siddharth Mishra-Sharma -
2022 Workshop: Machine Learning and the Physical Sciences »
Atilim Gunes Baydin · Adji Bousso Dieng · Emine Kucukbenli · Gilles Louppe · Siddharth Mishra-Sharma · Benjamin Nachman · Brian Nord · Savannah Thais · Anima Anandkumar · Kyle Cranmer · Lenka Zdeborová · Rianne van den Berg -
2021 Workshop: Machine Learning and the Physical Sciences »
Anima Anandkumar · Kyle Cranmer · Mr. Prabhat · Lenka Zdeborová · Atilim Gunes Baydin · Juan Carrasquilla · Emine Kucukbenli · Gilles Louppe · Benjamin Nachman · Brian Nord · Savannah Thais -
2020 Workshop: Machine Learning and the Physical Sciences »
Anima Anandkumar · Kyle Cranmer · Shirley Ho · Mr. Prabhat · Lenka Zdeborová · Atilim Gunes Baydin · Juan Carrasquilla · Adji Bousso Dieng · Karthik Kashinath · Gilles Louppe · Brian Nord · Michela Paganini · Savannah Thais