Breaking Curse of Dimensionality for Mutual Information Estimation with Vine Copulas

We propose a computationally efficient vine copula-based mutual information (MI) estimator. Unlike existing non-parametric density estimators that suffer from the curse of dimensionality, a non-parametric vine copula has a convergence rate that is independent of the dimensionality of data. We leverage this property to tackle the challenging task of MI estimation and propose an interpretable MI estimator. Extensive experiments on datasets with known ground-truth MI values across dimensions, data types, and (input, output) dependence structures demonstrate a superior trade-off between MI estimation error and computational time compared to SotA neural MI estimators.