`

Timezone: »

 
Spotlight
Telescoping Density-Ratio Estimation
Benjamin Rhodes · Kai Xu · Michael Gutmann

Thu Dec 10 07:20 AM -- 07:30 AM (PST) @ Orals & Spotlights: Unsupervised/Probabilistic
in Orals & Spotlights Track 27: Unsupervised/Probabilistic »

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.

[ Paper ]

Author Information

Benjamin Rhodes (University of Edinburgh)
Kai Xu (University of Edinburgh)
Michael Gutmann (University of Edinburgh)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors