Disentanglement as Identifiable Pushforward Factorisation
Carl Allen
Abstract
We characterise disentanglement for smooth generative pushforward models, such as in VAEs and GANs. For a generator/decoder $g:\\mathcal{Z}\\to\\mathcal{X}$ and factorised prior $p(z)=\\prod_i p_i(z_i)$, we define disentanglement as standard statistical independence expressed over the generated manifold: {the pushforward density} $p_\\mu = g_\\#p$ factorises into one–dimensional "seam" factors, each controlled by a distinct latent coordinate. We prove that $p_\\mu$ factorises according to the SVD of $g$'s Jacobian; that disentanglement equates to two conditions on $g$ (C1-C2); and that under such conditions the seam factors are identifiable, up to permutation and sign. In the special case of Gaussian ($\\beta$-)VAEs, we show via an identity how diagonal posteriors promote C1-C2, in expectation, explaining why disentanglement arises modulated by $\\beta$. Experiments illustrate this mechanism on Gaussian data, dSprites, and CelebA.
Chat is not available.
Successful Page Load