RB-LDC: Redundancy-Balanced Latent Coding for Robust Diffusion
Hyunseok Jeong ⋅ Jaeho Jeon ⋅ Young-Sik Kim
Abstract
Latent diffusion models (LDMs) achieve high-fidelity generation by moving diffusion from pixels to a compact latent space, but standard tokenizers are optimized for compression, not robustness under diffusion-time corruption. We formulate latent tokenization as diffusion-aware joint source--channel coding and diagnose standard LDM pipelines as missing an explicit channel-coding step. A diffusion-aware rate--robustness functional admits an explicit spectral characterization in $G^\top G$, yielding the Redundancy-Balance Principle: optimal redundancy is not uniform, but aligns with semantic sensitivity and equalizes marginal integrated-risk reduction across active directions. The resulting optima take weighted tight-frame and water-filling forms; isotropic bottlenecks are minimax-suboptimal under heterogeneous sensitivity, and a group-conditioned extension identifies rare groups as capacity-limiting under a fixed latent budget. We instantiate the theory as RB-LDC (Redundancy-Balanced Latent-Diffusion Coding), a lightweight tokenizer modification that inserts a learned coding layer before diffusion. Six controlled experiments validate the theorem sequence up to $k{=}1024$, with a $53.5\times$ alignment gap, $659\times$ minimax gap, and real-image VAE learnability via Adam recovery of the KKT spectrum with relative error below 0.5 percent. On SD-VAE and VA-VAE at ImageNet-256, RB-LDC outperforms isotropic uniform redundancy in all 20 perturbed metric cells over a 15-SNR grid; VA-VAE block-local obtains $-15.40$ average pFID and $-105.79$ peak pFID at $\log_{10}\rho{=}+1$, while preserving clean rFID parity (gap $-0.0001$). RB-LDC turns latent tokenizer design into a reliability problem for diffusion models.
Chat is not available.
Successful Page Load