Magnitude-preserving Layers Enable Efficient GANs

GANs are appealing because they generate sharp images in a single forward pass. Recent works have stabilized GAN training, but FID may still plateau on diverse datasets like ImageNet-256 because the discriminator's activation magnitudes increase through training. We trace this pathology to the discriminator's incentive to sharpen its decision boundary by rescaling activations rather than by finding better features. Then, building on R3GAN, we treat magnitude preservation as a first-class architectural target across the network: $\ell_2$ weight normalization with centering, magnitude-preserving LReLU and residuals, forced post-update normalization, and feed-forward classifier heads. Across a roadmap of configurations, we perform diagnostic ablations to clarify what does and does not work and why. The resulting models reach FIDs of 2.20 @ 17 M parameters, 1.63 @ 60 M, and 1.52 @ 130 M on class-conditional ImageNet-256. At 1 NFE, this is substantially more efficient than competing diffusion and autoregressive methods, and than prior GAN baselines too (e.g., 10$\times$ more efficient than StyleGAN-XL for similar FID). In sum, our work presents a new SOTA in FID/parameter efficiency derived from simple adversarial training with well-behaved magnitude preserving layers.