dRAE: Representation Autoencoder with Hyper-Spherical Codes

Multi-modal models require visual tokenizers that jointly capture high semantic density and fine-grained structural fidelity. Representation Autoencoders (RAEs) offer a promising direction by decoding images directly from a pre-trained semantic space using high-dimensional continuous tokens, bypassing the information bottleneck of VAEs and benefiting both visual understanding and generation. In this work, we aim to discretize these continuous vision representations to bridge the gap with language models — a non-trivial challenge, as existing quantization methods suffer from codebook collapse, failing to scale while preserving semantic coherence. We identify the root cause as **metric mismatch**: standard Euclidean codebook objectives are fundamentally misaligned with the anisotropic geometry of representation space, leading to codebook embeddings with high-variance magnitude scales and uneven angular distributions that hinder scalability. To address this, we propose **Hyper-Spherical Quantization (HSQ)** , which decouples semantic content from feature magnitude via angular routing, preventing code assignment from being dominated by scale rather than meaning. The resulting **discrete Representation Autoencoder (dRAE)** achieves high-fidelity reconstruction while preserving semantic integrity and supporting scalable discrete latent spaces. Extensive experiments demonstrate consistent performance gains as the vocabulary size scales to $131,072$, along with strong unified performance across both visual understanding and generation benchmarks.