Perceptio: Perception-Enhanced Vision-Language Models via Spatial Token Generation

Large Vision–Language Models (LVLMs) excel at semantic understanding but struggle with fine-grained spatial grounding, as the model must implicitly infer complex geometry without ever producing a spatial interpretation. We present \textit{Perceptio}, a perception-enhanced LVLM with 2D–3D spatial reasoning abilities, enabled via explicit semantic-segmentation tokens and depth tokens generated directly within the autoregressive sequence. Concretely, we (i) distill a VQ-VAE depth codebook from a strong monocular teacher to tokenize dense depth into compact sequences, and (ii) integrate SAM2-based semantic-segmentation tokens and VQ-VAE depth tokens inside the LLM so the model first emits spatial tokens and then answers. To stabilize depth token generation, we introduce novel composite depth-token objectives (marker, token, and count losses) and a soft-merging technique for differentiable reconstruction. We adopt a multi-task co-training strategy across diverse datasets, letting the model learn perception tokens to tackle multiple downstream tasks. Building on InternVL, Perceptio achieves state-of-the-art performance across benchmarks: improving referring expression segmentation by +0.8/+1.4/+1.1 cIoU on RefCOCO/+/g, HardBLINK spatial understanding accuracy by 10.3%, and MMBench accuracy by 1.0%, demonstrating that explicit spatial chain-of-thought materially strengthens spatial grounding in LVLMs.