Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). The filler tokens outwardly carry no information, leaving open what, if anything, is represented in the model’s internal states and how those states contribute to the reasoning. We study this question on four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. Compared to the same model reasoning with chain-of-thought, intermediates are not encoded one at a time in the order they would be written. The model instead moves back and forth between them or works through them by layer depth, depending on the task. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80–95\% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Together, these results show that filler-token computation has structured, causally relevant internal organization that can be largely recovered from the residual stream even when the generated tokens themselves reveal nothing about the reasoning. Thus, on these tasks, failure of behavioral CoT monitoring does not imply that the underlying computation is inaccessible to interpretability methods.