Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

A paraphrase pair is a small piece of evidence that two activations should agree. We treat that as a sheaf condition and ask what its cohomology looks like at scale. A cellular sheaf over a paraphrase graph defines a coboundary δ⁰; its kernel is the space of activations that paste consistently across the graph, and the rest of the sheaf Laplacian's spectrum reads off how badly they fail to. On a matching graph this collapses to projected within-pair covariance, so the framework only earns its keep once depth supplies cycles: stacking the same activations across the transformer's own layers turns the construction into a 2-D grid with b₁ = N(L−1) algebraic 1-cycles, one per elementary square asking whether layer-l → l+1 commutes with paraphrase equivalence. The model-derived Hodge harmonic mass at L=2 varies by four orders of magnitude across architectures (from 10⁻⁴ on Mistral-7B and 10⁻³ on Llama-3-8B to 0.08 on Llama-2-7B), and the ordering by harmonic mass matches the ordering by steering fragility we measure independently. The framework supplies a mechanistic prediction that variance-matched and probe baselines do not. Three validations follow. (i) Across nine architectures from 124M to 13B, the spectral complement of LF carries 5.6 to 26.5× the causal influence of variance-matched controls (p < 10⁻¹⁵). (ii) On held-out CounterFact across eight architectures, sheaf H⁰ at 20 dimensions beats LEACE's preserved subspace at full hidden dimension: 85.6% vs. 42.7% on GPT-2, 87.1% vs. 77.7% on Mistral-7B, mean gap +17.9 pp. (iii) On Llama-2-7B, contrastive sheaf steering preserves 7.4× more facts than random under style transfer (31.0% vs. 4.2%, n=1000, McNemar p < 10⁻⁵⁰). The same operators read off representational collapse: tr(LF) − β log det Cov reproduces VICReg's invariance + variance + covariance recipe in coordinates and holds encoder rank at 57–60/64 where naive consistency collapses to 14–42/64 across seven architectures.