Language, Not Code Abstraction: Natural-Language Training Shapes Brain-Aligned Geometry

Do language-brain scores reflect natural-language learning, or can abstraction learned from code produce the same effect? We train 25 GPT-2-small models, five seeds each on TinyStories, Python, C, assembly, and raw binary, while fixing the architecture, tokenizer, optimization, nominal token budget, and neural readout. Three matched random-weight models establish what the shared pipeline provides before transformer training. On Pereira-243 and Tuckute2024,natural-language training raises mean alignment above random weights by $\Deltar=0.06$ and $0.05$ and above the four-code mean by $0.05$ and $0.06$; code training changes alignment by $0.00$ and $-0.01$. The boundary appears at 11 of 12 blocks on both Pereira subsets, while code corpora show no stable ordering from source code to assembly and binary. Geometry mirrors this result: the within- versus between-corpus CKA gap reaches $0.39$ by block 4, and fit-free brain-model RSA peaks at $\rho=0.14$ for TinyStories while every code corpus peaks at or below $0.07$. Binary is highly seed-consistent but collapses to a near-two-dimensional manifold, showing that reproducibility alone is not brain alignment. English embedding adaptation improves every code receiver by$\Delta r=0.02$, whereas matched own-corpus adaptation does not; deterministic pseudoword substitution preserves $r=0.13$. Natural-language training therefore links a distinct representation geometry to neural alignment, and absolute encoding scores require a matched random-weight reference \footnote{Code, datasets, and trained checkpoints to be released upon acceptance.}.