Can a Transformer Run Hartree-Fock? Compiling Self-Consistent Field Theory into Recurrent Transformer Weights

Neural networks are increasingly used to approximate electronic-structure calculations by predicting Hamiltonians, Fock matrices, electron densities, density matrices, density functionals, or learned self-consistent maps. We investigate a deliberately different question: can the numerical self-consistent field (SCF) algorithm itself be represented by transformer weights, without training? We construct a proof-of-concept restricted Hartree-Fock (RHF) solver in which arithmetic operations and control decisions are compiled deterministically into transformer parameters. We formulate SCF as a recurrent continuous-state transformer computation, analogous to autoregressive generation but with density matrices and solver state replacing discrete tokens. We replace the eigendecomposition normally performed in every SCF iteration with second-order spectral projection (SP2), which obtains the occupied-space projector through repeated matrix multiplication, trace evaluation, comparison, and polynomial selection. Across all 7,165 QM7 molecules, the compiled single-step calculation retains small numerical errors, and intervention-free recurrent rollouts converge for all ten molecules tested. These results show that a scientific fixed-point algorithm can be compiled into standard transformer operations without fitting model parameters to data.