To Infinite and Beyond: How Neural Networks Learn Compact Lie Groups, a Mechanistic Study

How neural networks learn group operations is well understood for finite groups, where small models trained on a multiplication table discover the group's irreducible representations. Whether the same structure emerges for continuous groups, which have infinitely many elements and no multiplication table, has remained open. We train small networks to identify products in compact Lie groups such as $\operatorname{SU}(2)$ and $\operatorname{SU}(3)$, presenting each element through a fixed encoding, and we analyze the representations they learn. Across nine group forms obtained by varying the centers of four compact groups, the trained features concentrate on the matrix coefficients of a smallest faithful representation of the exact group underlying the data, robustly across seeds, so the internals distinguish groups that share a Lie algebra, such as $\operatorname{SU}(2)$ from $\operatorname{SO}(3)$. We support this account with ablations, activation patching, fitted translation operators, and a new two-sided translation test, and we find that the learned features occupy the complete matrix-coefficient space of the selected representation at lower parameter cost than smaller faithful alternatives. Preliminary follow-up studies appear in the appendix.