Can Small Transformers Learn the Language of Coxeter Groups?

The field of mechanistic interpretability is well-suited to collaborative projects between mathematicians and AI researchers. This work describes one such project. We trained a small transformer to learn the language of the Coxeter group $\widetilde{A}_2$, best known as the symmetries of the tiling of the Euclidean plane by equilateral triangles. We then formulated and proved a word-theoretic conjecture about $\widetilde{A}_2$ after observing the attention patterns of the trained model.