Out-of-training properties in frozen interatomic potential embeddings are decodable and ordered by physical scale

Foundation models in language and proteins encode layer-resolved representations that extend far beyond their training objectives. We ask whether the same holds for machine-learned interatomic potentials (MLIPs) — foundation models trained on DFT energies, forces and stresses — and where in the network that information lives. We probe the frozen, layer-wise embeddings of two architecturally opposite models, ORB-v3 (non-equivariant) and UMA-S (SO(3)-equivariant), across 154,879 Materials Project structures and nine properties. Both models' embeddings decode five properties absent from the training objective — band gap, bulk modulus, magnetic moment density, metal/insulator and magnetic character. Against a Magpie composition-only baseline the embeddings add a statistically significant gain for every target, cutting the prediction error in physical units by 35–73 % for ORB and 41–76 % for UMA, except on the magnetic pair and crystal system. The information is organised by physical scale: composition-linked properties are readable at the input, while properties that depend on long-range atomic arrangement are assembled through message passing. UMA's equivariant angular channels give it a clear advantage on crystal-system classification, which depends on coordination geometry. Supervised decodability is uniformly higher than unsupervised geometric organisation, and clustering recovers element identity better than property. The best probes carry an expected error (RMSE) of 0.50 / 0.46 eV on band gap and 17 / 16 GPa on bulk modulus (ORB / UMA) — enough to rank and filter candidates for screening.