Sparse Biological Features Reveal Early Functional Commitment in Diffusion Protein Language Models
Abstract
Generative protein models are increasingly used for functional sequence design, yet the process by which biological information is organized during generation remains poorly understood. Diffusion protein language models provide a tractable setting for this question, as sequence generation proceeds through an explicit denoising trajectory from highly corrupted states to complete proteins. This trajectory enables a temporal view of interpretability: beyond identifying which biological signals are represented, it allows us to examine when these signals emerge and when specific residues become committed. Here, we analyze DPLM representations using sparse autoencoders trained across layers and noise levels. The resulting features capture biologically meaningful signals at both residue and protein scales, align with functional annotations, and preserve downstream biological information under reconstruction. Following these features along the denoising trajectory reveals a consistent functional ordering: catalytic-enriched features pre-activate at still-masked catalytic positions before residue identity is resolved, and catalytic residues are subsequently recovered earlier by the iterative denoiser. This prioritization remains significant after controlling for prediction difficulty, sequence context, amino-acid identity, structural environment, and evolutionary conservation, and is not reproduced by a random-feature null. Across ProteinGym deep-mutational-scanning assays, residues recovered earlier during denoising are also more mutation-sensitive. These results reveal a temporal organization of biological information in diffusion-based protein generation, in which functionally important residues are not only represented, but preferentially committed during sequence formation.