Representational Identifiability Limits Action Selection in a Protein-Design Agent
Saanvi S Subramanian
Abstract
Autonomous AI agents for biological discovery can generate plausible but incorrect hypotheses. Before evaluating whether a hypothesis is correct, we ask whether the internal representation used by the agent can distinguish the candidate actions it must choose among. We study this problem in a hypothesize--simulate--validate loop for mutational-driver discovery using the ESM-2 protein language model and the COSMIC catalogue of somatic cancer mutation profiles. We define \emph{action-space identifiability} as the extent to which candidate actions produce distinguishable destinations in representation space. Across all 3,003 pairs of the 78 COSMIC signatures, biologically distinct mutational processes often reach similar destinations. A composition-matched control shows that ESM-2 separates marginal base-substitution composition more strongly than the trinucleotide context that distinguishes mutational processes. Measurement choices also substantially affect apparent separation. Stop-at-first-stop translation truncates $25.8\%$ of simulated mutants to one-residue placeholders, and placeholder count explains up to $R^2=0.870$ of pairwise destination distance. After controlling for sequence length, the fraction below the same threshold is strongly model-dependent: $23.8\%$ in ESM-2 8M and $72.5\%$ in ESM-2 650M. We do not select one scale as a catalogue-wide collapse estimate. Finally, direct optimization reduces target distance beyond the range observed for natural proteins. Both optimization and validation use ESM-2 representations, so this result also shows that validation in the same representation provides limited independent evidence. Together, these results demonstrate that scientific-agent evaluations should test whether the representation resolves the biological distinction claimed by the agent and whether validation remains informative under optimization.
Chat is not available.
Successful Page Load