Verifying Persistent Abstractions in Adaptive AI Scientists
Abstract
AI systems for scientific discovery increasingly generate hypotheses, programs, and intermediate representations with limited human intervention. Most verification methods, however, focus on final outputs. We argue that this is insufficient when AI systems create persistent abstractions that are reused in later reasoning. Unlike an incorrect final hypothesis, an incorrect persistent abstraction can become a shared dependency of many downstream hypotheses, producing correlated failures. We call this an unverified abstraction cascade. We propose that verification effort should scale with both uncertainty and an abstraction’s blast radius, or its expected downstream impact. High-impact abstractions should therefore undergo verification before unrestricted use. We use scientific equation discovery as a concrete test domain and outline experiments comparing unrestricted abstraction admission, final-output verification, and admission-gated verification. Our claim is that if persistent wrong abstractions do not increase downstream error correlation or recovery cost, abstraction-level verification provides little additional value.