Stable but Hidden: Identifiability and Experimental Design under Missing Compositional Support
Harry Sevi ⋅ Joonatan Laulainen
Abstract
AI scientific pipelines increasingly rely on cheap predictors to route scarce verification measurements toward promising hypotheses; the verification budget is spent well only when the observations actually identify the target quantity. We isolate a verification failure mode that no amount of additional same-support data or ensemble agreement can resolve: when relevant components are never observed jointly, distinct response functions can agree on every observed composition while disagreeing on the missing joint region. The resulting unidentifiable directions form an invisible subspace $\mathcal{N}_T = \ker \mathcal{R}_T$. Its geometry determines zero-shot ambiguity and minimax prediction risk on the missing region; when $\dim \mathcal{N}_T = r < \infty$, a small number of joint measurements turns the verification problem into experimental design, and $I$-optimal measurements on $\mathcal{N}_T$ achieve minimax risk $\Theta(\sigma^2 r/m)$. We validate this picture on materials. Controlled simulations confirm the predicted sample-efficiency advantage of identifiability-directed design near the minimum budget needed to identify the interaction basis. On real electrocatalysis libraries, it selects substantially different experiments from Gaussian-process (GP) uncertainty and improves paired prediction error on $85\%$ of element pairs. On Materials Project, removing the relevant joint support degrades prediction on $90\%$ of pairs, whereas matched removals of other data of the same size and chemistry do not reproduce the effect, isolating a support-specific rather than data-volume deficit. Pair difficulty is reproducible across learners yet hidden from zero-label model disagreement; a few joint measurements recover enough information for pair-level routing. The practical takeaway for AI-guided scientific verification: the most informative experiment is not where a fitted model is most uncertain, but where the observations cannot identify the target interaction at all. A model can be uncertain about its prediction; the data can be insufficient to identify the answer itself.
Chat is not available.
Successful Page Load