Evidence-Disjoint Evaluation of Reliability-Aware Source Selection in Biological Agents

Artificial intelligence (AI) agents must evaluate conflicting evidence from tools and decide which sources to trust. Yet evaluations that expose the target answer directly through a tool may reward retrieval without requiring agents to assess source reliability. We introduce an evidence-disjoint protocol to remove every record directly linking a query to its answer. In a gene-partner ranking task, this leaves two evidence sources whose informativeness varies across queries: masked curated annotations and image-derived measurements of cell morphology following genetic perturbation. Across two commercial large language models, providing source reliability estimates calibrated on separate queries improves source selection, with mean reciprocal rank gains of 0.222–0.232 in a prospectively selected evaluation gene set. Benefits vary between models, with no detectable performance loss when either already favors the more reliable source. In a separate experiment with one model, prompting it to assess each source without supplied reliability estimates reduces performance compared with a plain prompt. These findings demonstrate that agents can use externally calibrated source reliability even when they cannot reliably infer it through zero-shot deliberation. This gap between using and inferring reliability motivates the incorporation of empirical estimates into future scientific agent infrastructure. Moreover, our results underscore the need to evaluate evidence selection and combination across sources as distinct capabilities.