The Silent Mutation of Evidence: Tracing Citation Provenance in Agentic Biomedical Hypothesis Generation
Abstract
Agentic systems for scientific hypothesis generation are evaluated on the hypotheses they produce, which cannot show where each hypothesis's evidence came from. We instrument a deployed biomedical pipeline operator by operator, logging every model call and citation without modifying production code. In 17 of 18 research questions the mutation step is shown the first 25 papers in retrieval order, disjoint from the 60 relevance-ranked papers the generator read, so nothing a mutated hypothesis cites of its own accord was available to the step that wrote its parent. A word-overlap heuristic then fills most of the bibliography from the wider corpus and masks the gap. In a pre-registered paired test that changes only that list, relevance-ranked candidates raise the similarity of chosen evidence in every affected question (0.721 to 0.793, difference 0.073 [0.051, 0.102]), with no effect in a negative control. The pipeline's own evaluator, the same model that wrote the hypotheses, rates the better-grounded arm higher (0.453 to 0.690) from four paper titles, a third of them supplied by the linker. Nothing errors.