Verify, Rebind, Repair: Enforcing Evidence Consistency in Small Biomedical Agents
Abstract
Small biomedical language models can reconstruct relations present in supplied evidence while omitting citations, proposing unsupported edges, or failing to abstain when no supported answer exists. We study which failures can be corrected without additional model inference using an evidence contract: explicit conditions for consistency between an answer and its supplied evidence. On 1,280 graph tasks derived from 256 held-out drugs, deterministic citation rebinding and abstention filtering improve success by 26.8–86.6 percentage points across three small open models, without changing their proposed relations. In the prespecified repair comparison, exact violation feedback outperforms self-reflection for two models, primarily by improving abstention. A post-hoc replay of verification-based escalation reaches 98.9–99.5% success while invoking a 27B fallback on 23–41% of tasks, compared with 97.2% for that fallback with the same tools. This non-degradation property follows from sound acceptance under the graph endpoint, not from biological validation. On PubMed abstracts, a classifier label paired with a content-blind source prefix achieves 92.2% annotation agreement and literal validity, exposing the endpoint’s inability to measure quotation support. A prespecified specialist-feedback comparison is inconclusive. These results distinguish what deterministic evidence checks can certify from the semantic support they leave untested.