Approval Is Not Discovery: When Can a Model of the Scientist Steer a Discovery Agent?

Discovery agents can be guided by models that predict which experiments a scientist will approve, before any assay results exist. A natural rule trusts such a model once it agrees with the scientist as often as the scientist agrees with themselves. We show that this rule can certify the wrong model, and we give an audit that does not. We study an idealized reviewer who judges a proposal correctly when reviewing carefully and by plausibility alone otherwise. First, when careful review happens less than half the time, the best predictor of the scientist's approvals passes the agreement rule despite having lower balanced accuracy than a single review, while a perfectly accurate validator fails it. Second, showing each proposal to the reviewer several times can measure validity without an assay. If reviews are independent and careful review never errs and happens at a fixed rate known to be below one half, four reviews per proposal recover the true hit rate from approval counts alone, and three do not. Recording the reviewer's first impression of plausibility lowers this to two reviews and drops the bound. Third, we test these predictions in retrospective simulations on four genome-wide CRISPR screens, with plausibility set by a fixed gene annotation rule. Training on approvals raises the hit rate but cuts the share of proposals that are hits outside the plausible set by 79%-83%. With a real LLM as the reviewer, the agreement rule again rejects a perfectly accurate validator, though the LLM's errors break the audit assumptions. At an assumed ten reviews per assay, direct assays estimate the hit rate more precisely than the count-only audit in every tested setting, so the audits matter most when assays are unavailable or slow.