PatchTrace Receipts: Auditable Delta Records for Evaluator Maintenance
Abstract
Evaluator software changes after benchmark results have been produced, yet a typical evaluation record describes only one run. We introduce PatchTrace, a machine-readable patch-delta receipt for the boundary between an evaluator's old and fixed revisions. A receipt binds the two commits, the changed official source or configuration, preserved evaluation objects, immutable ground truth, execution provenance, paired outcomes, explicit assertions, and deterministic repeats. We instantiate the receipt on three analysis-frozen public maintenance incidents across two evaluator ecosystems, covering population, scoring, and parser changes. Revision-local replay exposes 5,700 versus 5,330 MMLU-Redux rows (370 removals), a one-item Natural Questions exact-match change from 0.0 to 1.0, and a one-instance SWE-bench change from 0 to 15 parsed tests and from RESOLVEDNO to RESOLVEDFULL. All primary/repeat pairs are byte-identical, and the validator rejects 9/9 targeted attacks on semantic bindings. These cases demonstrate compact serialized receipt metadata; they do not establish downstream impact, bug prevalence, or general semantic equivalence.