Reproducible Science: Machine-Checkable Claims from Preregistration to Publication

Every claim in a paper comes from somewhere. AI tooling allows us to quickly generate hypotheses, run analyses, and draft manuscripts, but we identify verification and provenance as open issues: (1) verifying that the finished paper agrees with the artifacts behind it typically requires manual or agentic verification, which is expensive and difficult to audit; (2) verification usually does not include a denominator for the number of sources or artifacts checked; (3) verification snapshots quickly become stale, and are not guaranteed to find every issue or reproduce the same issues on subsequent runs; (4) more experiments means more researcher degrees of freedom and post-hoc analysis, even if unintentional, or done by an agent; and (5) verification is typically done post-hoc, and does not cover all steps of the research lifecycle. We present Reproducible Science, a declarative, continuous, machine-checkable provenance layer for research claims, throughout the whole research lifecycle. prereg freezes a plan before the run, results seals its inputs, records its outputs and binds numbers in the manuscript to the run that produced them, citations pins exact quotations inside cited sources, and repro checks every declaration against the artifact it names. Our Claude Code plugin records provenance at the time of experimentation and reports drift on every edit through hooks. Applying this contract to this paper surfaced two errors, one of which had already gone stale twice. As agents take over more of the research lifecycle, the artifacts they leave behind are worth only as much as we can verify the claims inside them.