Independent Verification Paths Are Not Independent: Common-Mode Failure in Scientific Data Pipelines
Abstract
Automated science pipelines increasingly police themselves with redundant computation: derive every headline number by two independent routes and halt when they disagree. We report a live failure of that gate. In a published cross-catalogue integrity study of two open registers of Earth-orbiting objects, a gate comparing a set-based Python path against a SPARQL path over the emitted RDF graph printed ALL CROSS-CHECKS AGREE and exited zero on three numbers that were wrong. One was wrong by a factor of 3.6. The two paths agreed because both imported the same constants encoding the same misreading of the source's controlled vocabulary, so the error was common-mode and the gate was silent by construction. We give the mechanism, implement three alternative gates and measure them against the defective and the corrected code, and test whether the failure generalises to automated authors. Across 30 trials, asking a code-generating model for an independent verification path produced 30 dependent ones; forbidding code sharing changed how the second path acquired its vocabulary without changing whether that vocabulary was wrong. Redundancy verifies implementation. Semantic correctness needs verifiers that read the source's own account of itself.