Contaminated Ancestors: Three Confident Verifiers That Certify Everything Except the Property in Question
Christopher Y Huang ⋅ Ryan J Ahn ⋅ Wenhao Lu
Abstract
Ancestral sequence reconstruction (ASR) is a mature, widely trusted scientific inference pipeline that emits confidence numbers by default, and it is a clean instance of an output that looks verified without being so. When a protein family's history contains recombination, ASR's tree assumption is violated, and the pipeline does not fail loudly: it returns a chimeric ancestor that never existed, with a mean maximum per-site posterior of $0.93$ at $69\%$ site accuracy against a known true root. The posterior is not lying; it is answering a different question. The pathology is joint and every standard diagnostic is marginal. We take this as a case study in verifier design under imperfect verification, and report what happened when we built a verifier for it and then verified the verifier. Three results. First, a second confident certificate, AlphaFold2/ColabFold pLDDT, also fails to detect the property: across four real families our chimeric reconstructions fold at pLDDT $86.6$ to $97.0$, all confident, with detection uniformly absent, so fold confidence certifies foldability and is silent about historicity. Second, a pre-registered ablation shows the cheap verifier dominates: a two-term string-equality statistic using no structure and no network fires on $36/40$ contaminated families (CI $[77\%, 96\%]$) against an inverse-folding model's $11/40$ ($[16\%, 43\%]$), with direct consequences for verification-budget allocation. Third, the standard sequence-based competitor refuses to run on our real family: its own parameter-count requirement demands $235$ alignment sites where $58$ exist, which is verifier scarcity by inapplicability rather than by cost. We also report two verification-integrity incidents from our own repository: an accelerator backend that returned corrupted tensors without raising, converting $p = 0.030$ into $p = 1.000$; and an internal results artifact reporting a run whose data was never committed and which could not be regenerated because the aggregation code had silently drifted from the result schema. We argue for verifiers engineered to fail visibly, and give the design pattern we used.
Chat is not available.
Successful Page Load