The Block Catastrophe of Marginal Calibration in Certified Requirements Traceability
Jikun Wu ⋅ Dongxin Guo ⋅ Siu Ming Yiu
Abstract
Certifying authorities for safety-critical software audit recovered trace matrices module by module, not on average. Every existing distribution-free calibration delivers a marginal certificate, and we show the gap to the auditor's quantity is structural and unbounded. Under within-block exchangeability with cross-block heterogeneity in the conditional null score distribution, every procedure controlling marginal $\mathrm{FDR}$ at level $\alpha$ on a bipartite trace matrix with $K$ source modules admits worst-module false discovery proportion at least $1{-}\alpha{-}O(1/K)$ with probability at least $1{-}2/K$. We name this the block catastrophe and close it with $\mathrm{CERTRA}$, which wraps any black-box scorer, computes a within-block rank-indicator conformal $e$-value, and applies $e$-$\mathrm{BH}$ separately within each source module. The construction yields finite-sample per-module $\mathrm{FDR}$ control under arbitrary intra-module dependence, with matching power up to a rank-quantization factor. We release IndTrace-2026, a 12{,}406-link composite across aerospace, automotive, and medical-device specifications with professional ASIL/DAL/SIL annotation (Cohen's $\kappa{=}0.86$). $\mathrm{CERTRA}$ holds worst-block $\mathrm{FDP}$ at 0.118 where split-conformal drifts to 0.413, retains 0.741 of true links versus 0.572 for the strongest fair-operating-point baseline, and reduces audit deficiency 3.3-fold. Per-module distribution-free calibration is what an industrial audit can sign.
Chat is not available.
Successful Page Load