Can We Trust Gating Weights? Auditing the Validity of Explanations in Latent-Dynamics Mixtures

Evaluation of interpretability often collapses a chain of claims into a single proxy: a readable weight vector is reported as an explanation. We audit this practice in a four-expert latent-dynamics mixture for long-horizon traffic forecasting. The predictor improves normalized MSE over validation-selected single experts at three of four horizons, yet the protocol used to interpret its gate weights fails several validity checks. Source inspection finds independently trained expert pipelines rather than a shared encoder and latent state, confounding dynamics-family attribution. A separate gate is trained for every forecast horizon, so the purported horizon-conditioning feature is constant within each run. The supplied gate-weight visualization sums seed-level weights and clips the axis rather than displaying their mean; corrected means are SINDy-dominant at every horizon. At horizon 96, seed-level mean routes differ substantially (mean pairwise total variation 0.511), and the evaluation uses only 35, 17, 10, and 4 test origins across the four horizons. These findings do not establish that gating weights are unfaithful. They establish that the current evaluation does not identify the claimed construct. We introduce a claim-evidence ladder and a three-stage audit of structural validity, statistical validity, and causal validity, then specify the controls and interventions needed before routing weights can support claims about discovered dynamical regimes. The case study illustrates how an evaluation can be numerically precise while remaining inferentially invalid.