Forecasting Skill Is Not Interventional Validity: Evaluating Covariate-Aware Time-Series Foundation Models as Zero-Shot Counterfactual Predictors

Covariate-native time-series foundation models (TSFMs) accept known future covariates, and vendors advertise this for planning, an interventional use. We evaluate Chronos-2, TimesFM-3, TiRex-2 and TabPFN-TS-3 zero-shot on three simulators with ground-truth interventional effects (PK-PD tumour growth, 2R2C building thermal, promotion-demand) in observed-confounder, off-support and hidden-confounder regimes. The TSFMs' excess effect error over an in-context regression reference grows with the confounding knob. Factual skill is an unstable guide to interventional validity: on a common method set its rank correlation with effect error is +0.80 in tumour growth and -0.80 in building thermal, while TabPFN-TS-3 has the TSFMs' lowest effect error everywhere. Under a hidden confounder every observational method, the regression reference and domain-trained estimators included, shows a systematic sign inversion (sign accuracy <= 0.11). A randomised-action prefix removes a similar absolute amount from every model's gap: most of TabPFN-TS-3's, under a third of the others'.