A Large Forecast Oracle Gap Does Not Imply a Large Routing Opportunity
Abstract
Time-series foundation models (TSFMs) raise the question of which pretrained forecaster to use for a given series or forecast context. A common first diagnostic is a hindsight oracle: after the future is observed, choose the candidate with the smallest loss. Over eight forecasters this oracle improves on the best fixed model by 23.5%, apparently suggesting substantial routing potential. We ask what such an oracle gap says about model choice before the future is known. In controlled experiments with deterministic forecasters, including a frozen TSFM, a negative control has a positive oracle gap but exactly zero forecast-time selection value. More strongly, when the candidate predictions and the marginal future distribution are held fixed, the oracle gap stays at about 0.3128 MASE while its selectable share moves from 11% to 47% as an informative signal is revealed. On real data, the evaluated forecast-time selection rules gain at most 5.8%, well below the 23.5% oracle gain. Oracle-gap magnitude alone therefore does not identify routing opportunity.