Depth-Adjustable Time-Series Foundation Models
Levente Zólyomi ⋅ Martin Dallinger ⋅ Sebastian Böck ⋅ Sepp Hochreiter
Abstract
Time-series foundation models (TSFMs) generalize well in zero-shot settings, but their stored size and compute per forecast are fixed at training time. A single checkpoint therefore cannot be matched to the varying memory and latency budgets of edge deployments. We study weight-tied depth recurrence as a remedy: in a TSFM backbone a core of blocks is applied $T$ times, so the effective depth of the network grows with $T$ while stored parameters remain fixed. On an xLSTM and a Transformer backbone, evaluated zero-shot on GIFT-Eval, the looped model beats the plain network with the same parameters at every width and closes $\sim$60% of the gap to the plain network with the same compute, at a fraction of its stored size. Training with an adaptive loop count makes one checkpoint competent at several depth settings, turning $T$ into an inference-time knob that dials compute against accuracy. Because every intermediate forecast is usable, a simple per-window early-exit rule on the forecast's movement saves a further $\sim$30\% of block applications on the xLSTM within a 1\% relative CRPS tolerance. Our results show that looped TSFMs provide competitive performance while being more adaptable to different levels of resource constraints.
Chat is not available.
Successful Page Load