ShuffleMTM: Cross-Channel Masked Pre-training without Breaking Channel Independence
Abstract
Real-world multivariate time series (MTS) often exhibit coupled dynamics across channels, while reliable annotations remain costly and limited. Masked time series modeling (MTM) provides a self-supervised paradigm for learning from unlabeled data, and MTMs and time series foundation models (TSFMs) widely adopt channel-independent (CI) encoding for its simplicity and scalability. However, independently encoding each channel prevents the pre-training process from explicitly learning cross-channel dependencies. We present ShuffleMTM, a cross-channel masked pre-training framework without breaking channel independence. ShuffleMTM constructs shuffled univariate series by dynamically rearranging patches across channels, and processes the original and shuffled views through dual-branch Transformer encoders connected by lightweight dependency bridges. A dual-branch masked reconstruction objective jointly learns cross-time and cross-channel information useful for reconstructing each series, while retaining univariate encoding. Across forecasting and classification benchmarks, ShuffleMTM achieves competitive or superior performance against CI and cross-channel modeling approaches, and controlled analyses confirm its ability to capture cross-channel interactions. Furthermore, integrating ShuffleMTM into MOMENT, a CI TSFM, improves forecasting performance across most evaluated benchmarks, demonstrating its potential as a scalable cross-channel-aware pre-training strategy for TSFMs.