RTM: Single-Pass Unsupervised Adaptation for Multivariate Time Series Anomaly Detection

Multivariate time-series anomaly detectors deployed on live streams must operate causally and adapt as the underlying data distribution evolves, often without anomaly labels or repeated offline retraining. We introduce Residual Temporal Memory (RTM), a hierarchical extension of Hierarchical Temporal Memory (HTM) for single-pass streaming anomaly detection. RTM partitions high-dimensional inputs across small HTM modules arranged in a pyramid and propagates only prediction residuals activity that lower layers fail to explain, so that higher layers focus on unresolved temporal structure. A sparsity-preserving SoftUnion operator enables fixed-dimensional composition of intermediate sparse representations. We evaluate RTM on SWaT under a causal prequential protocol, considering both standard point-wise anomaly detection and four persistent distribution shifts. Standard RTM achieves the strongest overall post-shift robustness among the evaluated methods, obtaining the best post-drift F1 on rotation and sudden shifts, tying the best covariance result, and remaining within 0.01 of the best sign-flip result. A feature-resolution variant improves point-wise F1 to $0.63$, revealing a trade-off between detection accuracy and continual adaptation. Ablations further show that RTM's gains arise from residual propagation rather than hierarchy alone, while SoftUnion better preserves anomaly-relevant information under temporal compression. Together, these results suggest residual hierarchical temporal learning as a promising approach for adaptive, single-pass anomaly detection without offline retraining.