Robust and Efficient Continual Model Merging via Global Singular Subspace Separation and Restoration

Continual model merging addresses a more realistic setting in which task-specific models arrive sequentially. Naive extension of standard model merging approaches to the continual setting leads to task balance collapse and substantial performance degradation. To address this issue, we propose Global Singular Subspace Separation and Restoration (GS3R), a data-free and optimization-compatible framework for continual model merging. GS3R preserves task balance by globally separating singular components and introducing vector restoration matrices for perturbation-free recovery. To further enhance efficiency, we introduce Pre-merge Restoration and Flushing (PRF), which guarantees peak memory usage comparable to that of other methods. Experiments on vision and language tasks demonstrate that GS3R consistently outperforms prior continual model merging methods.