Gram-Calibrated Anchoring for Class-Incremental Learning

We introduce Gram-Calibrated Anchoring (GCA) for class-incremental learning (CIL) with pre-trained Vision Transformers. GCA rests on the observation that, under continual low-rank adaptation, anchor prototypes---class means extracted once from the frozen pre-trained backbone---are effective substitutes for prototypes recomputed from the adapted model; a formal stability analysis bounds decision-boundary shifts in terms of feature-level prototype drift, while empirical LoRA perturbation measurements support the small-drift regime. This motivates fixing the classification head to anchor prototypes computed upon each task's arrival rather than recalibrating prototypes after training. The fixed anchor geometry then enables Gram-Compensated Inference (GCI), which applies the regularized inverse of the anchor Gram matrix to deconvolve inter-class correlation that standard cosine classification ignores, and normalizes each coordinate by its estimation standard deviation for fair cross-class comparison. The resulting method trains a single continually evolved LoRA module with no exemplar storage or expanding adapter pools. Experiments on ImageNet-R, ImageNet-A, CIFAR-100, and CUB-200 with multiple backbones show that GCA achieves competitive accuracy across four CIL benchmarks while significantly reducing training cost over recent methods.