AMARIS: Merging Generalist and Specialist LLMs via Adaptive Subspace Inheritance

Domain specialization of large language models (LLMs) often improves target-domain performance but degrades broad capabilities. Model merging is an attractive training-free alternative, yet common parameter-space strategies (for example, interpolation and sparsified averaging) provide limited control over task-specific interference. We introduce Adaptive Model Alignment via Ranked Inherited Subspaces (AMARIS), a covariance-driven framework that transfers specialist updates only along selected activation directions. AMARIS formulates merging as budgeted subspace inheritance: it maximizes specialist activation capture while constraining base-activation deviation, then builds a low-rank projection gate from ranked generalized eigendirections under an explicit preservation budget. This yields a single static merged checkpoint that concentrates transfer in high-utility, low-interference directions. We evaluate AMARIS on Qwen3-based merges (8B and 32B) across finance, instruction-following, and multilingual SEA specialization. In our evaluated settings, AMARIS reaches Pareto operating points that are competitive with strong training-free baselines, including 98.37\% specialist retention with 95.76\% core retention (finance), 103.49\% core retention with 90.28\% specialist retention (instruction following), and 93.69\% specialist retention with 99.77\% core retention (SEA). These results support subspace-constrained merging as a practical route to compositional capability integration without additional training.