HALO: Heterogeneous-Aware LoRA Optimization via Rank Allocation and Client-Aware Projection

Federated Learning (FL) with Low-Rank Adaptation (LoRA) enables privacy-preserving collaborative fine-tuning of foundation models. Existing federated LoRA studies often assume homogeneous ranks and lack principled rank allocation, leading to underfitting or overfitting under data and resource heterogeneity. Although later heterogeneous-rank federated LoRA methods relax the homogeneous-rank assumption, they still ignore resource-adaptive rank allocation across clients, while their aggregation and distribution strategies suffer from structural limitations, including rank collapse in naive truncation and the loss of locally relevant update signals caused by objective mismatch in SVD truncation. To address these issues, we propose HALO, a heterogeneous federated LoRA framework that integrates adaptive rank allocation with client-aware aggregation and distribution. Under the neural tangent kernel regime, we derive a sufficient rank condition to guide client-specific rank allocation from local data geometry and resource constraints. We further propose Client-Aware Projection (CAP), which builds a product-summed reference and redistributes rank-compatible balanced LoRA factors through projections onto client-induced subspaces, improving generalization via cross-client information while preserving locally relevant update signals. We show that CAP controls the client-loss-relevant truncation residual under the NTK approximation without additional communication overhead, which stabilizes local training and improves performance. Extensive experiments demonstrate the effectiveness of HALO. Our code is available at https://anonymous.4open.science/r/artifact2026-4174.