Sparse Parameter Adaptation for Fair Model Transfer Across Domains

Machine learning models trained in one domain often face significant challenges when deployed in a different domain due to distribution shifts, which can degrade both predictive performance and fairness. This paper studies the problem of transferring fair models from a source domain to a target domain where labeled data are scarce or unavailable, and only limited unlabeled data are accessible. We focus on scenarios where the original training data are inaccessible due to privacy or regulatory constraints, and fairness requirements must still be maintained in the target domain. To address these challenges, we propose a framework that regularizes model updates with sparsity-promoting penalties to adapt only a subset of parameters, enabling interpretable and reliable transfer. For linear models, we use an $\ell_1$-norm proximity term coupled with covariance-based fairness constraints, while for deep neural networks, we extend this idea via group sparse regularization. Additionally, we explore nonlinear fairness notions by incorporating $\chi^2$-divergence-based measures inspired by the FERMI~\citep{lowy2022stochastic} framework. Empirical evaluations on the New Adult dataset demonstrate the effectiveness of our approach in transferring fair models from the source to target domain (different states) under limited target supervision. Our method achieves improved fairness-accuracy trade-offs while preserving interpretability, making it suitable for practical deployment in sensitive decision-making contexts such as credit eligibility across jurisdictions.