GUARD: Scalable Gradient-based Unlearning with Adversarial Robustness Defense

Machine Unlearning (MU) is an emerging task to remove specified training data from a trained model while preserving its utility, with Gradient-based Unlearning (GU) standing out as a prevalent category due to high scalability. However, standard MU which solely optimizes for clean accuracy, inadvertently damages adversarial robustness on the retain data. Although a few studies extend MU to adversarially trained models, they rely on strong assumptions (e.g., well-conditioned Hessians, smoothness) and costly Hessian-based updates, limiting scalability and efficiency. In this paper, we propose **GUARD**, a scalable and efficient framework that achieves unlearning while preserving retain-data robustness. Having identified the two causes of robustness degradation, we accordingly design *Robustness-Aware Gradient Projection* and *Importance-Guided Parameter Selection*. Both modules rely on a compact adversarially vulnerable dataset that we construct with or without retain data, to enable scalability under varying data constraints. Validated by theoretical analysis and experiments across 6 settings, GUARD outperforms 6 baselines, boosting robust accuracy by up to 6.3$\times$, closely matching the Retrain oracle in both unlearning and robustness.