Learning to Forget using Diffusion Hypernetworks

Making Machine Learning models forget specific data points is gaining increasing attention to remove adversarial data poisoning attacks from already trained models and to comply with privacy and AI regulations. The objective is to unlearn the effect of undesired data from a trained model while maintaining performance on the remaining data. This paper introduces HyperForget, a novel Machine Unlearning framework that leverages hypernetworks-neural networks that generate parameters for other networks - to dynamically sample models that forget targeted data while preserving essential capabilities. Leveraging diffusion models, we implement two Diffusion HyperForget Networks and used them to sample unlearned models in Proof-of-Concept experiments. The unlearned models obtained zero accuracy on the forget set, while preserving good accuracy on the retain sets, highlight the potential of HyperForget for dynamic targeted data removal and a promising direction for developing adaptive Machine Unlearning algorithms.