Phantom Transfer: Data Poisoning can Survive Data-Level Defences

We present a data poisoning attack---Phantom Transfer---with the property that, even if you know precisely how the poison was placed into an otherwise benign dataset, you cannot filter it out. We achieve this by modifying subliminal learning to work in real-world contexts and demonstrate that the attack works regardless of which model produces the data, which model is trained on the data or what the attack target is. The attack survives 11 tested data-level defences, including one where every sample is paraphrased by a different model. We characterise when this attack works best and show that it can be used to plant password-triggered behaviours into models while still beating defences. In short, we provide an existence proof that maximum-affordance defences can fail to stop sophisticated data poisoning attacks. We suggest that future defences should supplement by include white-box methods and post-training model audits.