Timezone: »

Accelerating Certified Robustness Training via Knowledge Transfer
Pratik Vaishnavi · Kevin Eykholt · Amir Rahmati

Thu Dec 08 09:00 AM -- 11:00 AM (PST) @

Training deep neural network classifiers that are certifiably robust against adversarial attacks is critical to ensuring the security and reliability of AI-controlled systems. Although numerous state-of-the-art certified training methods have been developed, they are computationally expensive and scale poorly with respect to both dataset and network complexity. Widespread usage of certified training is further hindered by the fact that periodic retraining is necessary to incorporate new data and network improvements. In this paper, we propose Certified Robustness Transfer (CRT), a general-purpose framework for reducing the computational overhead of any certifiably robust training method through knowledge transfer. Given a robust teacher, our framework uses a novel training loss to transfer the teacher’s robustness to the student. We provide theoretical and empirical validation of CRT. Our experiments on CIFAR-10 show that CRT speeds up certified robustness training by 8× on average across three different architecture generations while achieving comparable robustness to state-of-the-art methods. We also show that CRT can scale to large-scale datasets like ImageNet.

Author Information

Pratik Vaishnavi (Stony Brook University)
Kevin Eykholt (International Business Machines)
Amir Rahmati (Stony Brook University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors

  • 2022 : Benchmarking the Effect of Poisoning Defenses on the Security and Bias of the Final Model »
    Nathalie Baracaldo · Kevin Eykholt · Farhan Ahmed · Yi Zhou · Shriti Priya · Taesung Lee · Swanand Kadhe · Yusong Tan · Sridevi Polavaram · Sterling Suggs
  • 2022 : On the Feasibility of Compressing Certifiably Robust Neural Networks »
    Pratik Vaishnavi · Veena Krish · Farhan Ahmed · Kevin Eykholt · Amir Rahmati
  • 2022 Spotlight: Lightning Talks 5B-2 »
    Conglong Li · Mohammad Azizmalayeri · Mojan Javaheripi · Pratik Vaishnavi · Jon Hasselgren · Hao Lu · Kevin Eykholt · Arshia Soltani Moakhar · Wenze Liu · Gustavo de Rosa · Nikolai Hofmann · Minjia Zhang · Zixuan Ye · Jacob Munkberg · Amir Rahmati · Arman Zarei · Subhabrata Mukherjee · Yuxiong He · Shital Shah · Reihaneh Zohrabi · Hongtao Fu · Tomasz Religa · Yuliang Liu · Mohammad Manzuri · Mohammad Hossein Rohban · Zhiguo Cao · Caio Cesar Teodoro Mendes · Sebastien Bubeck · Farinaz Koushanfar · Debadeepta Dey