Timezone: »
The success of Deep Learning and its potential use in many safety-critical applications has motivated research on formal verification of Neural Network (NN) models. Despite the reputation of learned NN models to behave as black boxes and the theoretical hardness of proving their properties, researchers have been successful in verifying some classes of models by exploiting their piecewise linear structure and taking insights from formal methods such as Satisifiability Modulo Theory. These methods are however still far from scaling to realistic neural networks. To facilitate progress on this crucial area, we make two key contributions. First, we present a unified framework that encompasses previous methods. This analysis results in the identification of new methods that combine the strengths of multiple existing approaches, accomplishing a speedup of two orders of magnitude compared to the previous state of the art. Second, we propose a new data set of benchmarks which includes a collection of previously released testcases. We use the benchmark to provide the first experimental comparison of existing algorithms and identify the factors impacting the hardness of verification problems.
Author Information
Rudy Bunel (Oxford University)
Ilker Turkaslan (University of Oxford)
Philip Torr (University of Oxford)
Pushmeet Kohli (DeepMind)
Pawan K Mudigonda (University of Oxford)
More from the Same Authors
-
2021 : Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge »
Jiyang Qi · Yan Gao · Yao Hu · Xinggang Wang · Xiaoyu Liu · Xiang Bai · Serge Belongie · Alan Yuille · Philip Torr · Song Bai -
2021 : Faking Interpolation Until You Make It »
Alasdair Paren · Rudra Poudel · Pawan K Mudigonda -
2021 : Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs »
Dan Rosenbaum · Marta Garnelo · Michal Zielinski · Charles Beattie · Ellen Clancy · Andrea Huber · Pushmeet Kohli · Andrew Senior · John Jumper · Carl Doersch · S. M. Ali Eslami · Olaf Ronneberger · Jonas Adler -
2021 : Are Vision Transformers Always More Robust Than Convolutional Neural Networks? »
Francesco Pinto · Philip Torr · Puneet Dokania -
2021 : Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks »
Francesco Pinto · Harry Yang · Ser Nam Lim · Philip Torr · Puneet Dokania -
2023 Poster: ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentation »
Shuyang Sun · Weijun Wang · Andrew Howard · Qihang Yu · Philip Torr · Liang-Chieh Chen -
2023 Poster: Language Model Tokenizers Introduce Unfairness Between Languages »
Aleksandar Petrov · Emanuele La Malfa · Philip Torr · Adel Bibi -
2023 Poster: Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union »
Zifu Wang · Maxim Berman · Amal Rannen-Triki · Philip Torr · Devis Tuia · Tinne Tuytelaars · Luc V Gool · Jiaqian Yu · Matthew Blaschko -
2023 Poster: Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models »
Shuo Chen · Jindong Gu · Zhen Han · Yunpu Ma · Philip Torr · Volker Tresp -
2022 : Panel »
Jeevana Priya Inala · Pushmeet Kohli · Ann Kennedy · Sriram Rajamani · Yisong Yue -
2022 Poster: Using Mixup as a Regularizer Can Surprisingly Improve Accuracy & Out-of-Distribution Robustness »
Francesco Pinto · Harry Yang · Ser Nam Lim · Philip Torr · Puneet Dokania -
2022 Poster: Structure-Preserving 3D Garment Modeling with Neural Sewing Machines »
Xipeng Chen · Guangrun Wang · Dizhong Zhu · Xiaodan Liang · Philip Torr · Liang Lin -
2022 Poster: Learn what matters: cross-domain imitation learning with task-relevant embeddings »
Tim Franzmeyer · Philip Torr · João Henriques -
2022 Poster: In Defense of the Unitary Scalarization for Deep Multi-Task Learning »
Vitaly Kurin · Alessandro De Palma · Ilya Kostrikov · Shimon Whiteson · Pawan K Mudigonda -
2022 Poster: Make Some Noise: Reliable and Efficient Single-Step Adversarial Training »
Pau de Jorge Aranda · Adel Bibi · Riccardo Volpi · Amartya Sanyal · Philip Torr · Gregory Rogez · Puneet Dokania -
2022 Poster: FedSR: A Simple and Effective Domain Generalization Method for Federated Learning »
A. Tuan Nguyen · Philip Torr · Ser Nam Lim -
2021 : Shape-Tailored Deep Neural Networks With PDEs »
Naeemullah Khan · Angira Sharma · Philip Torr · Ganesh Sundaramoorthi -
2021 : Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs »
Dan Rosenbaum · Marta Garnelo · Michal Zielinski · Charles Beattie · Ellen Clancy · Andrea Huber · Pushmeet Kohli · Andrew Senior · John Jumper · Carl Doersch · S. M. Ali Eslami · Olaf Ronneberger · Jonas Adler -
2021 : Live Panel »
Max Welling · Bharath Ramsundar · Irina Rish · Karianne J Bergen · Pushmeet Kohli -
2021 Poster: You Never Cluster Alone »
Yuming Shen · Ziyi Shen · Menghan Wang · Jie Qin · Philip Torr · Ling Shao -
2021 Poster: Looking Beyond Single Images for Contrastive Semantic Segmentation Learning »
FEIHU ZHANG · Philip Torr · Rene Ranftl · Stephan Richter -
2021 Poster: FACMAC: Factored Multi-Agent Centralised Policy Gradients »
Bei Peng · Tabish Rashid · Christian Schroeder de Witt · Pierre-Alexandre Kamienny · Philip Torr · Wendelin Boehmer · Shimon Whiteson -
2021 Poster: Do Different Tracking Tasks Require Different Appearance Models? »
Zhongdao Wang · Hengshuang Zhao · Ya-Li Li · Shengjin Wang · Philip Torr · Luca Bertinetto -
2021 Poster: A Continuous Mapping For Augmentation Design »
Keyu Tian · Chen Lin · Ser Nam Lim · Wanli Ouyang · Puneet Dokania · Philip Torr -
2021 Poster: Overcoming the Convex Barrier for Simplex Inputs »
Harkirat Singh Behl · M. Pawan Kumar · Philip Torr · Krishnamurthy Dvijotham -
2020 Poster: STEER : Simple Temporal Regularization For Neural ODE »
Arnab Ghosh · Harkirat Singh Behl · Emilien Dupont · Philip Torr · Vinay Namboodiri -
2020 Poster: Calibrating Deep Neural Networks using Focal Loss »
Jishnu Mukhoti · Viveka Kulharia · Amartya Sanyal · Stuart Golodetz · Philip Torr · Puneet Dokania -
2020 Poster: Lightweight Generative Adversarial Networks for Text-Guided Image Manipulation »
Bowen Li · Xiaojuan Qi · Philip Torr · Thomas Lukasiewicz -
2020 Poster: Hybrid Models for Learning to Branch »
Prateek Gupta · Maxime Gasse · Elias Khalil · Pawan K Mudigonda · Andrea Lodi · Yoshua Bengio -
2020 Poster: Continual Learning in Low-rank Orthogonal Subspaces »
Arslan Chaudhry · Naeemullah Khan · Puneet Dokania · Philip Torr -
2020 Poster: Training Generative Adversarial Networks by Solving Ordinary Differential Equations »
Chongli Qin · Yan Wu · Jost Tobias Springenberg · Andy Brock · Jeff Donahue · Timothy Lillicrap · Pushmeet Kohli -
2020 Poster: Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming »
Sumanth Dathathri · Krishnamurthy Dvijotham · Alexey Kurakin · Aditi Raghunathan · Jonathan Uesato · Rudy Bunel · Shreya Shankar · Jacob Steinhardt · Ian Goodfellow · Percy Liang · Pushmeet Kohli -
2020 Spotlight: Training Generative Adversarial Networks by Solving Ordinary Differential Equations »
Chongli Qin · Yan Wu · Jost Tobias Springenberg · Andy Brock · Jeff Donahue · Timothy Lillicrap · Pushmeet Kohli -
2020 Poster: The Autoencoding Variational Autoencoder »
Taylan Cemgil · Sumedh Ghaisas · Krishnamurthy Dvijotham · Sven Gowal · Pushmeet Kohli -
2020 Spotlight: The Autoencoding Variational Autoencoder »
Taylan Cemgil · Sumedh Ghaisas · Krishnamurthy Dvijotham · Sven Gowal · Pushmeet Kohli -
2019 : Coffee + Posters »
Changhao Chen · Nils Gählert · Edouard Leurent · Johannes Lehner · Apratim Bhattacharyya · Harkirat Singh Behl · Teck Yian Lim · Shiho Kim · Jelena Novosel · Błażej Osiński · Arindam Das · Ruobing Shen · Jeffrey Hawke · Joachim Sicking · Babak Shahian Jahromi · Theja Tulabandhula · Claudio Michaelis · Evgenia Rusak · WENHANG BAO · Hazem Rashed · JP Chen · Amin Ansari · Jaekwang Cha · Mohamed Zahran · Daniele Reda · Jinhyuk Kim · Kim Dohyun · Ho Suk · Junekyo Jhung · Alexander Kister · Matthias Fahrland · Adam Jakubowski · Piotr Miłoś · Jean Mercat · Bruno Arsenali · Silviu Homoceanu · Xiao-Yang Liu · Philip Torr · Ahmad El Sallab · Ibrahim Sobh · Anurag Arnab · Krzysztof Galias -
2019 Poster: Learning Transferable Graph Exploration »
Hanjun Dai · Yujia Li · Chenglong Wang · Rishabh Singh · Po-Sen Huang · Pushmeet Kohli -
2019 Poster: Are Labels Required for Improving Adversarial Robustness? »
Jean-Baptiste Alayrac · Jonathan Uesato · Po-Sen Huang · Alhussein Fawzi · Robert Stanforth · Pushmeet Kohli -
2019 Poster: Adversarial Robustness through Local Linearization »
Chongli Qin · James Martens · Sven Gowal · Dilip Krishnan · Krishnamurthy Dvijotham · Alhussein Fawzi · Soham De · Robert Stanforth · Pushmeet Kohli -
2019 Poster: Multi-Agent Common Knowledge Reinforcement Learning »
Christian Schroeder de Witt · Jakob Foerster · Gregory Farquhar · Philip Torr · Wendelin Boehmer · Shimon Whiteson -
2019 Poster: Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model »
Atilim Gunes Baydin · Lei Shao · Wahid Bhimji · Lukas Heinrich · Saeid Naderiparizi · Andreas Munk · Jialin Liu · Bradley Gram-Hansen · Gilles Louppe · Lawrence Meadows · Philip Torr · Victor Lee · Kyle Cranmer · Mr. Prabhat · Frank Wood -
2019 Poster: Controllable Text-to-Image Generation »
Bowen Li · Xiaojuan Qi · Thomas Lukasiewicz · Philip Torr -
2018 Poster: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Spotlight: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2017 Poster: Neural Program Meta-Induction »
Jacob Devlin · Rudy Bunel · Rishabh Singh · Matthew Hausknecht · Pushmeet Kohli -
2017 Poster: Learning Disentangled Representations with Semi-Supervised Deep Generative Models »
Siddharth Narayanaswamy · Brooks Paige · Jan-Willem van de Meent · Alban Desmaison · Noah Goodman · Pushmeet Kohli · Frank Wood · Philip Torr -
2016 Poster: Adaptive Neural Compilation »
Rudy Bunel · Alban Desmaison · Pawan K Mudigonda · Pushmeet Kohli · Philip Torr -
2016 Poster: Learning feed-forward one-shot learners »
Luca Bertinetto · João Henriques · Jack Valmadre · Philip Torr · Andrea Vedaldi -
2016 Poster: DISCO Nets : DISsimilarity COefficients Networks »
Diane Bouchacourt · Pawan K Mudigonda · Sebastian Nowozin -
2013 Poster: Higher Order Priors for Joint Intrinsic Image, Objects, and Attributes Estimation »
Vibhav Vineet · Carsten Rother · Philip Torr -
2011 Poster: Learning Anchor Planes for Classification »
Ziming Zhang · Lubor Ladicky · Philip Torr · Amir Saffari -
2011 Demonstration: Online structured-output learning for real-time object tracking and detection »
Sam Hare · Amir Saffari · Philip Torr -
2008 Poster: Improved Moves for Truncated Convex Models »
Pawan K Mudigonda · Philip Torr -
2008 Spotlight: Improved Moves for Truncated Convex Models »
Pawan K Mudigonda · Philip Torr -
2007 Oral: An Analysis of Convex Relaxations for MAP Estimation »
Pawan K Mudigonda · Vladimir Kolmogorov · Philip Torr -
2007 Poster: An Analysis of Convex Relaxations for MAP Estimation »
Pawan K Mudigonda · Vladimir Kolmogorov · Philip Torr