Timezone: »
As the number of AI systems keeps growing, it is fundamental to implement and develop efficient control mechanisms to ensure the safe and proper functioning of machine learning (ML) systems. Reliable out-of-distribution (OOD) detection aims to detect test samples that are statistically far from the training distribution, as they might cause failures of in-production systems. In this paper, we propose a new detector called TRUSTED. Different from previous works, TRUSTED key components (i) include a novel OOD score relying on the concept of statistical data depth, (ii) rely on the idea’s full potential that all hidden layers of the network carry information regarding OOD. Our extensive experiments, comparing over 51k model configurations including different checkpoints, seed and various datasets, demonstrate that TRUSTED achieve state-of-the-art performances by producing an improvement of over 3 AUROC points.
Author Information
Pierre Colombo (MICS CentraleSupelec)
Eduardo Dadalto (CENTRALESUPELEC)
My name is Eduardo Dadalto Câmara Gomes and I'm a second year PhD Student in Machine Learning at L2S, CNRS, Université Paris Saclay. I am working towards safer Deep Learning algorithms, out-of-distribution detection, and misclassification detection.
Guillaume Staerman (Télécom ParisTech)
Nathan Noiry (Télécom Paris)
Pablo Piantanida (CentraleSupelec- CNRS - Université Paris Saclay - L2S - Mila)
Pablo Piantanida received both B.Sc. in Electrical Engineering and Mathematics, and M.Sc degrees from the University of Buenos Aires (Argentina) in 2003, and the Ph.D. from Université Paris-Sud (Orsay, France) in 2007. Since October 2007 he has joined the Laboratoire des Signaux et Systèmes (L2S), at CentraleSupélec together with CNRS (UMR 8506) and Université Paris-Sud, as an Associate Professor of Network Information Theory. He is an IEEE Senior Member, and coordinator of the Information Theory and its Applications group (ITA) at L2S and General Co-Chair of the 2019 IEEE International Symposium on Information Theory (ISIT).
More from the Same Authors
-
2021 : Igeood: An Information Geometry Approach to Out-of-Distribution Detection »
Eduardo Dadalto · Florence Alberge · Pierre Duhamel · Pablo Piantanida -
2022 Poster: What are the best Systems? New Perspectives on NLP Benchmarking »
Pierre Colombo · Nathan Noiry · Ekhine Irurozki · Stephan Clémençon -
2022 Poster: The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset »
Hugo Laurençon · Lucile Saulnier · Thomas Wang · Christopher Akiki · Albert Villanova del Moral · Teven Le Scao · Leandro Von Werra · Chenghao Mou · Eduardo González Ponferrada · Huu Nguyen · Jörg Frohberg · Mario Šaško · Quentin Lhoest · Angelina McMillan-Major · Gerard Dupont · Stella Biderman · Anna Rogers · Loubna Ben allal · Francesco De Toni · Giada Pistilli · Olivier Nguyen · Somaieh Nikpoor · Maraim Masoud · Pierre Colombo · Javier de la Rosa · Paulo Villegas · Tristan Thrush · Shayne Longpre · Sebastian Nagel · Leon Weber · Manuel Muñoz · Jian Zhu · Daniel Van Strien · Zaid Alyafeai · Khalid Almubarak · Minh Chien Vu · Itziar Gonzalez-Dios · Aitor Soroa · Kyle Lo · Manan Dey · Pedro Ortiz Suarez · Aaron Gokaslan · Shamik Bose · David Adelani · Long Phan · Hieu Tran · Ian Yu · Suhas Pai · Jenny Chim · Violette Lepercq · Suzana Ilic · Margaret Mitchell · Sasha Alexandra Luccioni · Yacine Jernite -
2021 Poster: Online Matching in Sparse Random Graphs: Non-Asymptotic Performances of Greedy Algorithm »
Nathan Noiry · Vianney Perchet · Flore Sentenac