Timezone: »
We propose the Sobolev Independence Criterion (SIC), an interpretable dependency measure between a high dimensional random variable X and a response variable Y. SIC decomposes to the sum of feature importance scores and hence can be used for nonlinear feature selection. SIC can be seen as a gradient regularized Integral Probability Metric (IPM) between the joint distribution of the two random variables and the product of their marginals. We use sparsity inducing gradient penalties to promote input sparsity of the critic of the IPM. In the kernel version we show that SIC can be cast as a convex optimization problem by introducing auxiliary variables that play an important role in feature selection as they are normalized feature importance scores. We then present a neural version of SIC where the critic is parameterized as a homogeneous neural network, improving its representation power as well as its interpretability. We conduct experiments validating SIC for feature selection in synthetic and real-world experiments. We show that SIC enables reliable and interpretable discoveries, when used in conjunction with the holdout randomization test and knockoffs to control the False Discovery Rate. Code is available at http://github.com/ibm/sic.
Author Information
Youssef Mroueh (IBM T.J Watson Research Center)
Tom Sercu (Facebook AI Research)
Mattia Rigotti (IBM Research AI)
Inkit Padhi (IBM Research)
Cicero Nogueira dos Santos (Amazon AWS AI)
More from the Same Authors
-
2021 Spotlight: Measuring Generalization with Optimal Transport »
Ching-Yao Chuang · Youssef Mroueh · Kristjan Greenewald · Antonio Torralba · Stefanie Jegelka -
2021 : Optimizing Functionals on the Space of Probabilities with Input Convex Neural Network »
David Alvarez-Melis · Yair Schiff · Youssef Mroueh -
2021 : Optimizing Functionals on the Space of Probabilities with Input Convex Neural Network »
David Alvarez-Melis · Yair Schiff · Youssef Mroueh -
2021 Poster: Measuring Generalization with Optimal Transport »
Ching-Yao Chuang · Youssef Mroueh · Kristjan Greenewald · Antonio Torralba · Stefanie Jegelka -
2021 Poster: Separation Results between Fixed-Kernel and Feature-Learning Probability Metrics »
Carles Domingo i Enrich · Youssef Mroueh -
2021 Oral: Separation Results between Fixed-Kernel and Feature-Learning Probability Metrics »
Carles Domingo i Enrich · Youssef Mroueh -
2020 Poster: Unbalanced Sobolev Descent »
Youssef Mroueh · Mattia Rigotti -
2020 Poster: A Decentralized Parallel Algorithm for Training Generative Adversarial Nets »
Mingrui Liu · Wei Zhang · Youssef Mroueh · Xiaodong Cui · Jarret Ross · Tianbao Yang · Payel Das -
2020 Poster: CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models »
Vijil Chenthamarakshan · Payel Das · Samuel Hoffman · Hendrik Strobelt · Inkit Padhi · Kar Wai Lim · Benjamin Hoover · Matteo Manica · Jannis Born · Teodoro Laino · Aleksandra Mojsilovic -
2017 Poster: Fisher GAN »
Youssef Mroueh · Tom Sercu