Timezone: »

 
Poster
Sobolev Independence Criterion
Youssef Mroueh · Tom Sercu · Mattia Rigotti · Inkit Padhi · Cicero Nogueira dos Santos

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #46
in Algorithms -- Components Analysis (e.g., CCA, ICA, LDA, PCA) »

We propose the Sobolev Independence Criterion (SIC), an interpretable dependency measure between a high dimensional random variable X and a response variable Y. SIC decomposes to the sum of feature importance scores and hence can be used for nonlinear feature selection. SIC can be seen as a gradient regularized Integral Probability Metric (IPM) between the joint distribution of the two random variables and the product of their marginals. We use sparsity inducing gradient penalties to promote input sparsity of the critic of the IPM. In the kernel version we show that SIC can be cast as a convex optimization problem by introducing auxiliary variables that play an important role in feature selection as they are normalized feature importance scores. We then present a neural version of SIC where the critic is parameterized as a homogeneous neural network, improving its representation power as well as its interpretability. We conduct experiments validating SIC for feature selection in synthetic and real-world experiments. We show that SIC enables reliable and interpretable discoveries, when used in conjunction with the holdout randomization test and knockoffs to control the False Discovery Rate. Code is available at http://github.com/ibm/sic.

Keywords: Algorithms · Algorithms -> Components Analysis (e.g., CCA, ICA, LDA, PCA); Algorithms -> Kernel Methods; Algorithms · Sparse Coding and Dim
[ Paper]  [ Slides] 

Author Information

Youssef Mroueh (IBM T.J Watson Research Center)
Tom Sercu (Facebook AI Research)
Mattia Rigotti (IBM Research AI)
Inkit Padhi (IBM Research)
Cicero Nogueira dos Santos (Amazon AWS AI)

More from the Same Authors