Timezone: »
Poster
Grouped Orthogonal Matching Pursuit for Variable Selection and Prediction
Aurelie Lozano · Grzegorz M Swirszcz · Naoki Abe
We consider the problem of variable group selection for least squares regression, namely, that of selecting groups of variables for best regression performance, leveraging and adhering to a natural grouping structure within the explanatory variables. We show that this problem can be efficiently addressed by using a certain greedy style algorithm. More precisely, we propose the Group Orthogonal Matching Pursuit algorithm (Group-OMP), which extends the standard OMP procedure (also referred to as ``forward greedy feature selection algorithm for least squares regression) to perform stage-wise group variable selection. We prove that under certain conditions Group-OMP can identify the correct (groups of) variables. We also provide an upperbound on the $l_\infty$ norm of the difference between the estimated regression coefficients and the true coefficients. Experimental results on simulated and real world datasets indicate that Group-OMP compares favorably to Group Lasso, OMP and Lasso, both in terms of variable selection and prediction accuracy.
Author Information
Aurelie Lozano (IBM Research)
Grzegorz M Swirszcz (IBM T.J. Watson Research Center)
Naoki Abe (IBM Research AI)
More from the Same Authors
-
2021 Poster: Adaptive Proximal Gradient Methods for Structured Neural Networks »
Jihun Yun · Aurelie Lozano · Eunho Yang -
2021 Poster: Cardinality-Regularized Hawkes-Granger Model »
Tsuyoshi Ide · Georgios Kollias · Dzung Phan · Naoki Abe -
2015 Poster: Closed-form Estimators for High-dimensional Generalized Linear Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2015 Spotlight: Closed-form Estimators for High-dimensional Generalized Linear Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2015 Poster: Robust Gaussian Graphical Modeling with the Trimmed Graphical Lasso »
Eunho Yang · Aurelie Lozano -
2014 Workshop: Out of the Box: Robustness in High Dimension »
Aurelie Lozano · Aleksandr Y Aravkin · Stephen Becker -
2014 Session: Oral Session 10 »
Aurelie Lozano -
2014 Poster: Elementary Estimators for Graphical Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2011 Poster: Non-parametric Group Orthogonal Matching Pursuit for Sparse Learning with Multiple Kernels »
Vikas Sindhwani · Aurelie Lozano -
2010 Workshop: Practical Application of Sparse Modeling: Open Issues and New Directions »
Irina Rish · Alexandru Niculescu-Mizil · Guillermo Cecchi · Aurelie Lozano -
2010 Poster: Block Variable Selection in Multivariate Regression and High-dimensional Causal Inference »
Aurelie Lozano · Vikas Sindhwani