Timezone: »
Poster
A Universal Analysis of Large-Scale Regularized Least Squares Solutions
Ashkan Panahi · Babak Hassibi
A problem that has been of recent interest in statistical inference, machine learning and signal processing is that of understanding the asymptotic behavior of regularized least squares solutions under random measurement matrices (or dictionaries). The Least Absolute Shrinkage and Selection Operator (LASSO or least-squares with $\ell_1$ regularization) is perhaps one of the most interesting examples. Precise expressions for the asymptotic performance of LASSO have been obtained for a number of different cases, in particular when the elements of the dictionary matrix are sampled independently from a Gaussian distribution. It has also been empirically observed that the resulting expressions remain valid when the entries of the dictionary matrix are independently sampled from certain non-Gaussian distributions. In this paper, we confirm these observations theoretically when the distribution is sub-Gaussian. We further generalize the previous expressions for a broader family of regularization functions and under milder conditions on the underlying random, possibly non-Gaussian, dictionary matrix. In particular, we establish the universality of the asymptotic statistics (e.g., the average quadratic risk) of LASSO with non-Gaussian dictionaries.
Author Information
Ashkan Panahi (North Carolina State University)
Babak Hassibi (Caltech)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Spotlight: A Universal Analysis of Large-Scale Regularized Least Squares Solutions »
Wed. Dec 6th 07:30 -- 07:35 PM Room Hall A
More from the Same Authors
-
2020 Poster: Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems »
Sahin Lale · Kamyar Azizzadenesheli · Babak Hassibi · Anima Anandkumar -
2019 Poster: Universality in Learning from Linear Measurements »
Ehsan Abbasi · Fariborz Salehi · Babak Hassibi -
2019 Poster: The Impact of Regularization on High-dimensional Logistic Regression »
Fariborz Salehi · Ehsan Abbasi · Babak Hassibi -
2018 Poster: Learning without the Phase: Regularized PhaseMax Achieves Optimal Sample Complexity »
Fariborz Salehi · Ehsan Abbasi · Babak Hassibi -
2016 Poster: Fundamental Limits of Budget-Fidelity Trade-off in Label Crowdsourcing »
Farshad Lahouti · Babak Hassibi -
2015 Poster: LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements »
CHRISTOS THRAMPOULIDIS · Ehsan Abbasi · Babak Hassibi -
2015 Spotlight: LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements »
CHRISTOS THRAMPOULIDIS · Ehsan Abbasi · Babak Hassibi