Poster
On the number of variables to use in principal component regression
Ji Xu · Daniel Hsu
East Exhibition Hall B, C #234
Keywords: [ Frequentist Statistics ] [ Theory ] [ Regularization ] [ Theory -> Large Deviations and Asymptotic Analysis; Theory ]
[
Abstract
]
Abstract:
We study least squares linear regression over uncorrelated Gaussian features that are selected in order of decreasing variance. When the number of selected features is at most the sample size , the estimator under consideration coincides with the principal component regression estimator; when , the estimator is the least norm solution over the selected features. We give an average-case analysis of the out-of-sample prediction error as with and , for some constants and . In this average-case setting, the prediction error exhibits a double descent'' shape as a function of . We also establish conditions under which the minimum risk is achieved in the interpolating () regime.
Live content is unavailable. Log in and register to view live content