Timezone: »

SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression
Steve Yadlowsky · Taedong Yun · Cory Y McLean · Alexander D'Amour

Logistic regression remains one of the most widely used tools in applied statistics, machine learning and data science. However, in moderately high-dimensional problems, where the number of features $d$ is a non-negligible fraction of the sample size $n$, the logistic regression maximum likelihood estimator (MLE), and statistical procedures based the large-sample approximation of its distribution, behave poorly. Recently, Sur and Candès (2019) showed that these issues can be corrected by applying a new approximation of the MLE's sampling distribution in this high-dimensional regime. Unfortunately, these corrections are difficult to implement in practice, because they require an estimate of the \emph{signal strength}, which is a function of the underlying parameters $\beta$ of the logistic regression. To address this issue, we propose SLOE, a fast and straightforward approach to estimate the signal strength in logistic regression. The key insight of SLOE is that the Sur and Candès (2019) correction can be reparameterized in terms of the corrupted signal strength, which is only a function of the estimated parameters $\widehat \beta$. We propose an estimator for this quantity, prove that it is consistent in the relevant high-dimensional regime, and show that dimensionality correction using SLOE is accurate in finite samples. Compared to the existing ProbeFrontier heuristic, SLOE is conceptually simpler and orders of magnitude faster, making it suitable for routine use. We demonstrate the importance of routine dimensionality correction in the Heart Disease dataset from the UCI repository, and a genomics application using data from the UK Biobank.

Author Information

Steve Yadlowsky (Stanford University)
Taedong Yun (Google Research)
Cory Y McLean (Google LLC)

Cory is a staff software engineer in Google Health who leads the Genomics research team. His research interests broadly include applying machine learning to the analysis and interpretation of genomic data and publishing tools and methods as open-source software. Prior to Google, Cory was at 23andMe where he developed algorithms and tools to improve identity-by-descent detection, haplotype phasing, and genotype imputation, and the application of genetic association study results to drug development. Cory received a PhD in computer science from Stanford, where he developed computational methods to understand vertebrate gene regulation, and a BS in computer science from MIT.

Alexander D'Amour (UC Berkeley)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors