Timezone: »
Hierarchical Bayesian methods enable information sharing across regression problems on multiple groups of data. While standard practice is to model regression parameters (effects) as (1) exchangeable across the groups and (2) correlated to differing degrees across covariates, we show that this approach exhibits poor statistical performance when the number of covariates exceeds the number of groups. For instance, in statistical genetics, we might regress dozens of traits (defining groups) for thousands of individuals (responses) on up to millions of genetic variants (covariates). When an analyst has more covariates than groups, we argue that it is often preferable to instead model effects as (1) exchangeable across covariates and (2) correlated to differing degrees across groups. To this end, we propose a hierarchical model expressing our alternative perspective. We devise an empirical Bayes estimator for learning the degree of correlation between groups. We develop theory that demonstrates that our method outperforms the classic approach when the number of covariates dominates the number of groups, and corroborate this result empirically on several high-dimensional multiple regression and classification problems.
Author Information
Brian Trippe (Massachusetts Institute of Technology)
Hilary Finucane (Broad Institute of Harvard/MIT)
Tamara Broderick (MIT)
More from the Same Authors
-
2022 : Gaussian processes at the Helm(holtz): A better way to model ocean currents »
Renato Berlinghieri · Tamara Broderick · Ryan Giordano · Tamay Ozgokmen · Kaushik Srinivasan · Brian Trippe · Junfei Xia -
2021 Poster: Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression »
Will Stephenson · Zachary Frangella · Madeleine Udell · Tamara Broderick -
2020 : Hilary Finucane »
Hilary Finucane -
2014 Workshop: Advances in Variational Inference »
David Blei · Shakir Mohamed · Michael Jordan · Charles Blundell · Tamara Broderick · Matthew D. Hoffman -
2013 Poster: Optimistic Concurrency Control for Distributed Unsupervised Learning »
Xinghao Pan · Joseph Gonzalez · Stefanie Jegelka · Tamara Broderick · Michael Jordan -
2013 Poster: Streaming Variational Bayes »
Tamara Broderick · Nicholas Boyd · Andre Wibisono · Ashia C Wilson · Michael Jordan