Timezone: »
In many problem settings, parameter vectors are not merely sparse, but dependent in such a way that non-zero coefficients tend to cluster together. We refer to this form of dependency as “region sparsity”. Classical sparse regression methods, such as the lasso and automatic relevance determination (ARD), model parameters as independent a priori, and therefore do not exploit such dependencies. Here we introduce a hierarchical model for smooth, region-sparse weight vectors and tensors in a linear regression setting. Our approach represents a hierarchical extension of the relevance determination framework, where we add a transformed Gaussian process to model the dependencies between the prior variances of regression weights. We combine this with a structured model of the prior variances of Fourier coefficients, which eliminates unnecessary high frequencies. The resulting prior encourages weights to be region-sparse in two different bases simultaneously. We develop efficient approximate inference methods and show substantial improvements over comparable methods (e.g., group lasso and smooth RVM) for both simulated and real datasets from brain imaging.
Author Information
Anqi Wu (Princeton University)
Mijung Park (UCL)
Sanmi Koyejo (University of Illinois at Urbana-Champaign & Google Research)
Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in the development and analysis of probabilistic and statistical machine learning techniques motivated by, and applied to various modern big data problems. He is particularly interested in the analysis of large scale neuroimaging data. Koyejo completed his Ph.D in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh, and completed postdoctoral research at Stanford University with a focus on developing Machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including the outstanding NCE/ECE student award, a best student paper award from the conference on uncertainty in artificial intelligence (UAI) and a trainee award from the Organization for Human Brain Mapping (OHBM).
Jonathan W Pillow (UT Austin)
Jonathan Pillow is an assistant professor in Psychology and Neurobiology at the University of Texas at Austin. He graduated from the University of Arizona in 1997 with a degree in mathematics and philosophy, and was a U.S. Fulbright fellow in Morocco in 1998. He received his Ph.D. in neuroscience from NYU in 2005, and was a Royal Society postdoctoral reserach fellow at the Gatsby Computational Neuroscience Unit, UCL from 2005 to 2008. His recent work involves statistical methods for understanding the neural code in single neurons and neural populations, and his lab conducts psychophysical experiments designed to test Bayesian models of human sensory perception.
More from the Same Authors
-
2021 : Probabilistic Performance Metric Elicitation »
Zachary Robertson · Hantao Zhang · Sanmi Koyejo -
2021 : Robust and Personalized Federated Learning with Spurious Features: an Adversarial Approach »
Xiaoyang Wang · Han Zhao · Klara Nahrstedt · Sanmi Koyejo -
2021 : RVFR: Robust Vertical Federated Learning via Feature Subspace Recovery »
Jing Liu · Chulin Xie · Krishnaram Kenthapadi · Sanmi Koyejo · Bo Li -
2021 : Secure Byzantine-Robust Distributed Learning via Clustering »
Raj Kiriti Velicheti · Sanmi Koyejo -
2021 : Exploiting Causal Chains for Domain Generalization »
Olawale Salaudeen · Sanmi Koyejo -
2021 : Distribution Preserving Bayesian Coresets using Set Constraints »
Shovik Guha · Rajiv Khanna · Sanmi Koyejo -
2020 Poster: CSER: Communication-efficient SGD with Error Reset »
Cong Xie · Shuai Zheng · Sanmi Koyejo · Indranil Gupta · Mu Li · Haibin Lin -
2020 Poster: Fairness with Overlapping Groups; a Probabilistic Perspective »
Forest Yang · Mouhamadou M Cisse · Sanmi Koyejo -
2020 Poster: Fair Performance Metric Elicitation »
Gaurush Hiranandani · Harikrishna Narasimhan · Sanmi Koyejo -
2019 Poster: Learning Sparse Distributions using Iterative Hard Thresholding »
Jacky Zhang · Rajiv Khanna · Anastasios Kyrillidis · Sanmi Koyejo -
2019 Poster: Multiclass Performance Metric Elicitation »
Gaurush Hiranandani · Shant Boodaghians · Ruta Mehta · Sanmi Koyejo -
2019 Tutorial: Representation Learning and Fairness »
Moustapha Cisse · Sanmi Koyejo -
2016 : Jonathan Pillow : Scalable Inference for Structured Hierarchical Receptive Field Models »
Jonathan W Pillow -
2016 Oral: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2016 Poster: Generalized Correspondence-LDA Models (GC-LDA) for Identifying Functional Regions in the Brain »
Timothy Rubin · Sanmi Koyejo · Michael Jones · Tal Yarkoni -
2016 Poster: Preference Completion from Partial Rankings »
Suriya Gunasekar · Sanmi Koyejo · Joydeep Ghosh -
2016 Poster: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2015 Poster: Bayesian Manifold Learning: The Locally Linear Latent Variable Model (LL-LVM) »
Mijung Park · Wittawat Jitkrittum · Ahmad Qamar · Zoltan Szabo · Lars Buesing · Maneesh Sahani -
2015 Poster: Consistent Multilabel Classification »
Oluwasanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2015 Poster: Unlocking neural population non-stationarities using hierarchical dynamics models »
Mijung Park · Gergo Bohner · Jakob H Macke -
2014 Poster: On Prior Distributions and Approximate Inference for Structured Variables »
Sanmi Koyejo · Rajiv Khanna · Joydeep Ghosh · Russell Poldrack -
2014 Poster: Consistent Binary Classification with Generalized Performance Metrics »
Sanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2014 Spotlight: Consistent Binary Classification with Generalized Performance Metrics »
Sanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2014 Poster: Optimal prior-dependent neural population codes under shared input noise »
Agnieszka Grabska-Barwinska · Jonathan W Pillow -
2014 Poster: Inferring sparse representations of continuous signals with continuous orthogonal matching pursuit »
Karin C Knudson · Jacob Yates · Alexander Huk · Jonathan W Pillow -
2014 Poster: Inferring synaptic conductances from spike trains with a biophysically inspired point process model »
Kenneth W Latimer · E.J. Chichilnisky · Fred Rieke · Jonathan W Pillow -
2014 Poster: Low-dimensional models of neural population activity in sensory cortical circuits »
Evan W Archer · Urs Koster · Jonathan W Pillow · Jakob H Macke -
2013 Poster: Spike train entropy-rate estimation using hierarchical Dirichlet process priors »
Karin C Knudson · Jonathan W Pillow -
2013 Poster: Bayesian entropy estimation for binary spike train data using parametric prior knowledge »
Evan W Archer · Il Memming Park · Jonathan W Pillow -
2013 Poster: Universal models for binary spike patterns using centered Dirichlet processes »
Il Memming Park · Evan W Archer · Kenneth W Latimer · Jonathan W Pillow -
2013 Spotlight: Bayesian entropy estimation for binary spike train data using parametric prior knowledge »
Evan W Archer · Il Memming Park · Jonathan W Pillow -
2013 Poster: Spectral methods for neural characterization using generalized quadratic models »
Il Memming Park · Evan W Archer · Nicholas Priebe · Jonathan W Pillow -
2013 Poster: Bayesian inference for low rank spatiotemporal neural receptive fields »
Mijung Park · Jonathan W Pillow -
2012 Poster: Fully Bayesian inference for neural models with negative-binomial spiking »
Jonathan W Pillow · James Scott -
2012 Poster: Bayesian active learning with localized priors for fast receptive field characterization »
Mijung Park · Jonathan W Pillow -
2012 Poster: Bayesian estimation of discrete entropy with mixtures of stick-breaking priors »
Evan W Archer · Jonathan W Pillow · Il Memming Park -
2011 Session: Oral Session 13 »
Jonathan W Pillow -
2011 Poster: Bayesian Spike-Triggered Covariance Analysis »
Il Memming Park · Jonathan W Pillow -
2011 Poster: Active learning of neural response functions with Gaussian processes »
Mijung Park · Greg Horwitz · Jonathan W Pillow -
2011 Spotlight: Active learning of neural response functions with Gaussian processes »
Mijung Park · Greg Horwitz · Jonathan W Pillow -
2011 Tutorial: Flexible, Multivariate Point Process Models for Unlocking the Neural Code »
Jonathan W Pillow -
2009 Oral: Time-rescaling Methods for the Estimation and Assessment of Non-Poisson Neural Encoding Models »
Jonathan W Pillow -
2009 Poster: Time-rescaling methods for the estimation and assessment of non-Poisson neural encoding models »
Jonathan W Pillow -
2008 Poster: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow -
2008 Spotlight: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow -
2007 Oral: Neural characterization in partially observed populations of spiking neurons »
Jonathan W Pillow · Peter E Latham -
2007 Poster: Neural characterization in partially observed populations of spiking neurons »
Jonathan W Pillow · Peter E Latham