Timezone: »
We present a general framework for constructing prior distributions with structured variables. The prior is defined as the information projection of a base distribution onto distributions supported on the constraint set of interest. In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain. We further analyze the special case of sparse structure. While the optimal prior is intractable in general, we show that approximate inference using convex subsets is tractable, and is equivalent to maximizing a submodular function subject to cardinality constraints. As a result, inference using greedy forward selection provably achieves within a factor of (1-1/e) of the optimal objective value. Our work is motivated by the predictive modeling of high-dimensional functional neuroimaging data. For this task, we employ the Gaussian base distribution induced by local partial correlations and consider the design of priors to capture the domain knowledge of sparse support. Experimental results on simulated data and high dimensional neuroimaging data show the effectiveness of our approach in terms of support recovery and predictive accuracy.
Author Information
Sanmi Koyejo (Illinois / Google)
Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in the development and analysis of probabilistic and statistical machine learning techniques motivated by, and applied to various modern big data problems. He is particularly interested in the analysis of large scale neuroimaging data. Koyejo completed his Ph.D in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh, and completed postdoctoral research at Stanford University with a focus on developing Machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including the outstanding NCE/ECE student award, a best student paper award from the conference on uncertainty in artificial intelligence (UAI) and a trainee award from the Organization for Human Brain Mapping (OHBM).
Rajiv Khanna (University of California at Berkeley)
Joydeep Ghosh (UT Austin)
Russell Poldrack (University of Texas)
More from the Same Authors
-
2020 Poster: CSER: Communication-efficient SGD with Error Reset »
Cong Xie · Shuai Zheng · Sanmi Koyejo · Indranil Gupta · Mu Li · Haibin Lin -
2020 Poster: Fairness with Overlapping Groups; a Probabilistic Perspective »
Forest Yang · Mouhamadou M Cisse · Sanmi Koyejo -
2020 Poster: Fair Performance Metric Elicitation »
Gaurush Hiranandani · Harikrishna Narasimhan · Sanmi Koyejo -
2019 Poster: Learning Sparse Distributions using Iterative Hard Thresholding »
Jacky Zhang · Rajiv Khanna · Anastasios Kyrillidis · Sanmi Koyejo -
2019 Poster: Multiclass Performance Metric Elicitation »
Gaurush Hiranandani · Shant Boodaghians · Ruta Mehta · Sanmi Koyejo -
2019 Tutorial: Representation Learning and Fairness »
Moustapha Cisse · Sanmi Koyejo -
2018 Poster: Boosting Black Box Variational Inference »
Francesco Locatello · Gideon Dresdner · Rajiv Khanna · Isabel Valera · Gunnar Raetsch -
2018 Spotlight: Boosting Black Box Variational Inference »
Francesco Locatello · Gideon Dresdner · Rajiv Khanna · Isabel Valera · Gunnar Raetsch -
2016 Oral: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2016 Poster: Generalized Correspondence-LDA Models (GC-LDA) for Identifying Functional Regions in the Brain »
Timothy Rubin · Sanmi Koyejo · Michael Jones · Tal Yarkoni -
2016 Poster: Preference Completion from Partial Rankings »
Suriya Gunasekar · Sanmi Koyejo · Joydeep Ghosh -
2016 Poster: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2015 Poster: Unified View of Matrix Completion under General Structural Constraints »
Suriya Gunasekar · Arindam Banerjee · Joydeep Ghosh -
2015 Poster: Consistent Multilabel Classification »
Oluwasanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2014 Poster: Consistent Binary Classification with Generalized Performance Metrics »
Sanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2014 Spotlight: Consistent Binary Classification with Generalized Performance Metrics »
Sanmi Koyejo · Nagarajan Natarajan · Pradeep Ravikumar · Inderjit Dhillon -
2014 Poster: Sparse Bayesian structure learning with dependent relevance determination prior »
Anqi Wu · Mijung Park · Sanmi Koyejo · Jonathan W Pillow -
2013 Poster: BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables »
Cho-Jui Hsieh · Matyas A Sustik · Inderjit Dhillon · Pradeep Ravikumar · Russell Poldrack -
2013 Oral: BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables »
Cho-Jui Hsieh · Matyas A Sustik · Inderjit Dhillon · Pradeep Ravikumar · Russell Poldrack