Poster
Learning Sparse Distributions using Iterative Hard Thresholding
Jacky Y Zhang · Rajiv Khanna · Anastasios Kyrillidis · Oluwasanmi Koyejo

Thu Dec 12th 10:45 AM -- 12:45 PM @ East Exhibition Hall B + C #202
in Optimization -- Non-Convex Optimization »

Iterative hard thresholding (IHT) is a projected gradient descent algorithm, known to achieve state of the art performance for a wide range of structured estimation problems, such as sparse inference. In this work, we consider IHT as a solution to the problem of learning sparse discrete distributions. We study the hardness of using IHT on the space of measures. As a practical alternative, we propose a greedy approximate projection which simultaneously captures appropriate notions of sparsity in distributions, while satisfying the simplex constraint, and investigate the convergence behavior of the resulting procedure in various settings. Our results show, both in theory and practice, that IHT can achieve state of the art results for learning sparse distributions.

Keywords: Probabilistic Methods · Algorithms -> Density Estimation; Optimization -> Combinatorial Optimization; Optimization -> Non-Convex Optimization; Theory -

Author Information

Jacky Y Zhang (UIUC)
Rajiv Khanna (University of California at Berkeley)
Anastasios Kyrillidis (Rice University)
Sanmi Koyejo (UIUC)

Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in the development and analysis of probabilistic and statistical machine learning techniques motivated by, and applied to various modern big data problems. He is particularly interested in the analysis of large scale neuroimaging data. Koyejo completed his Ph.D in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh, and completed postdoctoral research at Stanford University with a focus on developing Machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including the outstanding NCE/ECE student award, a best student paper award from the conference on uncertainty in artificial intelligence (UAI) and a trainee award from the Organization for Human Brain Mapping (OHBM).

More from the Same Authors