Timezone: »

Perfect Dimensionality Recovery by Variational Bayesian PCA
Shinichi Nakajima · Ryota Tomioka · Masashi Sugiyama · S. Derin Babacan

Tue Dec 04 07:00 PM -- 12:00 AM (PST) @ Harrah’s Special Events Center 2nd Floor

The variational Bayesian (VB) approach is one of the best tractable approximations to the Bayesian estimation, and it was demonstrated to perform well in many applications. However, its good performance was not fully understood theoretically. For example, VB sometimes produces a sparse solution, which is regarded as a practical advantage of VB, but such sparsity is hardly observed in the rigorous Bayesian estimation. In this paper, we focus on probabilistic PCA and give more theoretical insight into the empirical success of VB. More specifically, for the situation where the noise variance is unknown, we derive a sufficient condition for perfect recovery of the true PCA dimensionality in the large-scale limit when the size of an observed matrix goes to infinity. In our analysis, we obtain bounds for a noise variance estimator and simple closed-form solutions for other parameters, which themselves are actually very useful for better implementation of VB-PCA.

Author Information

Shinichi Nakajima (TU Berlin)
Ryota Tomioka (Microsoft Research AI4Science)
Masashi Sugiyama (RIKEN / University of Tokyo)
S. Derin Babacan (Google Research)

More from the Same Authors