Timezone: »

 
Poster
Learning Infinite RBMs with Frank-Wolfe
Wei Ping · Qiang Liu · Alexander Ihler

Mon Dec 05 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #67
in Monday Posters »

In this work, we propose an infinite restricted Boltzmann machine (RBM), whose maximum likelihood estimation (MLE) corresponds to a constrained convex optimization. We consider the Frank-Wolfe algorithm to solve the program, which provides a sparse solution that can be interpreted as inserting a hidden unit at each iteration, so that the optimization process takes the form of a sequence of finite models of increasing complexity. As a side benefit, this can be used to easily and efficiently identify an appropriate number of hidden units during the optimization. The resulting model can also be used as an initialization for typical state-of-the-art RBM training algorithms such as contrastive divergence, leading to models with consistently higher test likelihood than random initialization.

Keywords: Graphical Models
[ PDF] 

Author Information

Wei Ping (UC Irvine)
Qiang Liu (Dartmouth College)
Alexander Ihler (UC Irvine)

More from the Same Authors