Timezone: »

 
Spotlight
A Spectral Algorithm for Latent Dirichlet Allocation
Anima Anandkumar · Dean P Foster · Daniel Hsu · Sham M Kakade · Yi-Kai Liu

Wed Dec 05 05:56 PM -- 06:00 PM (PST) @ Harveys Convention Center Floor, CC
in Spotlight Session 8 »
Topic modeling is a generalization of clustering that posits that observations (words in a document) are generated by \emph{multiple} latent factors (topics), as opposed to just one. This increased representational power comes at the cost of a more challenging unsupervised learning problem of estimating the topic-word distributions when only words are observed, and the topics are hidden. This work provides a simple and efficient learning procedure that is guaranteed to recover the parameters for a wide class of topic models, including Latent Dirichlet Allocation (LDA). For LDA, the procedure correctly recovers both the topic-word distributions and the parameters of the Dirichlet prior over the topic mixtures, using only trigram statistics (\emph{i.e.}, third order moments, which may be estimated with documents containing just three words). The method, called Excess Correlation Analysis, is based on a spectral decomposition of low-order moments via two singular value decompositions (SVDs). Moreover, the algorithm is scalable, since the SVDs are carried out only on $k \times k$ matrices, where $k$ is the number of latent factors (topics) and is typically much smaller than the dimension of the observation (word) space.

Author Information

Anima Anandkumar (NVIDIA / Caltech)
Dean P Foster (University of Pennsylvania)
Daniel Hsu (Columbia University)

See <https://www.cs.columbia.edu/~djhsu/>

Sham M Kakade (Harvard University &amp; Amazon)
Yi-Kai Liu (National Institute of Standards and Technology)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors