Timezone: »

 
Spotlight
Rethinking LDA: Why Priors Matter
Hanna Wallach · David Mimno · Andrew McCallum

Mon Dec 07 06:39 PM -- 06:40 PM (PST) @
in Spotlights »

Implementations of topic models typically use symmetric Dirichlet priors with fixed concentration parameters, with the implicit assumption that such "smoothing parameters" have little practical effect. In this paper, we explore several classes of structured priors for topic models. We find that an asymmetric Dirichlet prior over the document-topic distributions has substantial advantages over a symmetric prior, while an asymmetric prior over the topic-word distributions provides no real benefit. Approximation of this prior structure through simple, efficient hyperparameter optimization steps is sufficient to achieve these performance gains. The prior structure we advocate substantially increases the robustness of topic models to variations in the number of topics and to the highly skewed word frequency distributions common in natural language. Since this prior structure can be implemented using efficient algorithms that add negligible cost beyond standard inference techniques, we recommend it as a new standard for topic modeling.

Author Information

Hanna Wallach (Microsoft)
David Mimno (Cornell University)
Andrew McCallum (UMass Amherst)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors