The online home for the publications of the American Statistical Association

Cookies Notification

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more.
Advanced and citation search

Journal of the American Statistical Association

Volume 101, Issue 476, 2006

Hierarchical Dirichlet Processes

Hierarchical Dirichlet Processes

DOI:
10.1198/016214506000000302
Yee Whye Teha, Michael I Jordana, Matthew J Beala & David M Bleia

pages 1566-1581

Article Views: 2274
Article usage statistics combine cumulative total PDF downloads and full-text HTML views from publication date (but no earlier than 25 Jun 2011, launch date of this website) to 20 Apr 2015. Article views are only counted from this site. Although these data are updated every 24 hours, there may be a 48-hour delay before the most recent numbers are available.

Abstract

We consider problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups. We assume that the number of mixture components is unknown a priori and is to be inferred from the data. In this setting it is natural to consider sets of Dirichlet processes, one for each group, where the well-known clustering property of the Dirichlet process provides a nonparametric prior for the number of mixture components within each group. Given our desire to tie the mixture models in the various groups, we consider a hierarchical model, specifically one in which the base measure for the child Dirichlet processes is itself distributed according to a Dirichlet process. Such a base measure being discrete, the child Dirichlet processes necessarily share atoms. Thus, as desired, the mixture models in the different groups necessarily share mixture components. We discuss representations of hierarchical Dirichlet processes in terms of a stick-breaking process, and a generalization of the Chinese restaurant process that we refer to as the “Chinese restaurant franchise.” We present Markov chain Monte Carlo algorithms for posterior inference in hierarchical Dirichlet process mixtures and describe applications to problems in information retrieval and text modeling.

KEY WORDS

 

Details

  • Published online: 01 Jan 2012

Author affiliations

  • a Yee Whye Teh is Lee Kuan Yew Postdoctoral Fellow, Department of Computer Science, National University of Singapore, Singapore . Michael I. Jordan is Professor of Electrical Engineering and Computer Science and Professor of Statistics, University of California, Berkeley, CA 94720 . Matthew J. Beal is Assistant Professor of Computer Science and Engineering, SUNY Buffalo, Buffalo, NY 14260 . David M. Blei is Assistant Professor of Computer Science, Princeton University, Princeton, NJ . Correspondence should be directed to Michael I. Jordan. This work was supported in part by Intel Corporation, Microsoft Research, and a grant from Darpa under contract number NBCHD030010. The authors wish to acknowledge helpful discussions with Lancelot James and Jim Pitman, and to thank the referees for useful comments.

Journal news