About This Page

This page collects resources (slides, tutorials and papers) and links to events (conferences and workshops) dealing with the application of Bayesian methods to natural language processing problems (and, more widely, to text problems in general). To contribute to the page, please contact Hal at .

Tutorial Slides

The slides from the tutorial I gave at HLT/NAACL 2006 in New York on June 4th are available in OpenDocument or PDF format. There will shortly be an accompanying text document convering roughly the same material, but with more details on the derivations of various algorithms.

Workshop

Yee Whye Teh and I organized a workshop, Bayesian Methods in NLP that took place in December 2005 at NIPS. The workshop led to a discussion on the NLP blog.

Bibliography

Applications

• David Blei, Andrew Ng and Michael Jordan. Latent Dirichlet allocation, JMLR03.
• Kobus Barnard, Pinar Duygulu, David Forsyth, Nando de Freitas, David Blei and Michael Jordan. Matching words and pictures. JMLR03.
• Hal Daume III and Daniel Marcu. Bayesian Query-Focused Summarization. ACL06.
• Tom Griffiths, Mark Steyvers, David Blei and Joshua Tenenbaum. Integrating topics and syntax. NIPS04.
• Andrew McCallum, Andres Corrada-Emmanuel and Xuerui Wang. Topic and Role Discovery in Social Networks. IJCAI05.
• Yi Zhang, Jamie Callan and Tom Minka. Novelty and Redundancy Detection in Adaptive Filtering. SIGIR02.

• James O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer, 1985.
• David MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
• Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2003.
• Christopher Bishop. Pattern Recognition and Machine Learning. Springer, 2006.

• Christophe Andrieu, Nando de Freitas, Arnaud Doucet and Michael Jordan. An Introduction to MCMC for Machine Learning. ML 2003
• Martin Wainwright and Michael Jordan. Graphical models, exponential families and variational inference. UCB Stat TR#649, 2003.
• Kevin Murphy. A Brief Introduction to Graphical Models and Bayesian Networks.

• Tom Minka. Using lower bounds to approximate integrals. 2003.
• Neil Lawrence. Fast sparse Gaussian process methods: the informative vector machine. NIPS 2003.
• Tom Minka. Expectation Propagation for Approximate Bayesian Inference. UAI 2001.
• Tom Minka. Divergence Measures and Message Passing. AI-Stats 2005.
• Radford Neal. Markov chain sampling methods for Dirichlet process mixture models. TR. 9815, Dept. of Statistics, University of Toronto.

People

• David Blei, Princeton University
• Hal Daume III, University of Southern California
• Tom Griffiths, Brown University
• Michael Jordan, University of California, Berkeley
• John Lafferty, Carnegie Mellon University
• Andrew McCallum, University of Massachusetts Amherst
• Tom Minka, Microsoft Research, Cambridge
• Mark Steyvers, University of California, Irvine
• Yee Whye Teh, National University of Singapore
• Josh Tenenbaum, Massachusetts Institute of Technology
• Hanna Wallach, Cambridge University
• Eric Xing, Carnegie Mellon University
• ...you? email me.

More to come...

Last updated 3 June 2006.