Bayesian nonparametrics

Online resources:

Methods

Mixture models

  • Dirichlet process
    • The most basic nonparametric mixture model
    • Aka: Chinese restaurant process, Beta stick-breaking process, GEM model (Griffiths, Engen and McCloskey)
    • Neal, 2000: Markov chain sampling methods for Dirichlet process mixture models (doi, pdf, tech report )
  • Pitman-Yor process, aka two-parameter Poisson-Dirichlet process
    • Generalizes Dirichlet process to allow cluster sizes with power law distribution
    • Teh, 2006: A hierarchical Bayesian language model based on Pitman-Yor processes (doi, tech report )
    • Buntine & Hutter, 2010: A Bayesian view of the Poisson-Dirichlet process (arxiv)
  • Hierarchical Dirichlet process
    • Beal et al, 2002: The infinite hidden Markov model (pdf)
    • Teh et al, 2006: Hierarchical Dirichlet processes (doi, pdf)
    • Mochihashi & Sumita, 2007: The infinite Markov model (pdf)

Latent variable models

  • Indian buffet process
    • Griffiths & Ghahramani, 2011: The Indian buffer process: An introduction and review (pdf)

Regression

  • Gaussian process
    • Rasmussen & Williams, 2006: Gaussian Processes for Machine Learning (online )
    • Neal, 1998: Regression and classification using Gaussian process priors (pdf)
  • BART
    • Chipman, George, McCulloch, 2010: BART: Bayesian additive regression trees (doi, arxiv)

Literature

Notes and surveys

  • Orbanz, 2013: Lecture notes on Bayesian nonparametrics (pdf)
  • Gershman and Blei, 2012: A tutorial on Bayesian nonparametric methods (doi, arxiv, pdf)
    • Friendly introduction to mixture models (CRP) and latent variable models (IBP)
  • Ghahramani, 2012: Bayesian nonparametrics and the probabilistic approach to modeling (doi, pdf)
  • Sudderth, 2006, PhD thesis: Graphical models for visual object recognition and tracking (uri , pdf), Sec 2.5: Dirichlet processes