A Monte Carlo Metropolis-Hastings Algorithm for Sampling from Distributions with Intractable Normalizing Constants
Type
ArticleAuthors
Liang, FamingJin, Ick-Hoon
KAUST Grant Number
KUS-C1-016-04Date
2013-08Permanent link to this record
http://hdl.handle.net/10754/597314
Metadata
Show full item recordAbstract
Simulating from distributions with intractable normalizing constants has been a long-standing problem inmachine learning. In this letter, we propose a new algorithm, the Monte Carlo Metropolis-Hastings (MCMH) algorithm, for tackling this problem. The MCMH algorithm is a Monte Carlo version of the Metropolis-Hastings algorithm. It replaces the unknown normalizing constant ratio by a Monte Carlo estimate in simulations, while still converges, as shown in the letter, to the desired target distribution under mild conditions. The MCMH algorithm is illustrated with spatial autologistic models and exponential random graph models. Unlike other auxiliary variable Markov chain Monte Carlo (MCMC) algorithms, such as the Møller and exchange algorithms, the MCMH algorithm avoids the requirement for perfect sampling, and thus can be applied to many statistical models for which perfect sampling is not available or very expensive. TheMCMHalgorithm can also be applied to Bayesian inference for random effect models and missing data problems that involve simulations from a distribution with intractable integrals. © 2013 Massachusetts Institute of Technology.Citation
Liang F, Jin I-H (2013) A Monte Carlo Metropolis-Hastings Algorithm for Sampling from Distributions with Intractable Normalizing Constants. Neural Computation 25: 2199–2234. Available: http://dx.doi.org/10.1162/NECO_a_00466.Sponsors
We thank the editor, associate editor, and two referees for their comments, which have led to significant improvement of this letter. F: L.'s research was partially supported by grants from the National Science Foundation (DMS-1007457 and DMS-1106494) and the award (KUS-C1-016-04) made by King Abdullah University of Science and Technology (KAUST).Publisher
MIT Press - JournalsJournal
Neural ComputationPubMed ID
23607562ae974a485f413a2113503eed53cd6c53
10.1162/NECO_a_00466
Scopus Count
Collections
Publications Acknowledging KAUST SupportRelated articles
- Bayesian phylogeny analysis via stochastic approximation Monte Carlo.
- Authors: Cheon S, Liang F
- Issue date: 2009 Nov
- Estimating the granularity coefficient of a Potts-Markov random field within a Markov chain Monte Carlo algorithm.
- Authors: Pereyra M, Dobigeon N, Batatia H, Tourneret JY
- Issue date: 2013 Jun
- Correcting for ascertainment bias in the inference of population structure.
- Authors: Guillot G, Foll M
- Issue date: 2009 Feb 15
- A quasi-Monte Carlo Metropolis algorithm.
- Authors: Owen AB, Tribble SD
- Issue date: 2005 Jun 21
- Searching for efficient Markov chain Monte Carlo proposal kernels.
- Authors: Yang Z, Rodríguez CE
- Issue date: 2013 Nov 26