Markov chain Monte Carlo with the Integrated Nested Laplace Approximation

Handle URI:
http://hdl.handle.net/10754/625832
Title:
Markov chain Monte Carlo with the Integrated Nested Laplace Approximation
Authors:
Gómez-Rubio, Virgilio ( 0000-0002-4791-3072 ) ; Rue, Haavard ( 0000-0002-0222-1881 )
Abstract:
The Integrated Nested Laplace Approximation (INLA) has established itself as a widely used method for approximate inference on Bayesian hierarchical models which can be represented as a latent Gaussian model (LGM). INLA is based on producing an accurate approximation to the posterior marginal distributions of the parameters in the model and some other quantities of interest by using repeated approximations to intermediate distributions and integrals that appear in the computation of the posterior marginals. INLA focuses on models whose latent effects are a Gaussian Markov random field. For this reason, we have explored alternative ways of expanding the number of possible models that can be fitted using the INLA methodology. In this paper, we present a novel approach that combines INLA and Markov chain Monte Carlo (MCMC). The aim is to consider a wider range of models that can be fitted with INLA only when some of the parameters of the model have been fixed. We show how new values of these parameters can be drawn from their posterior by using conditional models fitted with INLA and standard MCMC algorithms, such as Metropolis–Hastings. Hence, this will extend the use of INLA to fit models that can be expressed as a conditional LGM. Also, this new approach can be used to build simpler MCMC samplers for complex models as it allows sampling only on a limited number of parameters in the model. We will demonstrate how our approach can extend the class of models that could benefit from INLA, and how the R-INLA package will ease its implementation. We will go through simple examples of this new approach before we discuss more advanced applications with datasets taken from the relevant literature. In particular, INLA within MCMC will be used to fit models with Laplace priors in a Bayesian Lasso model, imputation of missing covariates in linear models, fitting spatial econometrics models with complex nonlinear terms in the linear predictor and classification of data with mixture models. Furthermore, in some of the examples we could exploit INLA within MCMC to make joint inference on an ensemble of model parameters.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Gómez-Rubio V, Rue H (2017) Markov chain Monte Carlo with the Integrated Nested Laplace Approximation. Statistics and Computing. Available: http://dx.doi.org/10.1007/s11222-017-9778-y.
Publisher:
Springer Nature
Journal:
Statistics and Computing
Issue Date:
6-Oct-2017
DOI:
10.1007/s11222-017-9778-y
ARXIV:
arXiv:1701.07844
Type:
Article
ISSN:
0960-3174; 1573-1375
Sponsors:
Virgilio Gómez-Rubio has been supported by Grant PPIC-2014-001, funded by Consejería de Educación, Cultura y Deportes (JCCM) and FEDER, and Grant MTM2016-77501-P, funded by Ministerio de Economía y Competitividad. We would also like to thank Prof. Aki Vehtari for his comments on a preliminary version of this paper.
Additional Links:
http://link.springer.com/article/10.1007/s11222-017-9778-y
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorGómez-Rubio, Virgilioen
dc.contributor.authorRue, Haavarden
dc.date.accessioned2017-10-09T06:12:01Z-
dc.date.available2017-10-09T06:12:01Z-
dc.date.issued2017-10-06en
dc.identifier.citationGómez-Rubio V, Rue H (2017) Markov chain Monte Carlo with the Integrated Nested Laplace Approximation. Statistics and Computing. Available: http://dx.doi.org/10.1007/s11222-017-9778-y.en
dc.identifier.issn0960-3174en
dc.identifier.issn1573-1375en
dc.identifier.doi10.1007/s11222-017-9778-yen
dc.identifier.urihttp://hdl.handle.net/10754/625832-
dc.description.abstractThe Integrated Nested Laplace Approximation (INLA) has established itself as a widely used method for approximate inference on Bayesian hierarchical models which can be represented as a latent Gaussian model (LGM). INLA is based on producing an accurate approximation to the posterior marginal distributions of the parameters in the model and some other quantities of interest by using repeated approximations to intermediate distributions and integrals that appear in the computation of the posterior marginals. INLA focuses on models whose latent effects are a Gaussian Markov random field. For this reason, we have explored alternative ways of expanding the number of possible models that can be fitted using the INLA methodology. In this paper, we present a novel approach that combines INLA and Markov chain Monte Carlo (MCMC). The aim is to consider a wider range of models that can be fitted with INLA only when some of the parameters of the model have been fixed. We show how new values of these parameters can be drawn from their posterior by using conditional models fitted with INLA and standard MCMC algorithms, such as Metropolis–Hastings. Hence, this will extend the use of INLA to fit models that can be expressed as a conditional LGM. Also, this new approach can be used to build simpler MCMC samplers for complex models as it allows sampling only on a limited number of parameters in the model. We will demonstrate how our approach can extend the class of models that could benefit from INLA, and how the R-INLA package will ease its implementation. We will go through simple examples of this new approach before we discuss more advanced applications with datasets taken from the relevant literature. In particular, INLA within MCMC will be used to fit models with Laplace priors in a Bayesian Lasso model, imputation of missing covariates in linear models, fitting spatial econometrics models with complex nonlinear terms in the linear predictor and classification of data with mixture models. Furthermore, in some of the examples we could exploit INLA within MCMC to make joint inference on an ensemble of model parameters.en
dc.description.sponsorshipVirgilio Gómez-Rubio has been supported by Grant PPIC-2014-001, funded by Consejería de Educación, Cultura y Deportes (JCCM) and FEDER, and Grant MTM2016-77501-P, funded by Ministerio de Economía y Competitividad. We would also like to thank Prof. Aki Vehtari for his comments on a preliminary version of this paper.en
dc.publisherSpringer Natureen
dc.relation.urlhttp://link.springer.com/article/10.1007/s11222-017-9778-yen
dc.rightsThe final publication is available at Springer via http://dx.doi.org/10.1007/s11222-017-9778-yen
dc.subjectBayesian Lassoen
dc.subjectINLA MCMCen
dc.subjectMissing valuesen
dc.subjectSpatial modelsen
dc.subjectMixture modelsen
dc.titleMarkov chain Monte Carlo with the Integrated Nested Laplace Approximationen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalStatistics and Computingen
dc.eprint.versionPost-printen
dc.contributor.institutionDepartment of Mathematics, School of Industrial Engineering, Universidad de Castilla-La Mancha, Albacete, Spainen
dc.identifier.arxividarXiv:1701.07844en
kaust.authorRue, Haavarden
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.