Accelerated Dimension-Independent Adaptive Metropolis

Handle URI:
http://hdl.handle.net/10754/621839
Title:
Accelerated Dimension-Independent Adaptive Metropolis
Authors:
Chen, Yuxin; Keyes, David E. ( 0000-0002-4052-7224 ) ; Law, Kody J H; Ltaief, Hatem ( 0000-0002-6897-1095 )
Abstract:
This work describes improvements by algorithmic and architectural means to black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm [H. Haario, E. Saksman, and J. Tamminen, Bernoulli, (2001), pp. 223--242] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional (with dimension $d \geq 1000$) targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justified a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL (math kernel library) parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) with fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.
KAUST Department:
Computer Science Program; Extreme Computing Research Center; Applied Mathematics and Computational Science Program
Citation:
Chen Y, Keyes D, Law KJH, Ltaief H (2016) Accelerated Dimension-Independent Adaptive Metropolis. SIAM Journal on Scientific Computing 38: S539–S565. Available: http://dx.doi.org/10.1137/15M1026432.
Publisher:
Society for Industrial & Applied Mathematics (SIAM)
Journal:
SIAM Journal on Scientific Computing
Issue Date:
27-Oct-2016
DOI:
10.1137/15M1026432
Type:
Article
ISSN:
1064-8275; 1095-7197
Sponsors:
This work was supported by the King Abdullah University of Science and Technology (KAUST). The work of the third author was partially supported by Oak Ridge National Laboratory Directed Research and Development Strategic Hire grant 32112590 LDRD
Additional Links:
http://epubs.siam.org/doi/10.1137/15M1026432
Appears in Collections:
Articles; Applied Mathematics and Computational Science Program; Extreme Computing Research Center; Computer Science Program

Full metadata record

DC FieldValue Language
dc.contributor.authorChen, Yuxinen
dc.contributor.authorKeyes, David E.en
dc.contributor.authorLaw, Kody J Hen
dc.contributor.authorLtaief, Hatemen
dc.date.accessioned2016-11-21T06:21:06Z-
dc.date.available2016-11-21T06:21:06Z-
dc.date.issued2016-10-27en
dc.identifier.citationChen Y, Keyes D, Law KJH, Ltaief H (2016) Accelerated Dimension-Independent Adaptive Metropolis. SIAM Journal on Scientific Computing 38: S539–S565. Available: http://dx.doi.org/10.1137/15M1026432.en
dc.identifier.issn1064-8275en
dc.identifier.issn1095-7197en
dc.identifier.doi10.1137/15M1026432en
dc.identifier.urihttp://hdl.handle.net/10754/621839-
dc.description.abstractThis work describes improvements by algorithmic and architectural means to black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm [H. Haario, E. Saksman, and J. Tamminen, Bernoulli, (2001), pp. 223--242] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional (with dimension $d \geq 1000$) targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justified a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL (math kernel library) parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) with fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.en
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST). The work of the third author was partially supported by Oak Ridge National Laboratory Directed Research and Development Strategic Hire grant 32112590 LDRDen
dc.publisherSociety for Industrial & Applied Mathematics (SIAM)en
dc.relation.urlhttp://epubs.siam.org/doi/10.1137/15M1026432en
dc.rightsArchived with thanks to SIAM Journal on Scientific Computingen
dc.subjectMarkov chain Monte Carloen
dc.subjectbig dataen
dc.subjectBayesian inferenceen
dc.subjectadaptive Metropolisen
dc.subjectMetropolis-Hastingsen
dc.subjectBLASen
dc.subjectGPU accelerationen
dc.subjecthigh performance computingen
dc.titleAccelerated Dimension-Independent Adaptive Metropolisen
dc.typeArticleen
dc.contributor.departmentComputer Science Programen
dc.contributor.departmentExtreme Computing Research Centeren
dc.contributor.departmentApplied Mathematics and Computational Science Programen
dc.identifier.journalSIAM Journal on Scientific Computingen
dc.eprint.versionPublisher's Version/PDFen
dc.contributor.institutionComputer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN 3783en
kaust.authorChen, Yuxinen
kaust.authorKeyes, David E.en
kaust.authorLtaief, Hatemen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.