The Method of Manufactured Universes for validating uncertainty quantification methods

Handle URI:
http://hdl.handle.net/10754/599936
Title:
The Method of Manufactured Universes for validating uncertainty quantification methods
Authors:
Stripling, H.F.; Adams, M.L.; McClarren, R.G.; Mallick, B.K.
Abstract:
The Method of Manufactured Universes is presented as a validation framework for uncertainty quantification (UQ) methodologies and as a tool for exploring the effects of statistical and modeling assumptions embedded in these methods. The framework calls for a manufactured reality from which experimental data are created (possibly with experimental error), an imperfect model (with uncertain inputs) from which simulation results are created (possibly with numerical error), the application of a system for quantifying uncertainties in model predictions, and an assessment of how accurately those uncertainties are quantified. The application presented in this paper manufactures a particle-transport universe, models it using diffusion theory with uncertain material parameters, and applies both Gaussian process and Bayesian MARS algorithms to make quantitative predictions about new experiments within the manufactured reality. The results of this preliminary study indicate that, even in a simple problem, the improper application of a specific UQ method or unrealized effects of a modeling assumption may produce inaccurate predictions. We conclude that the validation framework presented in this paper is a powerful and flexible tool for the investigation and understanding of UQ methodologies. © 2011 Elsevier Ltd. All rights reserved.
Citation:
Stripling HF, Adams ML, McClarren RG, Mallick BK (2011) The Method of Manufactured Universes for validating uncertainty quantification methods. Reliability Engineering & System Safety 96: 1242–1256. Available: http://dx.doi.org/10.1016/j.ress.2010.11.012.
Publisher:
Elsevier BV
Journal:
Reliability Engineering & System Safety
KAUST Grant Number:
KUS-C1-016-04
Issue Date:
Sep-2011
DOI:
10.1016/j.ress.2010.11.012
Type:
Article
ISSN:
0951-8320
Sponsors:
We thank Derek Bingham (Simon Fraser University) for helpful conversations and for sharing some helpful UQ software. Work by the first author was supported by the Department of Energy Computational Science Graduate Fellowship program under Grant no. DE-FG02-97ER25308. Work by other authors was partially supported by the Predictive Sciences Academic Alliances Program in DOE NNSA-ASC under Grant DE-FC52-08NA28616, and the Lawrence Livermore National Laboratory. This publication is based in part on work supported by Award No. KUS-C1-016-04 made by the King Abdullah University of Science and Technology (KAUST). Finally, we are grateful for the effort of one journal referee, whose comments and suggestions greatly strengthened the content of this paper.
Appears in Collections:
Publications Acknowledging KAUST Support

Full metadata record

DC FieldValue Language
dc.contributor.authorStripling, H.F.en
dc.contributor.authorAdams, M.L.en
dc.contributor.authorMcClarren, R.G.en
dc.contributor.authorMallick, B.K.en
dc.date.accessioned2016-02-28T06:32:49Zen
dc.date.available2016-02-28T06:32:49Zen
dc.date.issued2011-09en
dc.identifier.citationStripling HF, Adams ML, McClarren RG, Mallick BK (2011) The Method of Manufactured Universes for validating uncertainty quantification methods. Reliability Engineering & System Safety 96: 1242–1256. Available: http://dx.doi.org/10.1016/j.ress.2010.11.012.en
dc.identifier.issn0951-8320en
dc.identifier.doi10.1016/j.ress.2010.11.012en
dc.identifier.urihttp://hdl.handle.net/10754/599936en
dc.description.abstractThe Method of Manufactured Universes is presented as a validation framework for uncertainty quantification (UQ) methodologies and as a tool for exploring the effects of statistical and modeling assumptions embedded in these methods. The framework calls for a manufactured reality from which experimental data are created (possibly with experimental error), an imperfect model (with uncertain inputs) from which simulation results are created (possibly with numerical error), the application of a system for quantifying uncertainties in model predictions, and an assessment of how accurately those uncertainties are quantified. The application presented in this paper manufactures a particle-transport universe, models it using diffusion theory with uncertain material parameters, and applies both Gaussian process and Bayesian MARS algorithms to make quantitative predictions about new experiments within the manufactured reality. The results of this preliminary study indicate that, even in a simple problem, the improper application of a specific UQ method or unrealized effects of a modeling assumption may produce inaccurate predictions. We conclude that the validation framework presented in this paper is a powerful and flexible tool for the investigation and understanding of UQ methodologies. © 2011 Elsevier Ltd. All rights reserved.en
dc.description.sponsorshipWe thank Derek Bingham (Simon Fraser University) for helpful conversations and for sharing some helpful UQ software. Work by the first author was supported by the Department of Energy Computational Science Graduate Fellowship program under Grant no. DE-FG02-97ER25308. Work by other authors was partially supported by the Predictive Sciences Academic Alliances Program in DOE NNSA-ASC under Grant DE-FC52-08NA28616, and the Lawrence Livermore National Laboratory. This publication is based in part on work supported by Award No. KUS-C1-016-04 made by the King Abdullah University of Science and Technology (KAUST). Finally, we are grateful for the effort of one journal referee, whose comments and suggestions greatly strengthened the content of this paper.en
dc.publisherElsevier BVen
dc.subjectBayesian inferenceen
dc.subjectCalibrationen
dc.subjectEmulatoren
dc.subjectPredictive scienceen
dc.subjectUncertainty quantificationen
dc.subjectValidationen
dc.titleThe Method of Manufactured Universes for validating uncertainty quantification methodsen
dc.typeArticleen
dc.identifier.journalReliability Engineering & System Safetyen
dc.contributor.institutionTexas A and M University, College Station, United Statesen
kaust.grant.numberKUS-C1-016-04en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.