The Method of Manufactured Universes for validating uncertainty quantification methods
KAUST Grant NumberKUS-C1-016-04
MetadataShow full item record
AbstractThe Method of Manufactured Universes is presented as a validation framework for uncertainty quantification (UQ) methodologies and as a tool for exploring the effects of statistical and modeling assumptions embedded in these methods. The framework calls for a manufactured reality from which experimental data are created (possibly with experimental error), an imperfect model (with uncertain inputs) from which simulation results are created (possibly with numerical error), the application of a system for quantifying uncertainties in model predictions, and an assessment of how accurately those uncertainties are quantified. The application presented in this paper manufactures a particle-transport universe, models it using diffusion theory with uncertain material parameters, and applies both Gaussian process and Bayesian MARS algorithms to make quantitative predictions about new experiments within the manufactured reality. The results of this preliminary study indicate that, even in a simple problem, the improper application of a specific UQ method or unrealized effects of a modeling assumption may produce inaccurate predictions. We conclude that the validation framework presented in this paper is a powerful and flexible tool for the investigation and understanding of UQ methodologies. © 2011 Elsevier Ltd. All rights reserved.
CitationStripling HF, Adams ML, McClarren RG, Mallick BK (2011) The Method of Manufactured Universes for validating uncertainty quantification methods. Reliability Engineering & System Safety 96: 1242–1256. Available: http://dx.doi.org/10.1016/j.ress.2010.11.012.
SponsorsWe thank Derek Bingham (Simon Fraser University) for helpful conversations and for sharing some helpful UQ software. Work by the first author was supported by the Department of Energy Computational Science Graduate Fellowship program under Grant no. DE-FG02-97ER25308. Work by other authors was partially supported by the Predictive Sciences Academic Alliances Program in DOE NNSA-ASC under Grant DE-FC52-08NA28616, and the Lawrence Livermore National Laboratory. This publication is based in part on work supported by Award No. KUS-C1-016-04 made by the King Abdullah University of Science and Technology (KAUST). Finally, we are grateful for the effort of one journal referee, whose comments and suggestions greatly strengthened the content of this paper.