Exploiting low-rank covariance structures for computing high-dimensional normal and Student-t probabilities
KAUST DepartmentStatistics Program
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Applied Mathematics and Computational Science Program
Extreme Computing Research Center
Office of the President
Embargo End Date2022-01-12
Permanent link to this recordhttp://hdl.handle.net/10754/666234
MetadataShow full item record
AbstractWe present a preconditioned Monte Carlo method for computing high-dimensional multivariate normal and Student-t probabilities arising in spatial statistics. The approach combines a tile-low-rank representation of covariance matrices with a block-reordering scheme for efficient quasi-Monte Carlo simulation. The tile-low-rank representation decomposes the high-dimensional problem into many diagonal-block-size problems and low-rank connections. The block-reordering scheme reorders between and within the diagonal blocks to reduce the impact of integration variables from right to left, thus improving the Monte Carlo convergence rate. Simulations up to dimension 65,536 suggest that the new method can improve the run time by an order of magnitude compared with the hierarchical quasi-Monte Carlo method and two orders of magnitude compared with the dense quasi-Monte Carlo method. Our method also forms a strong substitute for the approximate conditioning methods as a more robust estimation with error guarantees. An application study to wind stochastic generators is provided to illustrate that the new computational method makes the maximum likelihood estimation feasible for high-dimensional skew-normal random fields.
CitationCao, J., Genton, M. G., Keyes, D. E., & Turkiyyah, G. M. (2021). Exploiting low-rank covariance structures for computing high-dimensional normal and Student-t probabilities. Statistics and Computing, 31(1). doi:10.1007/s11222-020-09978-y
SponsorsThe authors thank Prof. Georgiy Stenchikov at KAUST for providing the WRF data and the two anonymous reviewers for valuable comments that improved this manuscript.
JournalStatistics and Computing