Max-and-Smooth: a two-step approach for approximate Bayesian inference in latent Gaussian models

Abstract

With modern high-dimensional data, complex statistical models are necessary,requiring computationally feasible inference schemes. We introduceMax-and-Smooth, an approximate Bayesian inference scheme for a flexible classof latent Gaussian models (LGMs) where one or more of the likelihood parametersare modeled by latent additive Gaussian processes. Max-and-Smooth consists oftwo-steps. In the first step (Max), the likelihood function is approximated bya Gaussian density with mean and covariance equal to either (a) the maximumlikelihood estimate and the inverse observed information, respectively, or (b)the mean and covariance of the normalized likelihood function. In the secondstep (Smooth), the latent parameters and hyperparameters are inferred andsmoothed with the approximated likelihood function. The proposed method ensuresthat the uncertainty from the first step is correctly propagated to the secondstep. Since the approximated likelihood function is Gaussian, the approximateposterior density of the latent parameters of the LGM (conditional on thehyperparameters) is also Gaussian, thus facilitating efficient posteriorinference in high dimensions. Furthermore, the approximate marginal posteriordistribution of the hyperparameters is tractable, and as a result, thehyperparameters can be sampled independently of the latent parameters. In thecase of a large number of independent data replicates, sparse precisionmatrices, and high-dimensional latent vectors, the speedup is substantial incomparison to an MCMC scheme that infers the posterior density from the exactlikelihood function. The proposed inference scheme is demonstrated on onespatially referenced real dataset and on simulated data mimicking spatial,temporal, and spatio-temporal inference problems. Our results show thatMax-and-Smooth is accurate and fast.