Now showing items 1-20 of 590

• #### Spatio-Temporal Cross-Covariance Functions under the Lagrangian Framework with Multiple Advections

(Journal of the American Statistical Association, Informa UK Limited, 2022-05-17) [Article]
When analyzing the spatio-temporal dependence in most environmental and earth sciences variables such as pollutant concentrations at different levels of the atmosphere, a special property is observed: the covariances and cross-covariances are stronger in certain directions. This property is attributed to the presence of natural forces, such as wind, which cause the transport and dispersion of these variables. This spatio-temporal dynamics prompted the use of the Lagrangian reference frame alongside any Gaussian spatio-temporal geostatistical model. Under this modeling framework, a whole new class was birthed and was known as the class of spatio-temporal covariance functions under the Lagrangian framework, with several developments already established in the univariate setting, in both stationary and nonstationary formulations, but less so in the multivariate case. Despite the many advances in this modeling approach, efforts have yet to be directed to probing the case for the use of multiple advections, especially when several variables are involved. Accounting for multiple advections would make the Lagrangian framework a more viable approach in modeling realistic multivariate transport scenarios. In this work, we establish a class of Lagrangian spatio-temporal cross-covariance functions with multiple advections, study its properties, and demonstrate its use on a bivariate pollutant dataset of particulate matter in Saudi Arabia.
• #### Student-t Stochastic Volatility Model With Composite Likelihood EM-Algorithm

(Journal of Time Series Analysis, Wiley, 2022-05-06) [Article]
A new robust stochastic volatility (SV) model having Student-t marginals is proposed. Our process is defined through a linear normal regression model driven by a latent gamma process that controls temporal dependence. This gamma process is strategically chosen to enable us to find an explicit expression for the pairwise joint density function of the Student-t response process. With this at hand, we propose a composite likelihood (CL) based inference for our model, which can be straightforwardly implemented with a low computational cost. This is a remarkable feature of our Student-t process over existing SV models in the literature that involve computationally heavy algorithms for estimating parameters. Aiming at a precise estimation of the parameters related to the latent process, we propose a CL Expectation-Maximization algorithm and discuss a bootstrap approach to obtain standard errors. The finite-sample performance of our composite likelihood methods is assessed through Monte Carlo simulations. The methodology is motivated by an empirical application in the financial market. We analyze the relationship, across multiple time periods, between various US sector Exchange-Traded Funds returns and individual companies' stock price returns based on our novel Student-t model. This relationship is further utilized in selecting optimal financial portfolios. Generalizations of the Student-t SV model are also proposed.
• #### Forecasting FSW Material’s Behavior using an Artificial Intelligence-Driven Approach

(IEEE, 2022-05-02) [Conference Paper]
A flexible data-driven methodology was developed to forecast the mechanical behavior of an aluminum alloy, namely Al6061-T6, in the case of friction stir welding. Specifically, Gated recurrent unit (GRU), a deep learning model, was investigated in this study. This is the first time the GRU model has been used to forecast the stress-strain curve of a material. The major features of the GRU consist in its ability to model time-series data and rely only on historical and actual data from the investigated material. The performance of the GRU model has been demonstrated based on actual data collected by conducting uniaxial tensile testing on the base material, and friction stirred welded, both tested at a deformation speed of 10 −3 s −1 . Forecasting tensile tests results showed promising and accurate results of the GRU-driven forecasting.
• #### Forecasting of Bicycle and Pedestrian Traffic Using Flexible and Efficient Hybrid Deep Learning Approach

(Applied Sciences, MDPI AG, 2022-04-28) [Article]
Recently, increasing interest in managing pedestrian and bicycle flows has been demonstrated by cities and transportation professionals aiming to reach community goals related to health, safety, and the environment. Precise forecasting of pedestrian and bicycle traffic flow is crucial for identifying the potential use of bicycle and pedestrian infrastructure and improving bicyclists’ safety and comfort. Advances in sensory technology enable collecting massive traffic flow data, including road traffic, bicycle, and pedestrian traffic flow. This paper introduces a novel deep hybrid learning model with a fully guided-attention mechanism to improve bicycles and pedestrians’ traffic flow forecasting. Notably, the proposed approach extends the modeling capability of the Variational Autoencoder (VAE) by merging a long short-term memory (LSTM) model with the VAE’s decoder and using a self-attention mechanism at multi-stage of the VAE model (i.e., decoder and before data resampling). Specifically, LSTM improves the VAE decoder’s capacity in learning temporal dependencies, and the guided-attention units enable selecting relevant features based on the self-attention mechanism. This proposed deep hybrid learning model with a multi-stage guided-attention mechanism is called GAHD-VAE. Proposed methods were validated with traffic measurements from six publicly available pedestrian and bicycle traffic flow datasets. The proposed method provides promising forecasting results but requires no assumptions that the data are drawn from a given distribution. Results revealed that the GAHD-VAE methodology can efficiently enhance the traffic forecasting accuracy and achieved better performance than the deep learning methods VAE, LSTM, gated recurrent units (GRUs), bidirectional LSTM, bidirectional GRU, convolutional neural network (CNN), and convolutional LSTM (ConvLSTM), and four shallow methods, linear regression, lasso regression, ridge regression, and support vector regression.
• #### Topological Data Analysis for Multivariate Time Series Data

(arXiv, 2022-04-28) [Preprint]
Over the last two decades, topological data analysis (TDA) has emerged as a very powerful data analytic approach which can deal with various data modalities of varying complexities. One of the most commonly used tools in TDA is persistent homology (PH) which can extract topological properties from data at various scales. Our aim in this article is to introduce TDA concepts to a statistical audience and provide an approach to analyze multivariate time series data. The application focus will be on multivariate brain signals and brain connectivity networks. Finally, the paper concludes with an overview of some open problems and potential application of TDA to modeling directionality in a brain network as well as the casting of TDA in the context of mixed effects models to capture variations in the topological properties of data collected from multiple subjects.
• #### Frequency-Specific Non-Linear Granger Causality in a Network of Brain Signals

(IEEE, 2022-04-27) [Conference Paper]
We propose a novel algorithm to extract frequency-band specific and non-linear Granger causality (Spectral NLGC) connections between components of a multivariate time series. The advantage of our model over traditionally used VAR based models, as demonstrated in simulations, is the ability to capture complex dependence structures in a network. In addition to the simulations, the proposed method uncovered non-linear dynamics in an epileptic seizure EEG data. Spectral NLGC gives new meaningful insights into frequency specific connectivity changes at the onset of epileptic seizure. Results of both simulated and brain signals confirm the viability of the proposed algorithm as a good tool for exploration of directed connectivity in a network.
• #### Machine learning and deep learning-driven methods for predicting ambient particulate matters levels: A case study

(Concurrency and Computation: Practice and Experience, Wiley, 2022-04-26) [Article]
Dust, or particulate matter (PM2.5), is among the most harmful pollutants negatively affecting human health. Predicting indoor PM2.5 concentrations is essential to achieve acceptable indoor air quality. This study aims to investigate data-driven models to accurately predict PM 2.5 pollution. Notably, a comparative study has been conducted between twenty-one machine learning and deep learning models to predict PM2.5 levels. Specifically, we investigate the performance of machine learning and deep learning models to predict ambient PM2.5 concentrations based on other ambient pollutants, including SO 2 , NO 2 , O 3 , CO, and PM10. Here, we applied Bayesian optimization to optimally tune hyperparameters of the Gaussian process regression with different kernels and ensemble learning models (i.e., boosted trees and bagged trees) and investigated their prediction performance. Furthermore, to further enhance the forecasting performance of the investigated models, dynamic information has been incorporated by introducing lagged measurements in the construction of the considered models. Results show a significant improvement in the prediction performance when considering dynamic information from past data. Moreover, three methods, namely, random forest (RF), decision tree, and extreme gradient boosting, are applied to assess variables contribution and revealed that lagged PM2.5 data contribute significantly to the prediction performance and enables the construction of parsimonious models. Hourly concentration levels of ambient air pollution from the air quality monitoring network located in Seoul are employed to verify the prediction effectiveness of the studied models. Six measurements of effectiveness are used for assessing the prediction quality. Results showed that deep learning models are more efficient than the other investigated machine learning models (i.e., SVR, GPR, bagged and boosted trees, RF, and XGBoost). Also, the results showed that the bidirectional long short term memory (BiLSTM) and bidirectional gated recurrent units (BiGRU) networks produce higher performance than the investigated machine learning models (i.e., SVR, GPR, bagged and boosted trees, RF, and XGBoost) and deep learning models (i.e., LSTM, GRU, and convolutional neural network).
• #### Robust Two-Layer Partition Clustering of Sparse Multivariate Functional Data

(arXiv, 2022-04-26) [Preprint]
In this work, a novel elastic time distance for sparse multivariate functional data is proposed. This concept serves as the foundation for clustering functional data with various time measurements per subject. Subsequently, a robust two-layer partition clustering is introduced. With the proposed distance, our approach not only is applicable to both complete and imbalanced multivariate functional data but also is resistant to outliers and capable of detecting outliers that do not belong to any clusters. The classical distance-based clustering methods such as K-medoids and agglomerative hierarchical clustering are extended to the sparse multivariate functional case based on our proposed distance. Numerical experiments on the simulated data highlight that the performance of the proposed algorithm is superior to the performances of the existing model-based and extended distance-based methods. Using Northwest Pacific cyclone track data as an example, we demonstrate the effectiveness of the proposed approach. The code is available online for readers to apply our clustering method and replicate our analyses.
• #### Sparse Functional Boxplots for Multivariate Curves

(Journal of Computational and Graphical Statistics, Informa UK Limited, 2022-04-19) [Article]
This paper introduces the sparse functional boxplot and the intensity sparse functional boxplot as practical exploratory tools. Besides being available for complete functional data, they can be used in sparse univariate and multivariate functional data. The sparse functional boxplot, based on the functional boxplot, displays sparseness proportions within the 50% central region. The intensity sparse functional boxplot indicates the relative intensity of fitted sparse point patterns in the central region. The two-stage functional boxplot, which derives from the functional boxplot to detect outliers, is furthermore extended to its sparse form. We also contribute to sparse data fitting improvement and sparse multivariate functional data depth. In a simulation study, we evaluate the goodness of data fitting, several depth proposals for sparse multivariate functional data, and compare the results of outlier detection between the sparse functional boxplot and its two-stage version. The practical applications of the sparse functional boxplot and intensity sparse functional boxplot are illustrated with two public health datasets. Supplementary materials and codes are available for readers to apply our visualization tools and replicate the analysis
• #### Modeling and Simulation of Spatial Extremes Based on Max-Infinitely Divisible and Related Processes

(2022-04-17) [Dissertation]
Committee members: Genton, Marc G.; Jasra, Ajay; Cooley, Daniel
The statistical modeling of extreme natural hazards is becoming increasingly important due to climate change, whose effects have been increasingly visible throughout the last decades. It is thus crucial to understand the dependence structure of rare, high-impact events over space and time for realistic risk assessment. For spatial extremes, max-stable processes have played a central role in modeling block maxima. However, the spatial tail dependence strength is persistent across quantile levels in those models, which is often not realistic in practice. This lack of flexibility implies that max-stable processes cannot capture weakening dependence at increasingly extreme levels, resulting in a drastic overestimation of joint tail risk. To address this, we develop new dependence models in this thesis from the class of max-infinitely divisible (max-id) processes, which contain max-stable processes as a subclass and are flexible enough to capture different types of dependence structures. Furthermore, exact simulation algorithms for general max-id processes are typically not straightforward due to their complex formulations. Both simulation and inference can be computationally prohibitive in high dimensions. Fast and exact simulation algorithms to simulate max-id processes are provided, together with methods to implement our models in high dimensions based on the Vecchia approximation method. These proposed methodologies are illustrated through various environmental datasets, including air temperature data in South-Eastern Europe in an attempt to assess the effect of climate change on heatwave hazards, and sea surface temperature data for the entire Red Sea. In another application focused on assessing how the spatial extent of extreme precipitation has changed over time, we develop new time-varying $r$-Pareto processes, which are the counterparts of max-stable processes for high threshold exceedances.
• #### Efficient quantile tracking using an oracle

(Applied Intelligence, Springer Science and Business Media LLC, 2022-04-14) [Article]
Concept drift is a well-known issue that arises when working with data streams. In this paper, we present a procedure that allows a quantile tracking procedure to cope with concept drift. We suggest using expected quantile loss, a popular loss function in quantile regression, to monitor the quantile tracking error, which, in turn, is used to efficiently adapt to concept drift. The suggested procedures adapt efficiently to concept drift, and the tracking performance is close to theoretically optimal. The procedures were further applied to three real-life streaming data sets related to Twitter event detection, activity recognition, and stock trading. The results show that the procedures are efficient at adapting to concept drift, thereby documenting the real-world applicability of the procedures. We further used asymptotic theory from statistics to show the appealing theoretical property that, if the data stream distribution is stationary over time, the procedures converge to the true quantile.
• #### A new avenue for Bayesian inference with INLA

(arXiv, 2022-04-14) [Preprint]
Integrated Nested Laplace Approximations (INLA) has been a successful approximate Bayesian inference framework since its proposal by [38]. The increased computational efficiency and accuracy when compared with sampling based methods for Bayesian inference like MCMC methods, are some contributors to its success. Ongoing research in the INLA methodology and implementation thereof in the R package R-INLA, ensures continued relevance for practitioners and improved performance and applicability of INLA. The era of big data and some recent research developments, presents an opportunity to reformulate some aspects of the classic INLA formulation, to achieve even faster inference, improved numerical stability and scalability. The improvement is especially noticeable for data-rich models. We demonstrate the efficiency gains with various examples of data-rich models, like Cox’s proportional hazards model, an item-response theory model, a spatial model including prediction, and a 3-dimensional model for fMRI data.
• #### Parallelized integrated nested Laplace approximations for fast Bayesian inference

(arXiv, 2022-04-10) [Preprint]
There is a growing demand for performing larger-scale Bayesian inference tasks, arising from greater data availability and higher-dimensional model parameter spaces. In this work we present parallelization strategies for the methodology of integrated nested Laplace approximations (INLA), a popular framework for performing approximate Bayesian inference on the class of Latent Gaussian models. Our approach makes use of nested OpenMP parallelism, a parallel line search procedure using robust regression in INLA's optimization phase and the state-of-the-art sparse linear solver PARDISO. We leverage mutually independent function evaluations in the algorithm as well as advanced sparse linear algebra techniques. This way we can flexibly utilize the power of today's multi-core architectures. We demonstrate the performance of our new parallelization scheme on a number of different real-world applications. The introduction of parallelism leads to speedups of a factor 10 and more for all larger models. Our work is already integrated in the current version of the open-source R-INLA package, making its improved performance conveniently available to all users.
• #### Flexible Modeling of Non-Stationary Extremal Dependence Using Spatially-Fused LASSO and Ridge Penalties

(2022-04-05) [Thesis]
Committee members: Sun, Ying; Gomes, Diogo A.
Statistical modeling of a nonstationary spatial extremal dependence structure is a challenging problem. In practice, parametric max-stable processes are commonly used for modeling spatially-indexed block maxima data, where the stationarity assumption is often made to make inference easier. However, this assumption is unreliable for data observed over a large or complex domain. In this work, we develop a computationally-efficient method to estimate nonstationary extremal dependence using max-stable processes, which builds upon and extends an approach recently proposed in the classical geostatistical literature. More precisely, we divide the spatial domain into a fine grid of subregions, each having its own set of dependence-related parameters, and then impose LASSO ($L_1$) or Ridge ($L_2$) penalties to obtain spatially-smooth estimates. We then also subsequently merge the subregions sequentially together with a new algorithm to enhance the model's performance. Here we focus on the popular Brown-Resnick process, although extensions to other classes of max-stable processes are also possible. We discuss practical strategies for adequately defining the subregions and merging them back together. To make our method suitable for high-dimensional datasets, we exploit a pairwise likelihood approach and discuss the choice of pairs to achieve reasonable computational and statistical efficiency. We apply our proposed method to a dataset of annual maximum temperature in Nepal and show that our approach fits reasonably and realistically captures the complex non-stationarity in the extremal dependence.
• #### A hybrid deep learning method with attention for COVID-19 spread forecasting

(IOP Publishing, 2022-04) [Book Chapter]
This chapter introduces a hybrid deep learning model for COVID-19 spread forecasting. Specifically, the proposed approach combines the desirable characteristics of bidirectional long short-term memory (BiLSTM), convolutional neural networks (CNN), and an attention mechanism. Importantly, this combination, called BiLSTM-A-CNN, is intended to amalgamate the ability of LSTMs to model time dependencies, the capability of the attention mechanism to highlight relevant features, and the noted ability of CNNs to extract features from complex data. The use of the BiLSTM-A-CNN model is expected to improve the forecasting accuracy of future COVID-19 trends.
• #### Bayesian Modeling of Sub-Asymptotic Spatial Extremes

(2022-04) [Dissertation]
Committee members: Genton, Marc G.; Jasra, Ajay; Naveau, Philippe
In many environmental and climate applications, extreme data are spatial by nature, and hence statistics of spatial extremes is currently an important and active area of research dedicated to developing innovative and flexible statistical models that determine the location, intensity, and magnitude of extreme events. In particular, the development of flexible sub-asymptotic models is in trend due to their flexibility in modeling spatial high threshold exceedances in larger spatial dimensions and with little or no effects on the choice of threshold, which is complicated with classical extreme value processes, such as Pareto processes. In this thesis, we develop new flexible sub-asymptotic extreme value models for modeling spatial and spatio-temporal extremes that are combined with carefully designed gradient-based Markov chain Monte Carlo (MCMC) sampling schemes and that can be exploited to address important scientific questions related to risk assessment in a wide range of environmental applications. The methodological developments are centered around two distinct themes, namely (i) sub-asymptotic Bayesian models for extremes; and (ii) flexible marked point process models with sub-asymptotic marks. In the first part, we develop several types of new flexible models for light-tailed and heavy-tailed data, which extend a hierarchical representation of the classical generalized Pareto (GP) limit for threshold exceedances. Spatial dependence is modeled through latent processes. We study the theoretical properties of our new methodology and demonstrate it by simulation and applications to precipitation extremes in both Germany and Spain. In the second part, we construct new marked point process models, where interest mostly lies in the extremes of the mark distribution. Our proposed joint models exploit intrinsic CAR priors to capture the spatial effects in landslide counts and sizes, while the mark distribution is assumed to take various parametric forms. We demonstrate that having a sub-asymptotic distribution for landslide sizes provides extra flexibility to accurately capture small to large and especially extreme, devastating landslides.
• #### An Extended Simplified Laplace strategy for Approximate Bayesian inference of Latent Gaussian Models using R-INLA

(arXiv, 2022-03-27) [Preprint]
Various computational challenges arise when applying Bayesian inference approaches to complex hierarchical models. Sampling-based inference methods, such as Markov Chain Monte Carlo strategies, are renowned for providing accurate results but with high computational costs and slow or questionable convergence. On the contrary, approximate methods like the Integrated Nested Laplace Approximation (INLA) construct a deterministic approximation to the univariate posteriors through nested Laplace Approximations. This method enables fast inference performance in Latent Gaussian Models, which encode a large class of hierarchical models. R-INLA software mainly consists of three strategies to compute all the required posterior approximations depending on the accuracy requirements. The Simplified Laplace approximation (SLA) is the most attractive because of its speed performance since it is based on a Taylor expansion up to order three of a full Laplace Approximation. Here we enhance the methodology by simplifying the computations necessary for the skewness and modal configuration. Then we propose an expansion up to order four and use the Extended Skew Normal distribution as a new parametric fit. The resulting approximations to the marginal posterior densities are more accurate than those calculated with the SLA, with essentially no additional cost.
• #### Partially interpretable neural networks for high-dimensional extreme quantile regression: With application to wildfires within the Mediterranean Basin

(Copernicus GmbH, 2022-03-27) [Presentation]
Quantile regression is a particularly powerful tool for modelling environmental data which exhibits spatio-temporal non-stationarity in its marginal behaviour. If our interest lies in quantifying risk associated with particularly extreme or rare weather events, we may want to estimate conditional quantiles that are outside the range of observable data; in these cases, it is practical to describe the data using some parametric extreme value model with its parameters represented as functions of predictor variables. Classical approaches for parametric extreme quantile regression use linear or additive relationships, and such approaches suffer in either their predictive capabilities or computational efficiency in high-dimensions. Neural networks can capture complex non-linear relationships between variables and scale well to high-dimensional predictor sets. Whilst they have been successfully applied in the context of fitting extreme value models, statisticians may choose to forego the use of neural networks as a result of their “black box" nature; although they facilitate highly accurate prediction, it is difficult to do statistical inference with neural networks as their outputs cannot readily be interpreted. Inspired by the recent focus in machine learning literature on “explainable AI”, we propose a framework for performing extreme quantile regression using partially interpretable neural networks. Distribution parameters are represented as functions of predictors with three main components; a linear function, an additive function and a neural network that are applied separately to complementary subsets of predictors. The output from the linear and additive components is interpreted, whilst the neural network component contributes to the high prediction accuracy of our method. We use our approach to estimate extreme quantiles and occurrence probabilities for wildfires occurring within a large spatial domain that encompasses the entirety of the Mediterranean Basin.
• #### Realistic and Fast Modeling of Spatial Extremes over Large Geographical Domains

(Copernicus GmbH, 2022-03-27) [Presentation]
Various natural phenomena, such as precipitation, generally exhibit spatial extremal dependence at short distances only, while the dependence usually fades away as the distance between sites increases arbitrarily. However, the available models proposed in the literature for spatial extremes, which are based on max-stable or Pareto processes or comparatively less computationally demanding "sub-asymptotic" models based on Gaussian location and/or scale mixtures, generally assume that spatial extremal dependence persists across the entire spatial domain. This is a clear limitation when modeling extremes over large geographical domains, but surprisingly, it has been mostly overlooked in the literature. In this paper, we develop a more realistic Bayesian framework based on a novel Gaussian scale mixture model, where the Gaussian process component is defined by a stochastic partial differential equation that yields a sparse precision matrix, and the random scale component is modeled as a low-rank Pareto-tailed or Weibull-tailed spatial process determined by compactly supported basis functions. We show that our proposed model is approximately tail-stationary despite its non-stationary construction in terms of basis functions, and we demonstrate that it can capture a wide range of extremal dependence structures as a function of distance. Furthermore, the inherently sparse structure of our spatial model allows fast Bayesian computations, even in high spatial dimensions, based on a customized Markov chain Monte Carlo algorithm, which prioritize calibration in the tail. In our application, we fit our model to analyze heavy monsoon rainfall data in Bangladesh. Our study indicates that the proposed model outperforms some natural alternatives, and that the model fits precipitation extremes satisfactorily well. Finally, we use the fitted model to draw inferences on long-term return levels for marginal precipitation at each site, and for spatial aggregates.
• #### Scalable Computation of Predictive Probabilities in Probit Models with Gaussian Process Priors

(JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, Informa UK Limited, 2022-03-24) [Article]
Predictive models for binary data are fundamental in various fields, and the growing complexity of modern applications has motivated several flexible specifications for modeling the relationship between the observed predictors and the binary responses. A widely-implemented solution is to express the probability parameter via a probit mapping of a Gaussian process indexed by predictors. However, unlike for continuous settings, there is a lack of closed-form results for predictive distributions in binary models with Gaussian process priors. Markov chain Monte Carlo methods and approximation strategies provide common solutions to this problem, but state-of-the-art algorithms are either computationally intractable or inaccurate in moderate-to-high dimensions. In this article, we aim to cover this gap by deriving closed-form expressions for the predictive probabilities in probit Gaussian processes that rely either on cumulative distribution functions of multivariate Gaussians or on functionals of multivariate truncated normals. To evaluate these quantities we develop novel scalable solutions based on tile-low-rank Monte Carlo methods for computing multivariate Gaussian probabilities, and on mean-field variational approximations of multivariate truncated normals. Closed-form expressions for the marginal likelihood and for the posterior distribution of the Gaussian process are also discussed. As shown in simulated and real-world empirical studies, the proposed methods scale to dimensions where state-of-the-art solutions are impractical.