### Recent Submissions

• #### Lower Bounds and Optimal Algorithms for Personalized Federated Learning

(arXiv, 2020-10-05) [Preprint]
In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richt\'arik (2020) which was shown to give an alternative explanation to the workings of local {\tt SGD} methods. Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity. Our second contribution is the design of several optimal methods matching these lower bounds in almost all regimes. These are the first provably optimal methods for personalized federated learning. Our optimal methods include an accelerated variant of {\tt FedProx}, and an accelerated variance-reduced version of {\tt FedAvg}/Local {\tt SGD}. We demonstrate the practical superiority of our methods through extensive numerical experiments.
• #### Forecasting emergency department overcrowding: A deep learning framework

(Chaos, Solitons and Fractals, Elsevier BV, 2020-09-21) [Article]
As the demand for medical cares has considerably expanded, the issue of managing patient flow in hospitals and especially in emergency departments (EDs) is certainly a key issue to be carefully mitigated. This can lead to overcrowding and the degradation of the quality of the provided medical services. Thus, the accurate modeling and forecasting of ED visits are critical for efficiently managing the overcrowding problems and enable appropriate optimization of the available resources. This paper proposed an effective method to forecast daily and hourly visits at an ED using Variational AutoEncoder (VAE) algorithm. Indeed, the VAE model as a deep learning-based model has gained special attention in features extraction and modeling due to its distribution-free assumptions and superior nonlinear approximation. Two types of forecasting were conducted: one- and multi-step-ahead forecasting. To the best of our knowledge, this is the first time that the VAE is investigated to improve forecasting of patient arrivals time-series data. Data sets from the pediatric emergency department at Lille regional hospital center, France, are employed to evaluate the forecasting performance of the introduced method. The VAE model was evaluated and compared with seven methods namely Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Convolutional LSTM Network (ConvLSTM), restricted Boltzmann machine (RBM), Gated recurrent units (GRUs), and convolutional neural network (CNN). The results clearly show the promising performance of these deep learning models in forecasting ED visits and emphasize the better performance of the VAE in comparison to the other models.
• #### Rejoinder to the discussion on A high-resolution bilevel skew-t stochastic generator for assessing Saudi Arabia's wind energy resources

(Environmetrics, Wiley, 2020-09-21) [Article]
This is the rejoinder of the discussion article: env-19-0145, DOI: 10.1002/env.2628.
• #### Spatiotemporal probabilistic wind vector forecasting over Saudi Arabia

(Annals of Applied Statistics, Institute of Mathematical Statistics, 2020-09-18) [Article]
Saudi Arabia has recently begun promoting renewable energy as a potential alternative to fossil fuels for domestic power generation. In order to efficiently connect wind energy to the existing power grids, reliable wind forecasts and an accurate way of quantifying the uncertainties of these forecasts are required. Motivated by a data set of hourly wind speeds from 28 stations in Saudi Arabia, we build spatiotemporal models for short-term probabilistic forecasts of wind vectors. Traditionally, wind speed and wind direction have been considered independently, without taking dependencies into account. However, in many situations, for example, energy management, it is essential to have information on the bivariate nature of the wind. We compare a coregionalization model for the wind vector with a univariate spatiotemporal model for the transformed wind speed in terms of sharpness and calibration. In both cases the linear predictor is a function of covariates, a smooth function to capture the daily seasonality in the wind and a latent Gaussian field to model the spatial and temporal dependencies. Substantial improvements in reliability are observed when modelling the full bivariate structure instead of only considering speed. Furthermore, the bivariate model has the advantage of also producing forecasts for the wind direction. A Bayesian framework is used to obtain forecasts that are accurate and reliable, even at stations without observations, with relatively low computational cost. Simulated highresolution data from a computer model are used to validate spatiotemporal forecasts. A detailed analysis on this case study shows how increasing the number of locations can improve the forecast performance.
• #### Flexible Modeling of Variable Asymmetries in Cross-Covariance Functions for Multivariate Random Fields

(Journal of Agricultural, Biological and Environmental Statistics, Springer Science and Business Media LLC, 2020-09-10) [Article]
The geostatistical analysis of multivariate spatial data for inference as well as joint predictions (co-kriging) ordinarily relies on modeling of the marginal and cross-covariance functions. While the former quantifies the spatial dependence within variables, the latter quantifies the spatial dependence across distinct variables. The marginal covariance functions are always symmetric; however, the cross-covariance functions often exhibit asymmetries in the real data. Asymmetric cross-covariance implies change in the value of cross covariance for interchanged locations on fixed order of variables. Such change of cross-covariance values is often caused due to the spatial delay in effect of the response of one variable on another variable. These spatial delays are common in environmental processes, especially when dynamic phenomena such as prevailing wind and ocean currents are involved. Here, we propose a novel approach to introduce flexible asymmetries in the cross-covariances of stationary multivariate covariance functions. The proposed approach involves modeling the phase component of the constrained cross-spectral features to allow for asymmetric cross-covariances. We show the capability of our proposed model to recover the cross-dependence structure and improve spatial predictions against traditionally used models through multiple simulation studies. Additionally, we illustrate our approach on a real trivariate dataset of particulate matter concentration (PM2.5), wind speed and relative humidity. The real data example shows that our approach outperforms the traditionally used models, in terms of model fit and spatial predictions.
• #### Efficient Importance Sampling for the Left Tail of Positive Gaussian Quadratic Forms

(arXiv, 2020-09-06) [Preprint]
Estimating the left tail of quadratic forms in Gaussian random vectors is of major practical importance in many applications. In this letter, we propose an efficient importance sampling estimator that is endowed with the bounded relative error property. This property significantly reduces the number of simulation runs required by the proposed estimator compared to naive Monte Carlo (MC), especially when the probability of interest is very small. Selected simulation results are presented to illustrate the efficiency of our estimator compared to naive MC as well as some of the well-known approximations.
• #### Scalable computation of predictive probabilities in probit models with Gaussian process priors

(arXiv, 2020-09-03) [Preprint]
Predictive models for binary data are fundamental in various fields, ranging from spatial statistics to machine learning. In such settings, the growing complexity of the phenomena to be analyzed has motivated a variety of flexible specifications that avoid strong parametric assumptions when defining the relationship between the observed predictors and the binary response data. A widely-implemented solution within this class expresses the probability parameter via a probit mapping of a Gaussian process indexed by the predictors. However, unlike for continuous settings with Gaussian responses, there is a lack of closed-form results for predictive distributions in binary models with Gaussian process priors. Markov chain Monte Carlo methods and approximate solutions provide common options to address this issue, but state-of-the-art strategies are either computationally intractable or lead to low-quality approximations in moderate-to-high dimensions. In this article, we aim to cover this gap by deriving closed-form expressions for the predictive probabilities in probit Gaussian processes that rely either on cumulative distribution functions of multivariate Gaussians or on functionals of multivariate truncated normals. To evaluate such quantities we develop novel scalable solutions based on tile-low-rank Monte Carlo methods for computing multivariate Gaussian probabilities and on accurate variational approximations of multivariate truncated normal densities. Closed-form expressions for the marginal likelihood and for the conditional distribution of the Gaussian process given the binary responses are also discussed. As illustrated in simulations and in a real-world environmental application, the proposed methods can scale to dimensions where state-of-the-art solutions are impractical.
• #### Are You All Normal? It Depends!

(arXiv, 2020-08-25) [Preprint]
The assumption of normality has underlain much of the development of statistics, including spatial statistics, and many tests have been proposed. In this work, we focus on the multivariate setting and we first provide a synopsis of the recent advances in multivariate normality tests for i.i.d. data, with emphasis on the skewness and kurtosis approaches. We show through simulation studies that some of these tests cannot be used directly for testing normality of spatial data, since the multivariate sample skewness and kurtosis measures, such as the Mardia's measures, deviate from their theoretical values under Gaussianity due to dependence, and some related tests exhibit inflated type I error, especially when the spatial dependence gets stronger. We review briefly the few existing tests under dependence (time or space), and then propose a new multivariate normality test for spatial data by accounting for the spatial dependence of the observations in the test statistic. The new test aggregates univariate Jarque-Bera (JB) statistics, which combine skewness and kurtosis measures, for individual variables. The asymptotic variances of sample skewness and kurtosis for standardized observations are derived given the dependence structure of the spatial data. Consistent estimators of the asymptotic variances are then constructed for finite samples. The test statistic is easy to compute, without any smoothing involved, and it is asymptotically $\chi^2_{2p}$ under normality, where $p$ is the number of variables. The new test has a good control of the type I error and a high empirical power, especially for large sample sizes.
• #### Probabilistic projection of the sex ratio at birth and missing female births by State and Union Territory in India.

(PloS one, Public Library of Science (PLoS), 2020-08-20) [Article]
The sex ratio at birth (SRB) in India has been reported to be imbalanced since the 1970s. Previous studies have shown there is a great variation in the SRB between geographic locations across India till 2016. Considering the enormous population and regional heterogeneity of India, producing probabilistic SRB projections at the state level is crucial for policy planning and population projection. In this paper, we implement a Bayesian hierarchical time series model to project the SRB across India by state. We generate SRB probabilistic projections from 2017 to 2030 for 29 States and Union Territories (UTs) in India, and present results for 21 States/UTs with data available from the Sample Registration System. Our analysis takes into account two state-specific factors that contribute to sex-selective abortion in India, resulting in sex imbalances at birth: the intensity of son preference and fertility squeeze. We project that the highest deficits in female births will occur in Uttar Pradesh, with a cumulative number of missing female births of 2.0 (95% credible interval [1.9; 2.2]) million from 2017 to 2030. The total female birth deficits during 2017-2030 for the whole of India is projected to be 6.8 [6.6; 7.0] million.
• #### Regularized matrix data clustering and its application to image analysis

(Biometrics, Wiley, 2020-08-16) [Article]
We propose a novel regularized mixture model for clustering matrix-valued data. The proposed method assumes a separable covariance structure for each cluster and imposes a sparsity structure (e.g., low rankness, spatial sparsity) for the mean signal of each cluster. We formulate the problem as a finite mixture model of matrix-normal distributions with regularization terms, and then develop an EM-type of algorithm for efficient computation. In theory, we show that the proposed estimators are strongly consistent for various choices of penalty functions. Simulation and two applications on brain signal studies confirm the excellent performance of the proposed method including a better prediction accuracy than the competitors and the scientific interpretability of the solution.
• #### Space-time landslide predictive modelling

(Earth-Science Reviews, Elsevier BV, 2020-08-11) [Article]
Landslides are nearly ubiquitous phenomena and pose severe threats to people, properties, and the environment in many areas. Investigators have for long attempted to estimate landslide hazard in an effort to determine where, when (or how frequently), and how large (or how destructive) landslides are expected to be in an area. This information may prove useful to design landslide mitigation strategies, and to reduce landslide risk and societal and economic losses. In the geomorphology literature, most of the attempts at predicting the occurrence of populations of landslides by adopting statistical approaches are based on the empirical observation that landslides occur as a result of multiple, interacting, conditioning and triggering factors. Based on this observation, and under the assumption that at the spatial and temporal scales of our investigation individual landslides are discrete “point” events in the landscape, we propose a Bayesian modelling framework for the prediction of the spatio-temporal occurrence of landslides of the slide type caused by weather triggers. We build our modelling effort on a Log-Gaussian Cox Process (LGCP) by assuming that individual landslides in an area are the result of a point process described by an unknown intensity function. The modelling framework has two stochastic components: (i) a Poisson component, which models the observed (random) landslide count in each terrain subdivision for a given landslide “intensity”, i.e., the expected number of landslides per terrain subdivision (which may be transformed into a corresponding landslide “susceptibility”); and (ii) a Gaussian component, used to account for the spatial distribution of the local environmental conditions that influence landslide occurrence, and for the spatio-temporal distribution of “unobserved” latent environmental controls on landslide occurrence. We tested our prediction framework in the Collazzone area, Umbria, Central Italy, for which a detailed multi-temporal landslide inventory covering the period from before 1941 to 2014 is available together with lithological and bedding data. We subdivided the 79 km2 area into 889 slope units (SUs). In each SU, we computed the mean and standard deviation of 16 morphometric covariates derived from a 10 m × 10 m digital elevation model. For 13 lithological and bedding attitude covariates obtained from a 1:10,000 scale geological map, we computed the proportion of each thematic class intersecting the given SU. We further counted how many of the 3,379 landslides in the multi-temporal inventory affect each SU and grouped them into six periods. We used this complex space-time information to prepare five models of increasing complexity. Our “baseline” model (Mod1) carries the spatial information only through the covariates mentioned above. It does not include any additional information about the spatial and temporal structure of the data, and it is therefore equivalent to the predominantly used landslide susceptibility model in the literature. The second model (Mod2) is analogous, but it allows for time-interval-specific regression constants. Our next two models are more complex. In particular, our third model (Mod3) also accounts for latent spatial dependencies among neighboring SUs. These are inferred for each of the six time intervals, to explain variations in the landslide intensity and susceptibility not explained by the thematic covariates. By contrast, our fourth model (Mod4) accounts for the latent temporal dependence, separately for each SU, disregarding neighboring influences. Ultimately, our most complex model (Mod5) contextually features all these relations. It contains the information carried by morphometric and thematic covariates, six time-interval-specific regression constants, and it also accounts for the latent temporal effects between consecutive slope instabilities at specific SUs as well as the latent spatial effects between adjacent SUs. We also show that the intensity is strongly related to the aggregated landslide area per SU. Because of this, our most complex model largely fulfills the definition of landslide hazard commonly accepted in the literature, at least for this study area. We quantified the spatial predictive performance of each of the five models using a 10-fold cross-validation procedure, and the temporal predictive performance using a leave-one-out cross-validation procedure. We found that Mod5 performed better than the others. We then used it to test a novel strategy to classify the model results in terms of both landslide intensity and susceptibility, which provides more information than traditional susceptibility zonations for land planning and management—hereafter we use the term “traditional” simply to refer to the majority of modelling procedures in the literature. We discuss the advantages and limitations of the new modelling framework, and its potential application in other areas, making specific and general hazard and geomorphological considerations. We also give a perspective on possible developments in landslide prediction modelling and zoning. We expect our novel approach to the spatio-temporal prediction of landslides to enhance the currently limited ability to evaluate landslide hazard and its temporal and spatial variations. We further expect it to lead to better projections of future landslides, and to improve our collective understanding of the evolution of complex landscapes dominated by mass-wasting processes under multiple geophysical and weather triggers.
• #### Malicious attacks detection in crowded areas using deep learning-based approach

(IEEE Instrumentation and Measurement Magazine, Institute of Electrical and Electronics Engineers Inc., 2020-08-01) [Article]
With the increasing need to ensure people's safety in crowded areas, the development of a systematic anomaly detection mechanism is becoming indispensable. Here are a few examples of recent malicious attacks targeting crowded areas in big cities: in 2016, a truck driver attacked and killed 84 persons walking in the promenade in Nice, France; and on 19 December, 2016, a truck was deliberately driven into the Christmas market, in Berlin, Germany, killing 12 people and injuring 56 others. These attacks demonstrate the need for efficient monitoring systems to avoid such devastating attacks. To do so, early detection and prevention abilities are vital. Detecting and localizing abnormal events in crowded scenes is important and has significant implications in video surveillance applications. Video surveillance can be challenging, as abnormal events can be unpredictable and changing, based on the context. Accurately detecting and localizing anomalies in videos is a powerful tool that can help to improve security and understand the behavior of anomalies. In this paper, we present an automated visionbased monitoring scheme specifically designed for atypical event-detection and localization in crowded areas.
• #### Estimating the mean and variance from the five-number summary of a log-normal distribution

(Statistics and its Interface, International Press of Boston, 2020-07-31) [Article]
In the past several decades, meta-analysis has been widely used to pool multiple studies for evidence-based practice. To conduct a meta-analysis, the mean and variance from each study are often required; whereas in certain studies, the five-number summary may instead be reported that consists of the median, the first and third quartiles, and/or the minimum and maximum values. To transform the fivenumber summary back to the mean and variance, several popular methods have emerged in the literature. However, we note that most existing methods are developed under the normality assumption; and when this assumption is violated, these methods may not be able to provide a reliable transformation. In this paper, we propose to estimate the mean and variance from the five-number summary of a log-normal distribution. Specifically, we first make the log-transformation of the reported quantiles. With the existing mean estimators and newly proposed variance estimators under the normality assumption, we construct the estimators of the log-scale mean and variance. Finally, we transform them back to the original scale for the final estimators. We also propose a biascorrected method to further improve the estimation of the mean and variance. Simulation studies demonstrate that our new estimators have smaller biases and smaller relative risks in most settings. A real data example is used to illustrate the practical usefulness of our new estimators.
• #### Clustering Brain Signals: A Robust Approach Using Functional Data Ranking

(arXiv, 2020-07-28) [Preprint]
In this paper, we analyze electroencephalograms (EEG) which are recordings of brain electrical activity. We develop new clustering methods for identifying synchronized brain regions, where the EEGs show similar oscillations or waveforms according to their spectral densities. We treat the estimated spectral densities from many epochs or trials as functional data and develop clustering algorithms based on functional data ranking. The two proposed clustering algorithms use different dissimilarity measures: distance of the functional medians and the area of the central region. The performance of the proposed algorithms is examined by simulation studies. We show that, when contaminations are present, the proposed methods for clustering spectral densities are more robust than the mean-based methods. The developed methods are applied to two stages of resting state EEG data from a male college student, corresponding to early exploration of functional connectivity in the human brain.
• #### DeepKriging: Spatially Dependent Deep Neural Networks for Spatial Prediction

(arXiv, 2020-07-23) [Preprint]
In spatial statistics, a common objective is to predict the values of a spatial process at unobserved locations by exploiting spatial dependence. In geostatistics, Kriging provides the best linear unbiased predictor using covariance functions and is often associated with Gaussian processes. However, when considering non-linear prediction for non-Gaussian and categorical data, the Kriging prediction is not necessarily optimal, and the associated variance is often overly optimistic. We propose to use deep neural networks (DNNs) for spatial prediction. Although DNNs are widely used for general classification and prediction, they have not been studied thoroughly for data with spatial dependence. In this work, we propose a novel neural network structure for spatial prediction by adding an embedding layer of spatial coordinates with basis functions. We show in theory that the proposed DeepKriging method has multiple advantages over Kriging and classical DNNs only with spatial coordinates as features. We also provide density prediction for uncertainty quantification without any distributional assumption and apply the method to PM$_{2.5}$ concentrations across the continental United States.
• #### Assessing the risk of disruption of wind turbine operations in Saudi Arabia using Bayesian spatial extremes

(Extremes, Springer Science and Business Media LLC, 2020-07-22) [Article]
Saudi Arabia has been seeking to reduce its dependence on oil by diversifying its energy portfolio, including the largely underused energy potential from wind. However, extreme winds can possibly disrupt the wind turbine operations, thus preventing the stable and continuous production of wind energy. In this study, we assess the risk of disruptions of wind turbine operations, based on return levels with a hierarchical spatial extreme modeling approach for wind speeds in Saudi Arabia. Using a unique Weather Research and Forecasting dataset, we provide the first high-resolution risk assessment of wind extremes under spatial non-stationarity over the country. We account for the spatial dependence with a multivariate intrinsic autoregressive prior at the latent Gaussian process level. The computational efficiency is greatly improved by parallel computing on subregions from spatial clustering, and the maps are smoothed by fitting the model to cluster neighbors. Under the Bayesian hierarchical framework, we measure the uncertainty of return levels from the posterior Markov chain Monto Carlo samples, and produce probability maps of return levels exceeding the cut-out wind speed of wind turbines within their lifetime. The probability maps show that locations in the South of Saudi Arabia and near the Red Sea and the Persian Gulf are at very high risk of disruption of wind turbine operations.
• #### Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study

(Chaos, Solitons and Fractals, Elsevier BV, 2020-07-15) [Article]
The novel coronavirus (COVID-19) has significantly spread over the world and comes up with new challenges to the research community. Although governments imposing numerous containment and social distancing measures, the need for the healthcare systems has dramatically increased and the effective management of infected patients becomes a challenging problem for hospitals. Thus, accurate short-term forecasting of the number of new contaminated and recovered cases is crucial for optimizing the available resources and arresting or slowing down the progression of such diseases. Recently, deep learning models demonstrated important improvements when handling time-series data in different applications. This paper presents a comparative study of five deep learning methods to forecast the number of new cases and recovered cases. Specifically, simple Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Gated recurrent units (GRUs) and Variational AutoEncoder (VAE) algorithms have been applied for global forecasting of COVID-19 cases based on a small volume of data. This study is based on daily confirmed and recovered cases collected from six countries namely Italy, Spain, France, China, USA, and Australia. Results demonstrate the promising potential of the deep learning model in forecasting COVID-19 cases and highlight the superior performance of the VAE compared to the other algorithms.
• #### Advances in Statistical Modeling of Spatial Extremes

(arXiv, 2020-07-01) [Preprint]
The classical modeling of spatial extremes relies on asymptotic models (i.e., max-stable processes or $r$-Pareto processes) for block maxima or peaks over high thresholds, respectively. However, at finite levels, empirical evidence often suggests that such asymptotic models are too rigidly constrained, and that they do not adequately capture the frequent situation where more severe events tend to be spatially more localized. In other words, these asymptotic models have a strong tail dependence that persists at increasingly high levels, while data usually suggest that it should weaken instead. Another well-known limitation of classical spatial extremes models is that they are either computationally prohibitive to fit in high dimensions, or they need to be fitted using less efficient techniques. In this review paper, we describe recent progress in the modeling and inference for spatial extremes, focusing on new models that have more flexible tail structures that can bridge asymptotic dependence classes, and that are more easily amenable to likelihood-based inference for large datasets. In particular, we discuss various types of random scale constructions, as well as the conditional spatial extremes model, which have recently been getting increasing attention within the statistics of extremes community. We illustrate some of these new spatial models on two different environmental applications.
• #### Break Point Detection for Functional Covariance

(arXiv, 2020-06-24) [Preprint]
Many experiments record sequential trajectories that oscillate around zero. Such trajectories can be viewed as zero-mean functional data. When there are structural breaks (on the sequence of curves) in higher order moments, it is often difficult to spot these by mere visual inspection. Thus, we propose a detection and testing procedure to find the change-points in functional covariance. The method is fully functional in the sense that no dimension reduction is needed. We establish the asymptotic properties of the estimated change-point. The effectiveness of the proposed method is numerically validated in the simulation studies and an application to study structural changes in rat brain signals in a stroke experiment.
• #### Conditional Normal Extreme-Value Copulas

(arXiv, 2020-06-21) [Preprint]
We propose a new class of extreme-value copulas which are extreme-value limits of conditional normal models. Conditional normal models are generalizations of conditional independence models, where the dependence among observed variables is modeled using one unobserved factor. Conditional on this factor, the distribution of these variables is given by the Gaussian copula. This structure allows one to build flexible and parsimonious models for data with complex dependence structures, such as data with spatial or temporal dependence. We study the extreme-value limits of these models and show some interesting special cases of the proposed class of copulas. We develop estimation methods for the proposed models and conduct a simulation study to assess the performance of these algorithms. Finally, we apply these copula models to analyze data on monthly wind maxima and stock return minima.