RSS 1.0Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division
For more information visit: https://cemse.kaust.edu.sa/
Recent Submissions
-
HAP-enabled Communications in Rural Areas: When Diverse Services Meet Inadequate Communication Infrastructures(IEEE Open Journal of the Communications Society, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-25) [Article]The high altitude platform (HAP) network has been regarded as a cost-efficient solution for providing network access to rural or remote areas. Apart from network connectivity, rural areas are predicted to have demands for diverse real-time intelligent communication services, such as smart agriculture and digital forestry. The effectiveness of real-time decision-making applications depends on the timely updating of sensing data measurements used in generating decisions. As a performance metric capable of quantifying the freshness of transmitted information, the age of information (AoI) can evaluate the freshness-aware performance of the process of updating sensory data. However, unlike urban areas, the available communication resources in rural areas may not allow for maintaining dedicated infrastructures for different types of services, e.g., conventional non-freshness-aware services and freshness-aware real-time services, thereby requiring the proper resource allocation among different services. In this article, we first introduce the anticipated services and discuss the advances of rural networks. Next, a case study on the efficient resource allocation across heterogeneous services characterized by AoI and data rate in HAP networks is presented. We also explore the potential of employing the free-space optical (FSO) backhaul framework to enhance the performance of multi-layer HAP networks. To strike a balance between the AoI and data rate, we develop both static and deep reinforcement learning (DRL)-based dynamic resource allocation schemes to allocate the communication resources provided by HAP networks. The simulation results show that the proposed dynamic DRL-based method outperforms the heuristic algorithm and can surpass the performance ceiling achieved by the proposed static allocation scheme. In particular, our presented method can improve performance by nearly 2.5 times more than the ant colony optimization (ACO) method in terms of weighted sum performance improvements. Some insights on system design and promising future research directions are also given.
-
Automatic Animation of Hair Blowing in Still Portrait Photos(arXiv, 2023-09-25) [Preprint]We propose a novel approach to animate human hair in a still portrait photo. Existing work has largely studied the animation of fluid elements such as water and fire. However, hair animation for a real image remains underexplored, which is a challenging problem, due to the high complexity of hair structure and dynamics. Considering the complexity of hair structure, we innovatively treat hair wisp extraction as an instance segmentation problem, where a hair wisp is referred to as an instance. With advanced instance segmentation networks, our method extracts meaningful and natural hair wisps. Furthermore, we propose a wisp-aware animation module that animates hair wisps with pleasing motions without noticeable artifacts. The extensive experiments show the superiority of our method. Our method provides the most pleasing and compelling viewing experience in the qualitative experiments and outperforms state-of-the-art still-image animation methods by a large margin in the quantitative evaluation.
-
Bayesian Parameter Inference for Partially Observed Diffusions using Multilevel Stochastic Runge-Kutta Methods(arXiv, 2023-09-24) [Preprint]We consider the problem of Bayesian estimation of static parameters associated to a partially and discretely observed diffusion process. We assume that the exact transition dynamics of the diffusion process are unavailable, even up-to an unbiased estimator and that one must time-discretize the diffusion process. In such scenarios it has been shown how one can introduce the multilevel Monte Carlo method to reduce the cost to compute posterior expected values of the parameters for a pre-specified mean square error (MSE). These afore-mentioned methods rely on upon the Euler-Maruyama discretization scheme which is well-known in numerical analysis to have slow convergence properties. We adapt stochastic Runge-Kutta (SRK) methods for Bayesian parameter estimation of static parameters for diffusions. This can be implemented in high-dimensions of the diffusion and seemingly under-appreciated in the uncertainty quantification and statistics fields. For a class of diffusions and SRK methods, we consider the estimation of the posterior expectation of the parameters. We prove that to achieve a MSE of O(ϵ2), for ϵ>0 given, the associated work is O(ϵ−2). Whilst the latter is achievable for the Milstein scheme, this method is often not applicable for diffusions in dimension larger than two. We also illustrate our methodology in several numerical examples.
-
Comparing Aerial-RIS- and Aerial-Base-Station-Aided Post-Disaster Cellular Networks(IEEE Open Journal of Vehicular Technology, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-22) [Article]Reconfigurable intelligent surface (RIS) technology and its integration into existing wireless networks have recently attracted much interest. While an important use case of said technology consists in mounting RISs onto unmanned aerial vehicles (UAVs) to support the terrestrial infrastructure in post-disaster scenarios, the current literature lacks an analytical framework that captures the networks' topological aspects. Therefore, our study borrows stochastic geometry tools to estimate both the average and local coverage probability of a wireless network aided by an aerial RIS (ARIS); in particular, the surviving terrestrial base stations (TBSs) are modeled by means of an inhomogeneous Poisson point process, while the UAV is assumed to hover above the disaster epicenter. Our framework captures important aspects such as the TBSs' altitude, the fact that they may be in either line-of-sight or non-line-of-sight condition with a given node, and the Nakagami- m fading conditions of wireless links. By leveraging said aspects we accurately evaluate three possible scenarios, where TBSs are either: (i) not aided, (ii) aided by an ARIS, or (iii) aided by an aerial base station (ABS). Our selected numerical results reflect various situations, depending on parameters such as the environment's urbanization level, disaster radius, and the UAV's altitude.
-
Adaptive Differentiable Grids for Cryo-Electron Tomography Reconstruction and Denoising(Bioinformatics Advances, Oxford University Press (OUP), 2023-09-22) [Article]Motivation: Tilt-series cryo-Electron Tomography is a powerful tool widely used in structural biology to study three-dimensional structures of micro-organisms, macromolecular complexes, etc. Still the reconstruction process remains an arduous task due to several challenges: The missing-wedge acquisition, sample misalignment and motion, the need to process large data, and especially a low signal-to-noise ratio (SNR). Results: Inspired by the recently introduced neural representations, we propose an adaptive learned-based representation of the density field of the captured sample. This representation consists of an octree structure, where each node represents a 3D density grid optimized from the captured projections during the training process. This optimization is performed using a loss that combines a differentiable image formation model with different regularization terms: total variation, boundary consistency, and a cross-nodes non-local constraint. The final reconstruction is obtained by interpolating the learned density grid at the desired voxel positions. The evaluation of our approach using captured data of viruses and cells shows that our proposed representation is well-adapted to handle missing-wedges, and improves the SNR of the reconstructed tomogram. The reconstruction quality is highly improved in comparison to the state-of-the-art methods, while using the lowest computing time footprint.
-
AceGPT, Localizing Large Language Models in Arabic(arXiv, 2023-09-22) [Preprint]This paper explores the imperative need and methodology for developing a localized Large Language Model (LLM) tailored for Arabic, a language with unique cultural characteristics that are not adequately addressed by current mainstream models like ChatGPT. Key concerns additionally arise when considering cultural sensitivity and local values. To this end, the paper outlines a packaged solution, including further pre-training with Arabic texts, supervised fine-tuning (SFT) using native Arabic instructions and GPT-4 responses in Arabic, and reinforcement learning with AI feedback (RLAIF) using a reward model that is sensitive to local culture and values. The objective is to train culturally aware and value-aligned Arabic LLMs that can serve the diverse application-specific needs of Arabic-speaking communities. Extensive evaluations demonstrated that the resulting LLM called `\textbf{AceGPT}' is the SOTA open Arabic LLM in various benchmarks, including instruction-following benchmark (i.e., Arabic Vicuna-80 and Arabic AlpacaEval), knowledge benchmark (i.e., Arabic MMLU and EXAMs), as well as the newly-proposed Arabic cultural \& value alignment benchmark. Notably, AceGPT outperforms ChatGPT in the popular Vicuna-80 benchmark when evaluated with GPT-4, despite the benchmark's limited scale.
-
Oil Spill Risk Analysis For The NEOM Shoreline(2023-09-21) [Preprint]A risk analysis is conducted considering an array of release sources located around the NEOM shoreline. The sources are selected close to the coast and in neighboring regions of high marine traffic. The evolution of oil spills released by these sources is simulated using the MOHID model, driven by validated, high-resolution met-ocean fields of the Red Sea. For each source, simulations are conducted over a 4-week period, starting from first, tenth and twentieth days of each month, covering five consecutive years. A total of 48 simulations are thus conducted for each source location, adequately reflecting the variability of met-ocean conditions in the region. The risk associated with each source is described in terms of amount of oil beached, and by the elapsed time required for the spilled oil to reach the NEOM coast, extending from the Gulf of Aqaba in the North to Duba in the South. To further characterize the impact of individual sources, a finer analysis is performed by segmenting the NEOM shoreline, based on important coastal development and installation sites. For each subregion, source and release event considered, a histogram of the amount of volume beached is generated, also classifying individual events in terms of the corresponding arrival times. In addition, for each subregion considered, an inverse analysis is conducted to identify regions of dependence of the cumulative risk, estimated using the collection of all sources and events considered. The transport of oil around the NEOM shorelines is promoted by chaotic circulations and northwest winds in summer, and a dominant cyclonic eddy in winter. Hence, spills originating from release sources located close to the NEOM shorelines are characterized by large monthly variations in arrival times, ranging from less than a week to more than two weeks. Similarly, large variations in the volume fraction of beached oil, ranging from less then 50\% to more than 80\% are reported. The results of this study provide key information regarding the location of dominant oil spill risk sources, the severity of the potential release events, as well as the time frames within which mitigation actions may need to deployed.
-
The Bayesian Learning Rule(Accepted by Journal of Machine Learning Research, 2023-09-21) [Article]We show that many machine-learning algorithms are specific instances of a single algorithm called the Bayesian learning rule. The rule, derived from Bayesian principles, yields a wide-range of algorithms from fields such as optimization, deep learning, and graphical models. This includes classical algorithms such as ridge regression, Newton's method, and Kalman filter, as well as modern deep-learning algorithms such as stochastic-gradient descent, RMSprop, and Dropout. The key idea in deriving such algorithms is to approximate the posterior using candidate distributions estimated by using natural gradients. Different candidate distributions result in different algorithms and further approximations to natural gradients give rise to variants of those algorithms. Our work not only unifies, generalizes, and improves existing algorithms, but also helps us design new ones.
-
Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package(arXiv, 2023-09-21) [Preprint]We propose a parallel (distributed) version of the spectral proper orthogonal decomposition (SPOD) technique. The parallel SPOD algorithm distributes the spatial dimension of the dataset preserving time. This approach is adopted to preserve the non-distributed fast Fourier transform of the data in time, thereby avoiding the associated bottlenecks. The parallel SPOD algorithm is implemented in the PySPOD library and makes use of the standard message passing interface (MPI) library, implemented in Python via mpi4py. An extensive performance evaluation of the parallel package is provided, including strong and weak scalability analyses. The open-source library allows the analysis of large datasets of interest across the scientific community. Here, we present applications in fluid dynamics and geophysics, that are extremely difficult (if not impossible) to achieve without a parallel algorithm. This work opens the path toward modal analyses of big quasi-stationary data, helping to uncover new unexplored spatiotemporal patterns.
-
Artificial Intelligence Techniques for Solar Irradiance and PV Modeling and Forecasting(Energies, MDPI AG, 2023-09-21) [Article]
-
Pointing-and-Acquisition for Optical Wireless in 6G: From Algorithms to Performance Evaluation(arXiv, 2023-09-20) [Preprint]The increasing demand for wireless communication services has led to the development of non-terrestrial networks, which enables various air and space applications. Free-space optical (FSO) communication is considered one of the essential technologies capable of connecting terrestrial and non-terrestrial layers. In this article, we analyze considerations and challenges for FSO communications between gateways and aircraft from a pointing-and-acquisition perspective. Based on the analysis, we first develop a baseline method that utilizes conventional devices and mechanisms. Furthermore, we propose an algorithm that combines angle of arrival (AoA) estimation through supplementary radio frequency (RF) links and beam tracking using retroreflectors. Through extensive simulations, we demonstrate that the proposed method offers superior performance in terms of link acquisition and maintenance.
-
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute(arXiv, 2023-09-20) [Preprint]The Languini Kitchen serves as both a research collective and codebase designed to empower researchers with limited computational resources to contribute meaningfully to the field of language modelling. We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. The number of tokens on which a model is trained is defined by the model's throughput and the chosen compute class. Notably, this approach avoids constraints on critical hyperparameters which affect total parameters or floating-point operations. For evaluation, we pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. On it, we compare methods based on their empirical scaling trends which are estimated through experiments at various levels of compute. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput. While the GPT baseline achieves better perplexity throughout all our levels of compute, our LSTM baseline exhibits a predictable and more favourable scaling law. This is due to the improved throughput and the need for fewer training tokens to achieve the same decrease in test perplexity. Extrapolating the scaling laws leads of both models results in an intersection at roughly 50,000 accelerator hours. We hope this work can serve as the foundation for meaningful and reproducible language modelling research.
-
A Weighted Convex Optimized Phase Retrieval Method for Short-Time Fourier Transform Measurement With Outliers(IEEE Transactions on Instrumentation and Measurement, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-20) [Article]As a problem of reconstructing the original signal from phaseless short-time Fourier transform (STFT) measurement, STFT phase retrieval (PR) is widespread in many fields. The existing PR algorithms for STFT measurement can uniquely determine the original signal (up to a global phase), however they are invalid when outlier interference exists. Aiming at this problem, we propose a weighted convex optimization PR method by introducing a weight matrix that can identify outliers. Meanwhile, a two-channel phaseless measurement structure based on the mask technique and the corresponding calculation algorithm are proposed to obtain the weight matrix. In particular, by accumulating the measurement results of all short-time segments, the support of outliers can be well identified in the weight matrix calculation. Simulation and hardware experiments demonstrate the performance improvement of the proposed approach compared to the existing methods, which shows its effectiveness in suppressing outlier interference.
-
Unsupervised Deep Basis Pursuit Based Resolution Enhancement for Forward Looking MIMO SAR Imaging(IEEE Transactions on Aerospace and Electronic Systems, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-20) [Article]Nowadays, radar based image reconstruction is becoming important in higher-level automated driving, especially for all weather conditions. In this paper, we present unsupervised deep learning method for forward looking multiple-input multipleoutput synthetic aperture radar (FL-MIMO SAR) to enhance the angular resolution. We present mathematical analysis for the composite antenna pattern generated by FL-MIMO SAR as well as image reconstruction with deep learning for FL-MIMO SAR. We present a computationally efficient deep basis pursuit (DBP) method to solve for convolutional neural network (CNN) with unsupervised learning (i.e. without ground truth) and present modified back projection (MBP) algorithm to reconstruct SAR image with enhanced angular resolution. We present experimental results to verify our proposed methodology and compare the performance with compressed sensing based backprojection algorithm on both simulation and real data.
-
Precoding for High Throughput Satellite Communication Systems: A Survey(IEEE Communications Surveys & Tutorials, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-20) [Article]With the expanding demand for high data rates and extensive coverage, high throughput satellite (HTS) communication systems are emerging as a key technology for future communication generations. However, current frequency bands are increasingly congested. Until the maturity of communication systems to operate on higher bands, the solution is to exploit the already existing frequency bands more efficiently. In this context, precoding emerges as one of the prolific approaches to increasing spectral efficiency. This survey presents an overview and a classification of the recent precoding techniques for HTS communication systems from two main perspectives: 1) a problem formulation perspective and 2) a system design perspective. From a problem formulation point of view, precoding techniques are classified according to the precoding optimization problem, group, and level. From a system design standpoint, precoding is categorized based on the system architecture, the precoding implementation, and the type of the provided service. Further, practical system impairments are discussed, and robust precoding techniques are presented. Finally, future trends in precoding for satellites are addressed to spur further research.
-
Past, Present, and Future of Software for Bayesian Inference(Accepted by Statistical Science, 2023-09-19) [Article]Software tools for Bayesian inference have undergone rapid evolution in the past three decades, following popularisation of the first generation MCMC-sampler implementations. More recently, exponential growth in the number of users has been stimulated both by the active development of new packages by the machine learning community and popularity of specialist software for particular applications. This review aims to summarize the most popular software and provide a useful map for a reader to navigate the world of Bayesian computation. We anticipate a vigorous continued development of algorithms and corresponding software in multiple research fields, such as probabilistic programming, likelihood-free inference, and Bayesian neural networks, which will further broaden the possibilities for employing the Bayesian paradigm in exciting applications.
-
Unbiased Parameter Estimation for Partially Observed Diffusions(arXiv, 2023-09-19) [Preprint]In this article we consider the estimation of static parameters for partially observed diffusion process with discrete-time observations over a fixed time interval. In particular, we assume that one must time-discretize the partially observed diffusion process and work with the model with bias and consider maximizing the resulting log-likelihood. Using a novel double randomization scheme, based upon Markovian stochastic approximation we develop a new method to unbiasedly estimate the static parameters, that is, to obtain the maximum likelihood estimator with no time discretization bias. Under assumptions we prove that our estimator is unbiased and investigate the method in several numerical examples, showing that it can empirically out-perform existing unbiased methodology.
-
Sex differences in mortality among children, adolescents, and young people aged 0-24 years: a systematic assessment of national, regional, and global trends from 1990 to 2021.(The Lancet. Global health, Elsevier BV, 2023-09-19) [Article]Background: Differences in mortality exist between sexes because of biological, genetic, and social factors. Sex differentials are well documented in children younger than 5 years but have not been systematically examined for ages 5–24 years. We aimed to estimate the sex ratio of mortality from birth to age 24 years and reconstruct trends in sex-specific mortality between 1990 and 2021 for 200 countries, major regions, and the world. Methods: We compiled comprehensive databases on the mortality sex ratio (ratio of male to female mortality rates) for individuals aged 0–4 years, 5–14 years, and 15–24 years. The databases contain mortality rates from death registration systems, full birth and sibling histories from surveys, and reports on household deaths in censuses. We modelled the sex ratio of age-specific mortality as a function of the mortality in both sexes using Bayesian hierarchical time-series models. We report the levels and trends of sex ratios and estimate the expected female mortality and excess female mortality rates (the difference between the estimated female mortality and the expected female mortality) to identify countries with outlying sex ratios. Findings: Globally, the mortality sex ratio was 1·13 (ie, boys were more likely to die than girls of the same age) for ages 0–4 years (90% uncertainty interval 1·11 to 1·15) in 2021. This ratio increased with age to 1·16 (1·12 to 1·20) for 5–14 years, reaching 1·65 for 15–24 years (1·52 to 1·75). In all age groups, the global sex ratio of mortality increased between 1990 and 2021, driven by faster declines in female mortality. In 2021, the probability of a newborn male reaching age 25 years was 94·1% (93·7 to 94·4), compared with 95·1% for a newborn female (94·7 to 95·3). We found a disadvantage of females versus males (compared with countries with similar total mortality) in 2021 in five countries for ages 0–4 years (Algeria, Bangladesh, Egypt, India, and Iran), one country (Suriname) for ages 5–14 years, and 13 countries for ages 15–24 years (including Bangladesh and India). We found the reverse pattern (disadvantage of males vs females compared with countries of similar total mortality) in one country in ages 0–4 years (Vietnam) and eight countries in ages 15–24 years (including Brazil and Mexico). Globally, the number of excess female deaths from birth to age 24 years was 86 563 (–6059 to 164 000) in 2021, down from 544 636 (453 982 to 633 265) in 1990. Interpretation: The global sex ratio of mortality for all age groups in the first 25 years of life increased between 1990 and 2021. Targeted interventions should focus on countries with outlying sex ratios of mortality to reduce disparities due to discrimination in health care, nutrition, and violence.
-
Spread-Spectrum Modulated Multi-Channel Biosignal Acquisition Using a Shared Analog CMOS Front-End(IEEE Transactions on Biomedical Circuits and Systems, Institute of Electrical and Electronics Engineers (IEEE), 2023-09-19) [Article]The key challenges in designing a multi-channel biosignal acquisition system for an ambulatory or invasive medical application with a high channel count are reducing the power consumption, area consumption and the outgoing wire count. This paper proposes a spread-spectrum modulated biosignal acquisition system using a shared amplifier and an analog-to-digital converter (ADC). We propose a design method to optimize a recording system for a given application based on the required SNR performance, number of inputs, and area. The proposed method is tested and validated on real pre-recorded atrial electrograms and achieves an average percentage root- mean-square difference (PRD) performance of 2.65% and 3.02% for sinus rhythm (SR) and atrial fibrillation (AF), respectively by using pseudo-random binary-sequence (PRBS) codes with a code-length of 511, for 16 inputs. We implement a 4-input spread-spectrum analog front-end in a 0.18μm CMOS process to demonstrate the proposed approach. The analog front-end consists of a shared amplifier, a 2nd order ΣΔ ADC sampled at 7.8MHz , used for digitization, and an on-chip 7-bit PRBS generator. It achieves a number-of-inputs to outgoing-wire ratio of 4:1 while consuming 23μA /input including biasing from a 1.8V power supply and 0.067mm2 in area.
-
Repetitive DNA sequence detection and its role in the human genome(Communications Biology, Springer Science and Business Media LLC, 2023-09-19) [Article]Repetitive DNA sequences playing critical roles in driving evolution, inducing variation, and regulating gene expression. In this review, we summarized the definition, arrangement, and structural characteristics of repeats. Besides, we introduced diverse biological functions of repeats and reviewed existing methods for automatic repeat detection, classification, and masking. Finally, we analyzed the type, structure, and regulation of repeats in the human genome and their role in the induction of complex diseases. We believe that this review will facilitate a comprehensive understanding of repeats and provide guidance for repeat annotation and in-depth exploration of its association with human diseases.
