RSS 1.0Computer Science Program
For more information visit: https://cemse.kaust.edu.sa/cs
Recent Submissions
-
Smooth monotone stochastic variational inequalities and saddle point problems: A survey(European Mathematical Society Magazine, European Mathematical Society - EMS - Publishing House GmbH, 2023-03-22) [Article]This paper is a survey of methods for solving smooth, (strongly) monotone stochastic variational inequalities. To begin with, we present the deterministic foundation from which the stochastic methods eventually evolved. Then we review methods for the general stochastic formulation, and look at the finite-sum setup. The last parts of the paper are devoted to various recent (not necessarily stochastic) advances in algorithms for variational inequalities.
-
Interpretation, Verification and Privacy Techniques for Improving the Trustworthiness of Neural Networks(2023-03-22) [Dissertation]
Advisor: Canini, Marco
Committee members: Charalambos, Konstantinou; Shihada, Basem; Pescapè, AntonioNeural Networks are powerful tools used in Machine Learning to solve complex problems across many domains, including biological classification, self-driving cars, and automated management of distributed systems. However, practitioners' trust in Neural Network models is limited by their inability to answer important questions about their behavior, such as whether they will perform correctly or if they can be entrusted with private data. One major issue with Neural Networks is their "black-box" nature, which makes it challenging to inspect the trained parameters or to understand the learned function. To address this issue, this thesis proposes several new ways to increase the trustworthiness of Neural Network models. The first approach focuses specifically on Piecewise Linear Neural Networks, a popular flavor of Neural Networks used to tackle many practical problems. The thesis explores several different techniques to extract the weights of trained networks efficiently and use them to verify and understand the behavior of the models. The second approach shows how strengthening the training algorithms can provide guarantees that are theoretically proven to hold even for the black-box model. The first part of the thesis identifies errors that can exist in trained Neural Networks, highlighting the importance of domain knowledge and the pitfalls to avoid with trained models. The second part aims to verify the outputs and decisions of the model by adapting the technique of Mixed Integer Linear Programming to efficiently explore the possible states of the Neural Network and verify properties of its outputs. The third part extends the Linear Programming technique to explain the behavior of a Piecewise Linear Neural Network by breaking it down into its linear components, generating model explanations that are both continuous on the input features and without approximations. Finally, the thesis addresses privacy concerns by using Trusted Execution and Differential Privacy during the training process. The techniques proposed in this thesis provide strong, theoretically provable guarantees about Neural Networks, despite their black-box nature, and enable practitioners to verify, extend, and protect the privacy of expert domain knowledge. By improving the trustworthiness of models, these techniques make Neural Networks more likely to be deployed in real-world applications. -
A Resource Allocation Scheme for Energy Demand Management in 6G-enabled Smart Grid(IEEE, 2023-03-22) [Conference Paper]Smart grid (SG) systems enhance grid resilience and efficient operation, leveraging the bidirectional flow of energy and information between generation facilities and prosumers. For energy demand management (EDM), the SG network requires computing a large amount of data generated by massive Internet-of-things sensors and advanced metering infrastructure (AMI) with minimal latency. This paper proposes a deep reinforcement learning (DRL)-based resource allocation scheme in a 6G-enabled SG edge network to offload resource-consuming EDM computation to edge servers. Automatic resource provisioning is achieved by harnessing the computational capabilities of smart meters in the dynamic edge network. To enforce DRL-assisted policies in dense 6G networks, the state information from multiple edge servers is required. However, adversaries can “poison” such information through false state injection (FSI) attacks, exhausting SG edge computing resources. Toward addressing this issue, we investigate the impact of such FSI attacks with respect to abusive utilization of edge resources, and develop a lightweight FSI detection mechanism based on supervised classifiers. Simulation results demonstrate the efficacy of DRL in dynamic resource allocation, the impact of the FSI attacks, and the effectiveness of the detection technique.
-
AB-Gen: Antibody Library Design with Generative Pre-trained Transformer and Deep Reinforcement Learning(Cold Spring Harbor Laboratory, 2023-03-21) [Preprint]Antibody leads must fulfill multiple desirable properties to be clinical candidates. Primarily due to the low throughput in the experimental procedure, the need for such multi-property optimization causes the bottleneck in preclinical antibody discovery and development, because addressing one issue usually causes another. We developed a reinforcement learning (RL) method, named AB-Gen, for antibody library design using a generative pre-trained Transformer (GPT) as the policy network of the RL agent. We showed that this model can learn the antibody space of heavy chain complementarity determining region 3 (CDRH3) and generate sequences with similar property distributions. Besides, when using HER2 as the target, the agent model of AB-Gen was able to generate novel CDRH3 sequences that fulfill multi-property constraints. 509 generated sequences were able to pass all property filters and three highly conserved residues were identified. The importance of these residues was further demonstrated by molecular dynamics simulations, which consolidated that the agent model was capable of grasping important information in this complex optimization task. Overall, the AB-Gen method is able to design novel antibody sequences with an improved success rate than the traditional propose-then-filter approach. It has the potential to be used in practical antibody design, thus empowering the antibody discovery and development process.
-
A comprehensive benchmarking with practical guidelines for cellular deconvolution of spatial transcriptomics(Nature Communications, Springer Science and Business Media LLC, 2023-03-21) [Article]Spatial transcriptomics technologies are used to profile transcriptomes while preserving spatial information, which enables high-resolution characterization of transcriptional patterns and reconstruction of tissue architecture. Due to the existence of low-resolution spots in recent spatial transcriptomics technologies, uncovering cellular heterogeneity is crucial for disentangling the spatial patterns of cell types, and many related methods have been proposed. Here, we benchmark 18 existing methods resolving a cellular deconvolution task with 50 real-world and simulated datasets by evaluating the accuracy, robustness, and usability of the methods. We compare these methods comprehensively using different metrics, resolutions, spatial transcriptomics technologies, spot numbers, and gene numbers. In terms of performance, CARD, Cell2location, and Tangram are the best methods for conducting the cellular deconvolution task. To refine our comparative results, we provide decision-tree-style guidelines and recommendations for method selection and their additional features, which will help users easily choose the best method for fulfilling their concerns.
-
Generalization of the Orthodiagonal Involutive Type of Kokotsakis Flexible Polyhedra(arXiv, 2023-03-19) [Preprint]In this paper we introduce and study a remarkable class of mechanisms formed by a 3×3 arrangement of rigid and skew quadrilateral faces with revolute joints at the common edges. These Kokotsakis-type mechanisms with a quadrangular base and non-planar faces are a generalization of Izmestiev's orthodiagonal involutive type of Kokotsakis polyhedra formed by planar quadrilateral faces. Our algebraic approach yields a complete characterization of all complexes of the orthodiagonal involutive type. It is shown that one has 8 degrees of freedom to construct such mechanisms. This is illustrated by several examples, including cases that are not possible with planar faces.
-
A universal framework for single-cell multi-omics data integration with graph convolutional networks(Briefings in bioinformatics, Oxford University Press (OUP), 2023-03-17) [Article]Single-cell omics data are growing at an unprecedented rate, whereas effective integration of them remains challenging due to different sequencing methods, quality, and expression pattern of each omics data. In this study, we propose a universal framework for the integration of single-cell multi-omics data based on graph convolutional network (GCN-SC). Among the multiple single-cell data, GCN-SC usually selects one data with the largest number of cells as the reference and the rest as the query dataset. It utilizes mutual nearest neighbor algorithm to identify cell-pairs, which provide connections between cells both within and across the reference and query datasets. A GCN algorithm further takes the mixed graph constructed from these cell-pairs to adjust count matrices from the query datasets. Finally, dimension reduction is performed by using non-negative matrix factorization before visualization. By applying GCN-SC on six datasets, we show that GCN-SC can effectively integrate sequencing data from multiple single-cell sequencing technologies, species or different omics, which outperforms the state-of-the-art methods, including Seurat, LIGER, GLUER and Pamona.
-
miProBERT: identification of microRNA promoters based on the pre-trained model BERT.(Briefings in bioinformatics, Oxford University Press (OUP), 2023-03-17) [Article]Accurate prediction of promoter regions driving miRNA gene expression has become a major challenge due to the lack of annotation information for pri-miRNA transcripts. This defect hinders our understanding of miRNA-mediated regulatory networks. Some algorithms have been designed during the past decade to detect miRNA promoters. However, these methods rely on biosignal data such as CpG islands and still need to be improved. Here, we propose miProBERT, a BERT-based model for predicting promoters directly from gene sequences without using any structural or biological signals. According to our information, it is the first time a BERT-based model has been employed to identify miRNA promoters. We use the pre-trained model DNABERT, fine-tune the pre-trained model on the gene promoter dataset so that the model includes information about the richer biological properties of promoter sequences in its representation, and then systematically scan the upstream regions of each intergenic miRNA using the fine-tuned model. About, 665 miRNA promoters are found. The innovative use of a random substitution strategy to construct a negative dataset improves the discriminative ability of the model and further reduces the false positive rate (FPR) to as low as 0.0421. On independent datasets, miProBERT outperformed other gene promoter prediction methods. With comparison on 33 experimentally validated miRNA promoter datasets, miProBERT significantly outperformed previously developed miRNA promoter prediction programs with 78.13% precision and 75.76% recall. We further verify the predicted promoter regions by analyzing conservation, CpG content and histone marks. The effectiveness and robustness of miProBERT are highlighted.
-
LEP-AD: Language Embedding of Proteins and Attention to Drugs predicts drug target interactions(Cold Spring Harbor Laboratory, 2023-03-15) [Preprint]Predicting drug-target interactions is a tremendous challenge for drug development and lead optimization. Recent advances include training algorithms to learn drug-target interactions from data and molecular simulations. Here we utilize Evolutionary Scale Modeling (ESM-2) models to establish a Transformer protein language model for drug-target interaction predictions. Our architecture, LEP- AD, combines pre-trained ESM-2 and Transformer-GCN models predicting bind-ing affinity values. We report new best-in-class state-of-the-art results compared to competing methods such as SimBoost, DeepCPI, Attention-DTA, GraphDTA, and more using multiple datasets, including Davis, KIBA, DTC, Metz, ToxCast, and STITCH. Finally, we find that a pre-trained model with embedding of proteins (the LED-AD) outperforms a model using an explicit alpha-fold 3D representation of proteins (e.g., LEP-AD supervised by Alphafold). The LEP-AD model scales favorably in performance with the size of training data.
-
Large Numerical Aperture Metalens with High Modulation Transfer Function(ACS Photonics, American Chemical Society (ACS), 2023-03-14) [Article]Large numerical aperture (NA) lenses with high modulation transfer functions (MTFs) promise high image resolution for advanced optical imaging. However, it is challenging to achieve a high MTF using traditional large-NA lenses, which are fundamentally limited by the amplitude mismatch. In contrast, metasurfaces are promising for realizing amplitude and phase matching for ideal lenses. However, current metalenses are mostly based on a phase-only (PO) profile because the strong coupling among the metaatoms in large-NA lenses makes perfect amplitude matching quite challenging to realize. Here, we derive a phase-and-amplitude (PA) profile that approaches the theoretical MTF limit for large-NA lenses and use interferometric unit cells combined with a segmented sampling approach to achieve the desired amplitude and phase control. For the first time, we show that the amplitude does not require a perfect match; realizing the trend of the required amplitude is sufficient to significantly increase the MTF of a large-NA lens. We demonstrated a 0.9 NA cylindrical metalens at 940 nm with a Struve ratio (SR), which describes how close the MTF is to the upper limit, increasing from 0.68 to 0.90 compared with the PO metalens. Experimentally, we achieved an SR of 0.77 for the 0.9 NA lens, which is even 0.09 higher than the simulated SR of the PO metalens. Our investigation provides new insights for large-NA lenses and has potential applications in high-image-resolution optical systems.
-
DualMix: Unleashing the Potential of Data Augmentation for Online Class-Incremental Learning(arXiv, 2023-03-14) [Preprint]Online Class-Incremental (OCI) learning has sparked new approaches to expand the previously trained model knowledge from sequentially arriving data streams with new classes. Unfortunately, OCI learning can suffer from catastrophic forgetting (CF) as the decision boundaries for old classes can become inaccurate when perturbated by new ones. Existing literature have applied the data augmentation (DA) to alleviate the model forgetting, while the role of DA in OCI has not been well understood so far. In this paper, we theoretically show that augmented samples with lower correlation to the original data are more effective in preventing forgetting. However, aggressive augmentation may also reduce the consistency between data and corresponding labels, which motivates us to exploit proper DA to boost the OCI performance and prevent the CF problem. We propose the Enhanced Mixup (EnMix) method that mixes the augmented samples and their labels simultaneously, which is shown to enhance the sample diversity while maintaining strong consistency with corresponding labels. Further, to solve the class imbalance problem, we design an Adaptive Mixup (AdpMix) method to calibrate the decision boundaries by mixing samples from both old and new classes and dynamically adjusting the label mixing ratio. Our approach is demonstrated to be effective on several benchmark datasets through extensive experiments, and it is shown to be compatible with other replay-based techniques.
-
Spatiotemporal data management and analytics for recommender systems(World Wide Web, Springer Science and Business Media LLC, 2023-03-13) [Article]
-
ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions(arXiv, 2023-03-12) [Preprint]Asking insightful questions is crucial for acquiring knowledge and expanding our understanding of the world. However, the importance of questioning has been largely overlooked in AI research, where models have been primarily developed to answer questions. With the recent advancements of large language models (LLMs) like ChatGPT, we discover their capability to ask high-quality questions when provided with a suitable prompt. This discovery presents a new opportunity to develop an automatic questioning system. In this paper, we introduce ChatCaptioner, a novel automatic-questioning method deployed in image captioning. Here, ChatGPT is prompted to ask a series of informative questions about images to BLIP-2, a strong vision question-answering model. By keeping acquiring new visual information from BLIP-2's answers, ChatCaptioner is able to generate more enriched image descriptions. We conduct human-subject evaluations on common image caption datasets such as COCO, Conceptual Caption, and WikiArt, and compare ChatCaptioner with BLIP-2 as well as ground truth. Our results demonstrate that ChatCaptioner's captions are significantly more informative, receiving three times as many votes from human evaluators for providing the most image information. Besides, ChatCaptioner identifies 53% more objects within the image than BLIP-2 alone measured by WordNet synset matching.
-
An Effective Security Scheme for Attacks on Sample Value Messages in IEC 61850 Automated Substations(IEEE Open Access Journal of Power and Energy, Institute of Electrical and Electronics Engineers (IEEE), 2023-03-10) [Article]The trend of transforming substations into smart automated facilities has led to their swift digitalization and automation. To facilitate data exchange among equipment within these substations, the IEC 61850 standard has become the predominant standard. However, this standardization has inadvertently made these substations more susceptible to cyberattacks, which is a significant concern given the confidential information that is transmitted. As a result, cybersecurity in substations is becoming an increasingly critical topic. IEC 62351 standard provides guidelines and considerations for securing the IEC 61850 messages to mitigate their vulnerabilities. While securing Generic Object-Oriented Substation Event (GOOSE) messages has received considerable attention in literature, the same level of scrutiny has not been applied to Sampled Value (SV) messages despite their susceptibility to cyberattacks and similar frame format. This paper presents the impact of replay and masquerade attacks on SV messages. It also develops a scheme for securing SV messages against these attacks. Due to high sampling rate and time critical nature of SV messages, the time complexity of security scheme is critical for its applicability to SV messages. Hence, in this work, SV emulators have been developed in order to send these modified secure SV messages and investigate their timing performance. The results show that the proposed scheme can mitigate replay and masquerade attacks on SV messages while providing the necessary high sampling rate and stringent timing requirements.
-
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression(arXiv, 2023-03-08) [Preprint]Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics. This paper studies variants of such algorithms called Error Feedback Langevin algorithms (ELF). In particular, we analyze the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo. We propose three algorithms: P-ELF, D-ELF, and B-ELF that use, respectively, primal, dual, and bidirectional compressors. We analyze the proposed methods under Log-Sobolev inequality and provide non-asymptotic convergence guarantees.
-
Aberration-Aware Depth-from-Focus(arXiv, 2023-03-08) [Preprint]Computer vision methods for depth estimation usually use simple camera models with idealized optics. For modern machine learning approaches, this creates an issue when attempting to train deep networks with simulated data, especially for focus-sensitive tasks like Depth-from-Focus. In this work, we investigate the domain gap caused by off-axis aberrations that will affect the decision of the best-focused frame in a focal stack. We then explore bridging this domain gap through aberration-aware training (AAT). Our approach involves a lightweight network that models lens aberrations at different positions and focus distances, which is then integrated into the conventional network training pipeline. We evaluate the generality of pretrained models on both synthetic and real-world data. Our experimental results demonstrate that the proposed AAT scheme can improve depth estimation accuracy without fine-tuning the model or modifying the network architecture.
-
Diabetic cardiomyopathy: The role of microRNAs and long non-coding RNAs(Frontiers in Endocrinology, Frontiers Media SA, 2023-03-07) [Article]Diabetes mellitus (DM) is on the rise, necessitating the development of novel therapeutic and preventive strategies to mitigate the disease’s debilitating effects. Diabetic cardiomyopathy (DCMP) is among the leading causes of morbidity and mortality in diabetic patients globally. DCMP manifests as cardiomyocyte hypertrophy, apoptosis, and myocardial interstitial fibrosis before progressing to heart failure. Evidence suggests that non-coding RNAs, such as long non-coding RNAs (lncRNAs) and microRNAs (miRNAs), regulate diabetic cardiomyopathy-related processes such as insulin resistance, cardiomyocyte apoptosis and inflammation, emphasizing their heart-protective effects. This paper reviewed the literature data from animal and human studies on the non-trivial roles of miRNAs and lncRNAs in the context of DCMP in diabetes and demonstrated their future potential in DCMP treatment in diabetic patients.
-
A Comprehensive Empirical Study of Heterogeneity in Federated Learning(IEEE Internet of Things Journal, Institute of Electrical and Electronics Engineers (IEEE), 2023-03-07) [Article]Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployment in production environments, and it has been adopted in services such as virtual keyboards, auto-completion, item recommendation, and several IoT applications. However, FL comes with the challenge of performing training over largely heterogeneous datasets, devices, and networks that are out of the control of the centralized FL server. Motivated by this inherent challenge, we aim to empirically characterize the impact of device and behavioral heterogeneity on the trained model. We conduct an extensive empirical study spanning nearly 1.5K unique configurations on five popular FL benchmarks. Our analysis shows that these sources of heterogeneity have a major impact on both model quality and fairness, causing up to 4.6× and 2.2× degradation in the quality and fairness, respectively, thus shedding light on the importance of considering heterogeneity in FL system design.
-
Satellite-Aerial Communications With Multi-aircraft Interference(IEEE Transactions on Wireless Communications, Institute of Electrical and Electronics Engineers (IEEE), 2023-03-02) [Article]Satellite-aerial communication (SAC) is envisioned as a fundamental component of the sixth-generation (6G) wireless networks. Motivated by its importance, we investigate a SAC system including a geostationary satellite (S), a target aircraft (TA), and a set of interfering aircraft (IA). Specifically, TA sends signals to S in the presence of IA interference. Considering the trajectory, hierarchy, and safety distance of the aircraft’s flight routes, we propose a novel three-dimensional stacked Poisson line hardcore point process. That is, we introduce safety distances to the stacked Poisson line Cox process in order to describe the locations of IA in the sky. We also propose two approximations, namely, the equi-dense model and the discretization model, to maintain the tractability of the analysis. To this end, the uplink coverage probability is studied by using the two proposed mathematical models. Moreover, we investigate the coverage probability of the aviation use case with predefined flight altitudes. Finally, numerical results and Monte Carlo simulations are presented to validate the accuracy of the proposed analysis.
-
Computational network analysis of host genetic risk variants of severe COVID-19.(Human genomics, Springer Science and Business Media LLC, 2023-03-02) [Article]Background: Genome-wide association studies have identified numerous human host genetic risk variants that play a substantial role in the host immune response to SARS-CoV-2. Although these genetic risk variants significantly increase the severity of COVID-19, their influence on body systems is poorly understood. Therefore, we aim to interpret the biological mechanisms and pathways associated with the genetic risk factors and immune responses in severe COVID-19. We perform a deep analysis of previously identified risk variants and infer the hidden interactions between their molecular networks through disease mapping and the similarity of the molecular functions between constructed networks. Results: We designed a four-stage computational workflow for systematic genetic analysis of the risk variants. We integrated the molecular profiles of the risk factors with associated diseases, then constructed protein–protein interaction networks. We identified 24 protein–protein interaction networks with 939 interactions derived from 109 filtered risk variants in 60 risk genes and 56 proteins. The majority of molecular functions, interactions and pathways are involved in immune responses; several interactions and pathways are related to the metabolic and cardiovascular systems, which could lead to multi-organ complications and dysfunction. Conclusions: This study highlights the importance of analyzing molecular interactions and pathways to understand the heterogeneous susceptibility of the host immune response to SARS-CoV-2. We propose new insights into pathogenicity analysis of infections by including genetic risk information as essential factors to predict future complications during and after infection. This approach may assist more precise clinical decisions and accurate treatment plans to reduce COVID-19 complications.
