THE KAUST Repository is an initiative of the University Library to expand the impact of conference papers, technical reports, peer-reviewed articles, preprints, theses, images, data sets, and other research-related works of King Abdullah University of Science and Technology (KAUST).
Files in the repository are accessible through popular web search engines and are given persistent web addresses so links will not become broken over time.
KAUST researchers: To add your research to the repository, click on Deposit your Research, log in with your KAUST user name and password, and deposit the item in the appropriate collection.
If you have any questions, please contact firstname.lastname@example.org.
On Seven Fundamental Optimization Challenges in Machine Learning(2021-10-14) [Dissertation]
Advisor: Richtarik, Peter
Committee members: Carin, Lawrence; Ghanem, Bernard; Sra, Suvrit; Yin, WotaoMany recent successes of machine learning went hand in hand with advances in optimization. The exchange of ideas between these fields has worked both ways, with ' learning building on standard optimization procedures such as gradient descent, as well as with new directions in the optimization theory stemming from machine learning applications. In this thesis, we discuss new developments in optimization inspired by the needs and practice of machine learning, federated learning, and data science. In particular, we consider seven key challenges of mathematical optimization that are relevant to modern machine learning applications, and develop a solution to each. Our first contribution is the resolution of a key open problem in Federated Learning: we establish the first theoretical guarantees for the famous Local SGD algorithm in the crucially important heterogeneous data regime. As the second challenge, we close the gap between the upper and lower bounds for the theory of two incremental algorithms known as Random Reshuffling (RR) and Shuffle-Once that are widely used in practice, and in fact set as the default data selection strategies for SGD in modern machine learning software. Our third contribution can be seen as a combination of our new theory for proximal RR and Local SGD yielding a new algorithm, which we call FedRR. Unlike Local SGD, FedRR is the first local first-order method that can provably beat gradient descent in communication complexity in the heterogeneous data regime. The fourth challenge is related to the class of adaptive methods. In particular, we present the first parameter-free stepsize rule for gradient descent that provably works for any locally smooth convex objective. The fifth challenge we resolve in the affirmative is the development of an algorithm for distributed optimization with quantized updates that preserves global linear convergence of gradient descent. Finally, in our sixth and seventh challenges, we develop new VR mechanisms applicable to the non-smooth setting based on proximal operators and matrix splitting. In all cases, our theory is simpler, tighter and uses fewer assumptions than the prior literature. We accompany each chapter with numerical experiments to show the tightness of the proposed theoretical results.
Study on the Effect of Size on InGaN Red Micro-LEDs(Research Square Platform LLC, 2021-10-13) [Preprint]In this research, five sizes (100⊆100, 75⊆75, 50⊆50, 25⊆25, 10⊆10 µm2) of InGaN red micro-light emitting diode (LED) dies are produced using laser-based direct writing and maskless technology. It is observed that with increasing injection current, the smaller the size of the micro-LED, the more obvious the blue shift of the emission wavelength. When the injection current is increased from 0.1 to 1 mA, the emission wavelength of the 10×10 µm2 micro-LED is shifted from 617.15 to 576.87 nm. The obvious blue shift is attributed to the stress release and high current density injection. Moreover, the output power density is very similar for smaller chip micro-LEDs at the same injection current density. This behavior is different from AlGaInP micro-LEDs. The sidewall defect is more easily repaired by passivation, which is similar to the behavior of blue micro-LEDs. The results indicate that the red InGaN epilayer structure provides an opportunity to realize the full color LEDs fabricated by GaN-based LEDs.
Extending the natural adaptive capacity of coral holobionts(Nature Reviews Earth & Environment, Springer Science and Business Media LLC, 2021-10-12) [Article]Anthropogenic climate change and environmental degradation destroy coral reefs, the ecosystem services they provide, and the livelihoods of close to a billion people who depend on these services. Restoration approaches to increase the resilience of corals are therefore necessary to counter environmental pressures relevant to climate change projections. In this Review, we examine the natural processes that can increase the adaptive capacity of coral holobionts, with the aim of preserving ecosystem functioning under future ocean conditions. Current approaches that centre around restoring reef cover can be integrated with emerging approaches to enhance coral stress resilience and, thereby, allow reefs to regrow under a new set of environmental conditions. Emerging approaches such as standardized acute thermal stress assays, selective sexual propagation, coral probiotics, and environmental hardening could be feasible and scalable in the real world. However, they must follow decision-making criteria that consider the different reef, environmental, and ecological conditions. The implementation of adaptive interventions tailored around nature-based solutions will require standardized frameworks, appropriate ecological risk–benefit assessments, and analytical routines for consistent and effective utilization and global coordination.
Relation-aware Video Reading Comprehension for Temporal Language Grounding(arXiv, 2021-10-12) [Preprint]Temporal language grounding in videos aims to localize the temporal span relevant to the given query sentence. Previous methods treat it either as a boundary regression task or a span extraction task. This paper will formulate temporal language grounding into video reading comprehension and propose a Relation-aware Network (RaNet) to address it. This framework aims to select a video moment choice from the predefined answer set with the aid of coarse-and-fine choice-query interaction and choice-choice relation construction. A choice-query interactor is proposed to match the visual and textual information simultaneously in sentence-moment and token-moment levels, leading to a coarse-and-fine cross-modal interaction. Moreover, a novel multi-choice relation constructor is introduced by leveraging graph convolution to capture the dependencies among video moment choices for the best choice selection. Extensive experiments on ActivityNet-Captions, TACoS, and Charades-STA demonstrate the effectiveness of our solution. Codes will be released soon.
Nuclear magnetic resonance (NMR) studies of sintering effects on the lithium ion dynamics in Li1.5Al0.5Ti1.5(PO4)3(Zeitschrift für Physikalische Chemie, Walter de Gruyter GmbH, 2021-10-11) [Article]Various nuclear magnetic resonance (NMR) methods are combined to study the structure and dynamics of Li1.5Al0.5Ti1.5(PO4)3 (LATP) samples, which were obtained from sintering at various temperatures between 650 and 900 °C. 6Li, 27Al, and 31P magic angle spinning (MAS) NMR spectra show that LATP crystallites are better defined for higher calcination temperatures. Analysis of 7Li spin-lattice relaxation and line-shape changes indicates the existence of two species of lithium ions with clearly distinguishable jump dynamics, which can be attributed to crystalline and amorphous sample regions, respectively. An increase of the sintering temperature leads to higher fractions of the fast lithium species with respect to the slow one, but hardly affects the jump dynamics in either of the phases. Specifically, the fast and slow lithium ions show jumps in the nanoseconds regime near 300 and 700 K, respectively. The activation energy of the hopping motion in the LATP crystallites amounts to ca. 0.26 eV. 7Li field-gradient diffusometry reveals that the long-range ion migration is limited by the sample regions featuring slow transport. The high spatial resolution available from the high static field gradients of our setup allows the observation of the lithium ion diffusion inside the small (<100 nm) LATP crystallites, yielding a high self-diffusion coefficient of D = 2 × 10−12 m2/s at room temperature.