Recent Submissions

  • Controlling wave fronts with tunable disordered non-Hermitian multilayers.

    Novitsky, Denis V; Lyakhov, Dmitry; Michels, Dominik L.; Redka, Dmitrii; Pavlov, Alexander A; Shalin, Alexander S (Scientific reports, Springer Science and Business Media LLC, 2021-02-27) [Article]
    Unique and flexible properties of non-Hermitian photonic systems attract ever-increasing attention via delivering a whole bunch of novel optical effects and allowing for efficient tuning light-matter interactions on nano- and microscales. Together with an increasing demand for the fast and spatially compact methods of light governing, this peculiar approach paves a broad avenue to novel optical applications. Here, unifying the approaches of disordered metamaterials and non-Hermitian photonics, we propose a conceptually new and simple architecture driven by disordered loss-gain multilayers and, therefore, providing a powerful tool to control both the passage time and the wave-front shape of incident light with different switching times. For the first time we show the possibility to switch on and off kink formation by changing the level of disorder in the case of adiabatically raising wave fronts. At the same time, we deliver flexible tuning of the output intensity by using the nonlinear effect of loss and gain saturation. Since the disorder strength in our system can be conveniently controlled with the power of the external pump, our approach can be considered as a basis for different active photonic devices.
  • AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation

    Li, Bing; Zhu, Yuanlue; Wang, Yitong; Lin, Chia-Wen; Ghanem, Bernard; Shen, Linlin (arXiv, 2021-02-24) [Preprint]
    In this paper, we propose a novel framework to translate a portrait photo-face into an anime appearance. Our aim is to synthesize anime-faces which are style-consistent with a given reference anime-face. However, unlike typical translation tasks, such anime-face translation is challenging due to complex variations of appearances among anime-faces. Existing methods often fail to transfer the styles of reference anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces. We propose Ani- GAN, a novel GAN-based translator that synthesizes highquality anime-faces. Specifically, a new generator architecture is proposed to simultaneously transfer color/texture styles and transform local facial shapes into anime-like counterparts based on the style of a reference anime-face, while preserving the global structure of the source photoface. We propose a double-branch discriminator to learn both domain-specific distributions and domain-shared distributions, helping generate visually pleasing anime-faces and effectively mitigate artifacts. Extensive experiments qualitatively and quantitatively demonstrate the superiority of our method over state-of-the-art methods.
  • An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints

    Salim, Adil; Condat, Laurent Pierre; Kovalev, Dmitry; Richtarik, Peter (arXiv, 2021-02-22) [Preprint]
    Optimization problems under affine constraints appear in various areas of machine learning. We consider the task of minimizing a smooth strongly convex function $F(x)$ under the affine constraint $K x = b$, with an oracle providing evaluations of the gradient of $F$ and matrix-vector multiplications by $K$ and its transpose. We provide lower bounds on the number of gradient computations and matrix-vector multiplications to achieve a given accuracy. Then we propose an accelerated primal--dual algorithm achieving these lower bounds. Our algorithm is the first optimal algorithm for this class of problems.
  • Shape-Tailored Deep Neural Networks

    Khan, Naeemullah; Sharma, Angira; Sundaramoorthi, Ganesh; Torr, Philip H. S. (arXiv, 2021-02-16) [Preprint]
    We present Shape-Tailored Deep Neural Networks (ST-DNN). ST-DNN extend convolutional networks (CNN), which aggregate data from fixed shape (square) neighborhoods, to compute descriptors defined on arbitrarily shaped regions. This is natural for segmentation, where descriptors should describe regions (e.g., of objects) that have diverse shape. We formulate these descriptors through the Poisson partial differential equation (PDE), which can be used to generalize convolution to arbitrary regions. We stack multiple PDE layers to generalize a deep CNN to arbitrary regions, and apply it to segmentation. We show that ST-DNN are covariant to translations and rotations and robust to domain deformations, natural for segmentation, which existing CNN based methods lack. ST-DNN are 3-4 orders of magnitude smaller then CNNs used for segmentation. We show that they exceed segmentation performance compared to state-of-the-art CNN-based descriptors using 2-3 orders smaller training sets on the texture segmentation problem.
  • Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimization

    Safaryan, Mher; Hanzely, Filip; Richtarik, Peter (arXiv, 2021-02-14) [Preprint]
    Large scale distributed optimization has become the default tool for the training of supervised machine learning models with a large number of parameters and training data. Recent advancements in the field provide several mechanisms for speeding up the training, including {\em compressed communication}, {\em variance reduction} and {\em acceleration}. However, none of these methods is capable of exploiting the inherently rich data-dependent smoothness structure of the local losses beyond standard smoothness constants. In this paper, we argue that when training supervised models, {\em smoothness matrices} -- information-rich generalizations of the ubiquitous smoothness constants -- can and should be exploited for further dramatic gains, both in theory and practice. In order to further alleviate the communication burden inherent in distributed optimization, we propose a novel communication sparsification strategy that can take full advantage of the smoothness matrices associated with local losses. To showcase the power of this tool, we describe how our sparsification technique can be adapted to three distributed optimization algorithms -- DCGD, DIANA and ADIANA -- yielding significant savings in terms of communication complexity. The new methods always outperform the baselines, often dramatically so.
  • Distributed Second Order Methods with Fast Rates and Compressed Communication

    Islamov, Rustem; Qian, Xun; Richtarik, Peter (arXiv, 2021-02-14) [Preprint]
    We develop several new communication-efficient second-order methods for distributed optimization. Our first method, NEWTON-STAR, is a variant of Newton's method from which it inherits its fast local quadratic rate. However, unlike Newton's method, NEWTON-STAR enjoys the same per iteration communication cost as gradient descent. While this method is impractical as it relies on the use of certain unknown parameters characterizing the Hessian of the objective function at the optimum, it serves as the starting point which enables us design practical variants thereof with strong theoretical guarantees. In particular, we design a stochastic sparsification strategy for learning the unknown parameters in an iterative fashion in a communication efficient manner. Applying this strategy to NEWTON-STAR leads to our next method, NEWTON-LEARN, for which we prove local linear and superlinear rates independent of the condition number. When applicable, this method can have dramatically superior convergence behavior when compared to state-of-the-art methods. Finally, we develop a globalization strategy using cubic regularization which leads to our next method, CUBIC-NEWTON-LEARN, for which we prove global sublinear and linear convergence rates, and a fast superlinear rate. Our results are supported with experimental results on real datasets, and show several orders of magnitude improvement on baseline and state-of-the-art methods in terms of communication complexity.
  • Manhattan Room Layout Reconstruction from a Single 360 ∘ Image: A Comparative Study of State-of-the-Art Methods

    Zou, Chuhang; Su, Jheng Wei; Peng, Chi Han; Colburn, Alex; Shan, Qi; Wonka, Peter; Chu, Hung Kuo; Hoiem, Derek (International Journal of Computer Vision, Springer Science and Business Media LLC, 2021-02-09) [Article]
    Recent approaches for predicting layouts from 360∘ panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a pre-processing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g., SegNet or ResNet), type of elements predicted (e.g., corners, wall/floor boundaries, or semantic segmentation), or method of fitting the 3D layout. To address this challenge, we summarize and describe the common framework, the variants, and the impact of the design decisions. For a complete evaluation, we also propose extended annotations for the Matterport3D dataset (Chang et al.: Matterport3d: learning from rgb-d data in indoor environments. arXiv:1709.06158, 2017), and introduce two depth-based evaluation metrics.
  • A Visual Analytics Based Decision Making Environment for COVID-19 Modeling and Visualization

    Afzal, Shehzad; Ghani, Sohaib; Jenkins-Smith, Hank C.; Ebert, David S.; Hadwiger, Markus; Hoteit, Ibrahim (IEEE, 2021-02-01) [Conference Paper]
    Public health officials dealing with pandemics like COVID-19 have to evaluate and prepare response plans. This planning phase requires not only looking into the spatiotemporal dynamics and impact of the pandemic using simulation models, but they also need to plan and ensure the availability of resources under different spread scenarios. To this end, we have developed a visual analytics environment that enables public health officials to model, simulate, and explore the spread of COVID-19 by supplying county-level information such as population, demographics, and hospital beds. This environment facilitates users to explore spatiotemporal model simulation data relevant to COVID-19 through a geospatial map with linked statistical views, apply different decision measures at different points in time, and understand their potential impact. Users can drill-down to county-level details such as the number of sicknesses, deaths, needs for hospitalization, and variations in these statistics over time. We demonstrate the usefulness of this environment through a use case study and also provide feedback from domain experts. We also provide details about future extensions and potential applications of this work.
  • Efficient exponential time integration for simulating nonlinear coupled oscillators

    Luan, Vu Thai; Michels, Dominik L. (Journal of Computational and Applied Mathematics, Elsevier BV, 2021-01-27) [Article]
    In this paper, we propose an advanced time integration technique associated with explicit exponential Rosenbrock-based methods for the simulation of large stiff systems of nonlinear coupled oscillators. In particular, a novel reformulation of these systems is introduced and a general family of efficient exponential Rosenbrock schemes for simulating the reformulated system is derived. Moreover, we show the required regularity conditions and prove the convergence of these schemes for the system of coupled oscillators. We present an efficient implementation of this new approach and discuss several applications in scientific and visual computing. The accuracy and efficiency of our approach are demonstrated through a broad spectrum of numerical examples, including a nonlinear Fermi–Pasta–Ulam–Tsingou model, elastic and nonelastic deformations as well as collision scenarios focusing on relevant aspects such as stability and energy conservation, large numerical stiffness, high fidelity, and visual accuracy.
  • Lost photon enhances superresolution

    Mikhalychev, A.; Novik, P.; Karuseichyk, I.; Lyakhov, D. A.; Michels, Dominik L.; Mogilevtsev, D. (arXiv, 2021-01-14) [Preprint]
    Quantum imaging can beat classical resolution limits, imposed by diffraction of light. In particular, it is known that one can reduce the image blurring and increase the achievable resolution by illuminating an object by entangled light and measuring coincidences of photons. If an $n$-photon entangled state is used and the $n$th-order correlation function is measured, the point-spread function (PSF) effectively becomes $\sqrt n$ times narrower relatively to classical coherent imaging. Quite surprisingly, measuring $n$-photon correlations is not the best choice if an $n$-photon entangled state is available. We show that for measuring $(n-1)$-photon coincidences (thus, ignoring one of the available photons), PSF can be made even narrower. This observation paves a way for a strong conditional resolution enhancement by registering one of the photons outside the imaging area. We analyze the conditions necessary for the resolution increase and propose a practical scheme, suitable for observation and exploitation of the effect.
  • MAAS: Multi-modal Assignation for Active Speaker Detection

    León-Alcázar, Juan; Heilbron, Fabian Caba; Thabet, Ali Kassem; Ghanem, Bernard (arXiv, 2021-01-11) [Preprint]
    Active speaker detection requires a solid integration of multi-modal cues. While individual modalities can approximate a solution, accurate predictions can only be achieved by explicitly fusing the audio and visual features and modeling their temporal progression. Despite its inherent muti-modal nature, current methods still focus on modeling and fusing short-term audiovisual features for individual speakers, often at frame level. In this paper we present a novel approach to active speaker detection that directly addresses the multi-modal nature of the problem, and provides a straightforward strategy where independent visual features from potential speakers in the scene are assigned to a previously detected speech event. Our experiments show that, an small graph data structure built from a single frame, allows to approximate an instantaneous audio-visual assignment problem. Moreover, the temporal extension of this initial graph achieves a new state-of-the-art on the AVA-ActiveSpeaker dataset with a mAP of 88.8\%.
  • The influence of the synthesis conditions on the magnetic behaviour of the densely packed arrays of Ni nanowires in porous anodic alumina membranes

    Vorobjova, Alla; Tishkevich, Daria; Shimanovich, Dmitriy; Zubar, Tatiana; Astapovich, Ksenia; Kozlovskiy, Artem; Zdorovets, Maxim; Zhaludkevich, Aliaksandr; Lyakhov, Dmitry; Michels, Dominik L.; Vinnik, Denis; Fedosyuk, Valery; Trukhanov, Alex (RSC Advances, Royal Society of Chemistry (RSC), 2021) [Article]
    The densely packed arrays of Ni nanowires of 70 nm diameter and 6–12 μm length were obtained $\textit{via}$ electrodeposition into porous alumina membranes (PAAMs) of 55–75 μm thickness.
  • Towards an End-to-End Analysis and Prediction System for Weather, Climate, and Marine Applications in the Red Sea

    Hoteit, Ibrahim; Abualnaja, Yasser; Afzal, Shehzad; Ait-El-Fquih, Boujemaa; Akylas, Triantaphyllos; Antony, Charls; Dawson, Clint; Asfahani, Khaled; Brewin, Robert J.; Cavaleri, Luigi; Cerovecki, Ivana; Cornuelle, Bruce; Desamsetti, Srinivas; Attada, Raju; Dasari, Hari Prasad; Sanchez-Garrido, Jose; Genevier, Lily; El Gharamti, Mohamad; Gittings, John; Gokul, Elamurugu; Gopalakrishnan, Ganesh; Guo, Daquan; Hadri, Bilel; Hadwiger, Markus; Hammoud, Mohammed Abed; Hendershott, Myrl; Hittawe, Mohamad; Karumuri, Ashok; Knio, Omar; Köhl, Armin; Kortas, Samuel; Krokos, Georgios; Kunchala, Ravi; Issa, Leila; Lakkis, Issam; Langodan, Sabique; Lermusiaux, Pierre; Luong, Thang; Ma, Jingyi; Le Maitre, Olivier; Mazloff, Matthew; El Mohtar, Samah; Papadopoulos, Vassilis P.; Platt, Trevor; Pratt, Larry; Raboudi, Naila Mohammed Fathi; Racault, Marie-Fanny; Raitsos, Dionysios E.; Razak, Shanas; Sanikommu, Siva Reddy; Sathyendranath, Shubha; Sofianos, Sarantis; Subramanian, Aneesh; Sun, Rui; Titi, Edriss; Toye, Habib; Triantafyllou, George; Tsiaras, Kostas; Vasou, Panagiotis; Viswanadhapalli, Yesubabu; Wang, Yixin; Yao, Fengchao; Zhan, Peng; Zodiatis, George (Bulletin of the American Meteorological Society, American Meteorological Society, 2021-01) [Article]
    AbstractThe Red Sea, home to the second-longest coral reef system in the world, is a vital resource for the Kingdom of Saudi Arabia. The Red Sea provides 90% of the Kingdom’s potable water by desalinization, supporting tourism, shipping, aquaculture, and fishing industries, which together contribute about 10%–20% of the country’s GDP. All these activities, and those elsewhere in the Red Sea region, critically depend on oceanic and atmospheric conditions. At a time of mega-development projects along the Red Sea coast, and global warming, authorities are working on optimizing the harnessing of environmental resources, including renewable energy and rainwater harvesting. All these require high-resolution weather and climate information. Toward this end, we have undertaken a multipronged research and development activity in which we are developing an integrated data-driven regional coupled modeling system. The telescopically nested components include 5-km- to 600-m-resolution atmospheric models to address weather and climate challenges, 4-km- to 50-m-resolution ocean models with regional and coastal configurations to simulate and predict the general and mesoscale circulation, 4-km- to 100-m-resolution ecosystem models to simulate the biogeochemistry, and 1-km- to 50-m-resolution wave models. In addition, a complementary probabilistic transport modeling system predicts dispersion of contaminant plumes, oil spill, and marine ecosystem connectivity. Advanced ensemble data assimilation capabilities have also been implemented for accurate forecasting. Resulting achievements include significant advancement in our understanding of the regional circulation and its connection to the global climate, development, and validation of long-term Red Sea regional atmospheric–oceanic–wave reanalyses and forecasting capacities. These products are being extensively used by academia, government, and industry in various weather and marine studies and operations, environmental policies, renewable energy applications, impact assessment, flood forecasting, and more.
  • SALA: Soft Assignment Local Aggregation for 3D Semantic Segmentation

    Itani, Hani; Giancola, Silvio; Thabet, Ali Kassem; Ghanem, Bernard (arXiv, 2020-12-29) [Preprint]
    We introduce the idea of using learnable neighbor-togrid soft assignment in grid-based aggregation functions for the task of 3D semantic segmentation. Previous methods in literature operate on a predefined geometric grid such as local volume partitions or irregular kernel points. These methods use geometric functions to assign local neighbors to their corresponding grid. Such geometric heuristics are potentially sub-optimal for the end task of semantic segmentation. Furthermore, they are applied uniformly throughout the depth of the network. A more general alternative would allow the network to learn its own neighbor-to-grid assignment function that best suits the end task. Since it is learnable, this mapping has the flexibility to be different per layer. This paper leverages learned neighbor-to-grid soft assignment to define an aggregation function that balances efficiency and performance. We demonstrate the efficacy of our method by reaching state-of-the-art (SOTA) performance on S3DIS with almost 10× less parameters than the current reigning method. We also demonstrate competitive performance on ScanNet and PartNet as compared with much larger SOTA models.
  • DFT Study of NO Reduction Process on Ag/γ-Al2O3 Catalyst: Some Aspects of Mechanism and Catalyst Structure

    Matulis, Vitaly E.; Ragoyja, Ekaterina G.; Ivashkevich, Oleg A.; Lyakhov, Dmitry; Michels, Dominik L. (The Journal of Physical Chemistry C, American Chemical Society (ACS), 2020-12-23) [Article]
    Catalysts based on Ag/γ-Al2O3 are perspective systems for practical implementation of catalytic NO reduction. Nevertheless, the mechanism and regularities of this process have still not been fully investigated. Herein, we present the results of quantum-chemical research of the Ag/γ-Al2O3 catalyst surface and some aspects of the NO reduction mechanism on it. Proposed calculation methods using DFT and cluster models of the catalyst surface are compared and verified. The possibility of existence of small adsorbed neutral and cationic silver clusters on the surface of the catalyst is shown. It is demonstrated that NO adsorption on these clusters is energetically favorable, in the form of both monomers and dimers. The scheme of NO selective catalytic reduction (SCR) that explains increasing of N2O side-product amount on catalysts with silver fraction more than 2 wt % is proposed. The feasibility of this scheme is justified with calculated data. Some recommendations that allow decreasing amounts of N2O are developed.
  • On the N-Arylation of Acetamide Using 2-, 3- and 1’-Substituted Iodoferrocenes**

    Kadari, Lingaswamy; Erb, William; Halauko, Yury S.; Ivashkevich, Oleg A.; Matulis, Vadim E.; Lyakhov, Dmitry; Roisnel, Thierry; Radha Krishna, Palakodety; Mongin, Florence (European Journal of Inorganic Chemistry, Wiley, 2020-12-22) [Article]
    Various 2-, 3- and 1’-substituted iodoferrocenes were reacted with acetamide in the presence of copper(I) iodide (1 equiv), N,N’-dimethylethylenediamine (1 equiv), tripotassium phosphate (2 equiv) in dioxane at 90 °C for 14 h, and allowed a large range of original 1,2-, 1,3- and 1,1’-disubstituted ferrocenes to be obtained. The results were compared as a function of the substituent and its position on the ring. DFT calculations revealed higher activation barrier for the oxidative addition in the ferrocene series when compared with classical planar aromatics. Structure-property relationships were applied to rationalize the reactivity of the different iodoferrocenes.
  • Upgrading the Gemini Planet Imager calibration unit with a photon counting focal plane wavefront sensor

    Marois, Christian; Gerard, B.; Lardière, O.; Anthony, A.; Bradley, C.; Dunn, J.; Fu, Qiang; Hardy, T.; Heidrich, Wolfgang; Herriot, G.; Nielsen, E.; Sivanandam, S.; Sivransky, D.; Thibault, S.; Thompson, W.; Véran, J. P. (SPIE, 2020-12-14) [Conference Paper, Poster]
    High-contrast imaging instruments have advanced techniques to improve contrast, but they remain limited by uncorrected stellar speckles, often lacking a "second stage"correction to complement the Adaptive Optics (AO) correction. We are implementing a new second stage speckle-correction solution for the Gemini Planet Imager (GPI), replacing the instrument calibration unit (CAL) with the Fast Atmospheric Self coherent camera Technique (FAST), a new version of the self-coherent camera (SCC) concept. Our proposed upgrade (CAL2.0) will use a common-path interferometer design to enable speckle correction, through post-processing and/or by a feedback loop to the AO deformable mirror. FAST utilizes a new type of coronagraphic mask that will enable, for the first time, speckle correction down to millisecond timescales. The system's main goal is to improve the contrast by up to 100x in a halfdark hole to enable a new regime of science discoveries. Our team has been developing this new technology at the NRC's Extreme Wavefront control for Exoplanet and Adaptive optics Research Topics (NEW EARTH) laboratory over the past several years. The GPI CAL2.0 update is funded (November 2020), and the system's first light is expected late 2023.
  • Improved StyleGAN Embedding: Where are the Good Latents?

    Zhu, Peihao; Abdal, Rameen; Qin, Yipeng; Wonka, Peter (arXiv, 2020-12-13) [Preprint]
    StyleGAN is able to produce photorealistic images almost indistinguishable from real ones. Embedding images into the StyleGAN latent space is not a trivial task due to the reconstruction quality and editing quality trade-off. In this paper, we first introduce a new normalized space to analyze the diversity and the quality of the reconstructed latent codes. This space can help answer the question of where good latent codes are located in latent space. Second, we propose a framework to analyze the quality of different embedding algorithms. Third, we propose an improved embedding algorithm based on our analysis. We compare our results with the current state-of-the-art methods and achieve a better trade-off between reconstruction quality and editing quality.
  • Optical design and preliminary results of NEW EARTH, first Canadian high-contrast imaging laboratory test bench

    Lardière, Olivier; Gerard, Benjamin; Thompson, William; Marois, Christian; Véran, Jean Pierre; Blain, Célia; Heidrich, Wolfgang; Fu, Qiang (SPIE, 2020-12-13) [Conference Paper]
    The NEW EARTH Laboratory (NRC Extreme Wavefront control for Exoplanet Adaptive optics Research Topics at Herzberg) has recently been completed at NRC in Victoria. NEW EARTH is the first Canadian test-bed dedicated to high-contrast imaging. The bench optical design allows a wide range of applications that could require turbulent phase screens, segmented pupils, or custom coronagraphic masks. Super-polished off-axis parabolas are implemented to minimize optical aberrations, in addition to a 468-actuator ALPAO deformable mirror and a Shack Hartmann WFS. The laboratory's immediate goal is to validate the Fast Atmospheric Self-coherent camera Technique (FAST). The first results of this technique obtained in the NEW EARTH laboratory with a Tilt-Gaussian-Vortex focal plane mask, a reflective Lyot stop and Coherent Differential Imaging are encouraging. Future work will be aimed at expanding this technique to broader wavebands in the context of extremely large telescopes and at visible bands for space-based observatories.
  • Spatial and Hyperfine Characteristics of SiV– and SiV0 Color Centers in Diamond: DFT Simulation

    Nizovtsev, A. P.; Kilin, S. Ya; Pushkarchuk, A. L.; Kuten, S. A.; Poklonski, N. A.; Michels, Dominik L.; Lyakhov, Dmitry; Jelezko, F. (Semiconductors, Pleiades Publishing Ltd, 2020-12-04) [Article]
    Abstract: One of the most promising platforms to implement quantum technologies are coupled electron-nuclear spins in diamond in which the electrons of paramagnetic color centers play a role of “fast” qubits, while nuclear spins of nearby 13C atoms can store quantum information for a very long time due to their exceptionally high isolation from the environment. Essential prerequisite for a high-fidelity spin manipulation in these systems with tailored control pulse sequences is a complete knowledge of hyperfine interactions. Development of this understanding for the negatively charged “silicon-vacancy” (SiV–) and neutral (SiV0) color center, is a primary goal of this article, where we are presenting shortly our recent results of computer simulation of spatial and hyperfine characteristics of these SiV centers in H-terminated cluster C128[SiV]H98 along with their comparison with available experimental data.

View more