Recent Submissions

  • Molecumentary: Scalable Narrated Documentaries Using Molecular Visualization

    Kouřil, David; Strnad, Ondrej; Mindek, Peter; Halladjian, Sarkis; Isenberg, Tobias; Gröller, M. Eduard; Viola, Ivan (arXiv, 2020-11-04) [Preprint]
    We present a method for producing documentary-style content using real-time scientific visualization. We produce molecumentaries, i.e., molecular documentaries featuring structural models from molecular biology. We employ scalable methods instead of the rigid traditional production pipeline. Our method is motivated by the rapid evolution of interactive scientific visualization, which shows great potential in science dissemination. Without some form of explanation or guidance, however, novices and lay-persons often find it difficult to gain insights from the visualization itself. We integrate such knowledge using the verbal channel and provide it along an engaging visual presentation. To realize the synthesis of a molecumentary, we provide technical solutions along two major production steps: 1) preparing a story structure and 2) turning the story into a concrete narrative. In the first step, information about the model from heterogeneous sources is compiled into a story graph. Local knowledge is combined with remote sources to complete the story graph and enrich the final result. In the second step, a narrative, i.e., story elements presented in sequence, is synthesized using the story graph. We present a method for traversing the story graph and generating a virtual tour, using automated camera and visualization transitions. Texts written by domain experts are turned into verbal representations using text-to-speech functionality and provided as a commentary. Using the described framework we synthesize automatic fly-throughs with descriptions that mimic a manually authored documentary. Furthermore, we demonstrate a second scenario: guiding the documentary narrative by a textual input.
  • In Silico Design of Deep Space Optical Links

    Lee, Carlyn-Ann; Xie, Hua; Lee, Charles H.; Lyakhov, Dmitry; Michels, Dominik L. (American Institute of Aeronautics and Astronautics, 2020-11-02) [Conference Paper]
    As deep space links migrate toward higher frequency bands like Ka and optical, thorough trade-space exploration becomes increasingly valuable for designing reliable and efficient communications systems. In this contribution, we leveraged high-performance, concurrent simulations when the run-time complexity of simulation software overwhelms capabilities of ordinary desktop machines. The first part of this manuscript describes how to run error correcting code simulations concurrently on a high-performance supercomputer. The second part of this study describes a framework to produce azimuth and elevation terrain masks from imagery of the Lunar South Pole.
  • Class-Agnostic Segmentation Loss and Its Application to Salient Object Detection and Segmentation

    Sharma, Angira; Khan, Naeemullah; Sundaramoorthi, Ganesh; Torr, Philip (arXiv, 2020-10-28) [Preprint]
    In this paper we present a novel loss function, called class-agnostic segmentation (CAS) loss. With CAS loss the class descriptors are learned during training of the network. We don't require to define the label of a class a-priori, rather the CAS loss clusters regions with similar appearance together in a weakly-supervised manner. Furthermore, we show that the CAS loss function is sparse, bounded, and robust to class-imbalance. We apply our CAS loss function with fully-convolutional ResNet101 and DeepLab-v3 architectures to the binary segmentation problem of salient object detection. We investigate the performance against the state-of-the-art methods in two settings of low and high-fidelity training data on seven salient object detection datasets. For low-fidelity training data (incorrect class label) class-agnostic segmentation loss outperforms the state-of-the-art methods on salient object detection datasets by staggering margins of around 50%. For high-fidelity training data (correct class labels) class-agnostic segmentation models perform as good as the state-of-the-art approaches while beating the state-of-the-art methods on most datasets. In order to show the utility of the loss function across different domains we also test on general segmentation dataset, where class-agnostic segmentation loss outperforms cross-entropy based loss by huge margins on both region and edge metrics.
  • Surface-Enhanced Raman Spectroscopy of Organic Molecules and Living Cells with Gold-Plated Black Silicon.

    Golubewa, Lena; Karpicz, Renata; Matulaitiene, Ieva; Selskis, Algirdas; Rutkauskas, Danielis; Pushkarchuk, Aliaksandr; Khlopina, Tatsiana; Michels, Dominik L.; Lyakhov, Dmitry; Kulahava, Tatsiana; Shah, Ali; Svirko, Yuri; Kuzhir, Polina (ACS applied materials & interfaces, American Chemical Society (ACS), 2020-10-27) [Article]
    Black silicon (bSi) refers to an etched silicon surface comprising arrays of microcones that effectively suppress reflection from UV to near-infrared (NIR) while simultaneously enhancing the scattering and absorption of light. This makes bSi covered with a nm-thin layer of plasmonic metal, i.e., gold, an attractive substrate material for sensing of bio-macromolecules and living cells using surface-enhanced Raman spectroscopy (SERS). The performed Raman measurements accompanied with finite element numerical simulation and density functional theory analysis revealed that at the 785 nm excitation wavelength, the SERS enhancement factor of the bSi/Au substrate is as high as 108 due to a combination of electromagnetic and chemical mechanisms. This finding makes the SERS-active bSi/Au substrate suitable for detecting trace amounts of organic molecules. We demonstrate the outstanding performance of this substrate by highly sensitive and specific detection of a small organic molecule of 4-mercaptobenzoic acid and living C6 rat glioma cell nucleic acids/proteins/lipids. Specifically, the bSi/Au SERS-active substrate offers a unique opportunity to investigate the living cells' malignant transformation using characteristic protein disulfide Raman bands as a marker. Our findings evidence that bSi/Au provides a pathway to the highly sensitive and selective, scalable, and low-cost substrate for lab-on-a-chip SERS biosensors that can be integrated into silicon-based photonics devices.
  • A Visual Analytics Based Decision Making Environment for COVID-19 Modeling and Visualization

    Afzal, Shehzad; Ghani, Sohaib; Jenkins-Smith, Hank C.; Ebert, David S.; Hadwiger, Markus; Hoteit, Ibrahim (arXiv, 2020-10-22) [Preprint]
    Public health officials dealing with pandemics like COVID-19 have to evaluate and prepare response plans. This planning phase requires not only looking into the spatiotemporal dynamics and impact of the pandemic using simulation models, but they also need to plan and ensure the availability of resources under different spread scenarios. To this end, we have developed a visual analytics environment that enables public health officials to model, simulate, and explore the spread of COVID-19 by supplying county-level information such as population, demographics, and hospital beds. This environment facilitates users to explore spatiotemporal model simulation data relevant to COVID-19 through a geospatial map with linked statistical views, apply different decision measures at different points in time, and understand their potential impact. Users can drill-down to county-level details such as the number of sicknesses, deaths, needs for hospitalization, and variations in these statistics over time. We demonstrate the usefulness of this environment through a use case study and also provide feedback from domain experts. We also provide details about future extensions and potential applications of this work.
  • Controlling wave-front shape and propagation time with tunable disordered non-Hermitian multilayers

    Novitsky, Denis; Lyakhov, Dmitry; Michels, Dominik L.; Redka, Dmitrii; Pavlov, Alexander; Shalin, Alexander (arXiv, 2020-10-19) [Preprint]
    Unique and flexible properties of non-Hermitian photonic systems attract ever-increasing attention via delivering a whole bunch of novel optical effects and allowing for efficient tuning light-matter interactions on nano- and microscales. Together with an increasing demand for the fast and spatially compact methods of light governing, this peculiar approach paves a broad avenue to novel optical applications. Here, unifying the approaches of disordered metamaterials and non-Hermitian photonics, we propose a conceptually new and simple architecture driven by disordered loss-gain multilayers and, therefore, providing a powerful tool to control both the passage time and the wave-front shape of incident light with different switching times. For the first time we show the possibility to switch on and off kink formation by changing the level of disorder in the case of adiabatically raising wave fronts. At the same time, we deliver flexible tuning of the output intensity by using the nonlinear effect of loss and gain saturation. Since the disorder strength in our system can be conveniently controlled with the power of the external pump, our approach can be considered as a basis for different active photonic devices.
  • Characterizing envelopes of moving rotational cones and applications in CNC machining

    Skopenkov, Mikhail; Bo, Pengbo; Bartoň, Michael; Pottmann, Helmut (Elsevier, 2020-10-16) [Article]
    Motivated by applications in CNC machining, we provide a characterization of surfaces which are enveloped by a one-parametric family of congruent rotational cones. As limit cases, we also address developable surfaces and ruled surfaces. The characterizations are higher order nonlinear PDEs generalizing the ones by Gauss and Monge for developable surfaces and ruled surfaces, respectively. The derivation includes results on local approximations of a surface by cones of revolution, which are expressed by contact order in the space of planes. These results are themselves of interest in geometric computing, for example in cutter selection and positioning for flank CNC machining.
  • How Does Lipschitz Regularization Influence GAN Training?

    Qin, Yipeng; Mitra, Niloy; Wonka, Peter (Springer International Publishing, 2020-10-10) [Conference Paper]
    Despite the success of Lipschitz regularization in stabilizing GAN training, the exact reason of its effectiveness remains poorly understood. The direct effect of K-Lipschitz regularization is to restrict the L2-norm of the neural network gradient to be smaller than a threshold K (e.g.,) such that. In this work, we uncover an even more important effect of Lipschitz regularization by examining its impact on the loss function: It degenerates GAN loss functions to almost linear ones by restricting their domain and interval of attainable gradient values. Our analysis shows that loss functions are only successful if they are degenerated to almost linear ones. We also show that loss functions perform poorly if they are not degenerated and that a wide range of functions can be used as loss function as long as they are sufficiently degenerated by regularization. Basically, Lipschitz regularization ensures that all loss functions effectively work in the same way. Empirically, we verify our proposition on the MNIST, CIFAR10 and CelebA datasets.
  • Stereo Event-Based Particle Tracking Velocimetry for 3D Fluid Flow Reconstruction

    Wang, Yuanhao; Idoughi, Ramzi; Heidrich, Wolfgang (Springer International Publishing, 2020-10-07) [Conference Paper]
    Existing Particle Imaging Velocimetry techniques require the use of high-speed cameras to reconstruct time-resolved fluid flows. These cameras provide high-resolution images at high frame rates, which generates bandwidth and memory issues. By capturing only changes in the brightness with a very low latency and at low data rate, event-based cameras have the ability to tackle such issues. In this paper, we present a new framework that retrieves dense 3D measurements of the fluid velocity field using a pair of event-based cameras. First, we track particles inside the two event sequences in order to estimate their 2D velocity in the two sequences of images. A stereo-matching step is then performed to retrieve their 3D positions. These intermediate outputs are incorporated into an optimization framework that also includes physically plausible regularizers, in order to retrieve the 3D velocity field. Extensive experiments on both simulated and real data demonstrate the efficacy of our approach.
  • AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds

    Hamdi, Abdullah; Rojas, Sara; Thabet, Ali Kassem; Ghanem, Bernard (Springer International Publishing, 2020-10-07) [Conference Paper]
    Deep neural networks are vulnerable to adversarial attacks, in which imperceptible perturbations to their input lead to erroneous network predictions. This phenomenon has been extensively studied in the image domain, and has only recently been extended to 3D point clouds. In this work, we present novel data-driven adversarial attacks against 3D point cloud networks. We aim to address the following problems in current 3D point cloud adversarial attacks: they do not transfer well between different networks, and they are easy to defend against via simple statistical methods. To this extent, we develop a new point cloud attack (dubbed AdvPC) that exploits the input data distribution by adding an adversarial loss, after Auto-Encoder reconstruction, to the objective it optimizes. AdvPC leads to perturbations that are resilient against current defenses, while remaining highly transferable compared to state-of-the-art attacks. We test AdvPC using four popular point cloud networks: PointNet, PointNet++ (MSG and SSG), and DGCNN. Our proposed attack increases the attack success rate by up to 40% for those transferred to unseen networks (transferability), while maintaining a high success rate on the attacked network. AdvPC also increases the ability to break defenses by up to 38% as compared to other baselines on the ModelNet40 dataset. The code is available at
  • Multiclass Dictionary-Based Statistical Iterative Reconstruction for Low-Dose CT

    KAMOSHITA, Hiryu; Kitahara, Daichi; FUJIMOTO, Ken'ichi; Condat, Laurent; Hirabayashi, Akira (IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Institute of Electronics, Information and Communications Engineers (IEICE), 2020-10-05) [Article]
    This paper proposes a high-quality computed tomography (CT) image reconstruction method from low-dose X-ray projection data. A state-of-the-art method, proposed by Xu et al., exploits dictionary learning for image patches. This method generates an overcomplete dictionary from patches of standard-dose CT images and reconstructs low-dose CT images by minimizing the sum of a data fidelity and a regularization term based on sparse representations with the dictionary. However, this method does not take characteristics of each patch, such as textures or edges, into account. In this paper, we propose to classify all patches into several classes and utilize an individual dictionary with an individual regularization parameter for each class. Furthermore, for fast computation, we introduce the orthogonality to column vectors of each dictionary. Since similar patches are collected in the same cluster, accuracy degradation by the orthogonality hardly occurs. Our simulations show that the proposed method outperforms the state-of-the-art in terms of both accuracy and speed.
  • Error Compensated Distributed SGD Can Be Accelerated

    Qian, Xun; Richtarik, Peter; Zhang, Tong (arXiv, 2020-09-30) [Preprint]
    Gradient compression is a recent and increasingly popular technique for reducing the communication cost in distributed training of large-scale machine learning models. In this work we focus on developing efficient distributed methods that can work for any compressor satisfying a certain contraction property, which includes both unbiased (after appropriate scaling) and biased compressors such as RandK and TopK. Applied naively, gradient compression introduces errors that either slow down convergence or lead to divergence. A popular technique designed to tackle this issue is error compensation/error feedback. Due to the difficulties associated with analyzing biased compressors, it is not known whether gradient compression with error compensation can be combined with Nesterov's acceleration. In this work, we show for the first time that error compensated gradient compression methods can be accelerated. In particular, we propose and study the error compensated loopless Katyusha method, and establish an accelerated linear convergence rate under standard assumptions. We show through numerical experiments that the proposed method converges with substantially fewer communication rounds than previous error compensated algorithms.
  • Optimal correlation order in super-resolution optical fluctuation microscopy

    Vlasenko, S.; Mikhalychev, A. B.; Karuseichyk, I. L.; Lyakhov, D. A.; Michels, Dominik L.; Mogilevtsev, D. (arXiv, 2020-09-21) [Preprint]
    Here, we show that, contrary to the common opinion, the super-resolution optical fluctuation microscopy might not lead to ideally infinite super-resolution enhancement with increasing of the order of measured cumulants. Using information analysis for estimating error bounds on the determination of point sources positions, we show that reachable precision per measurement might be saturated with increasing of the order of the measured cumulants in the super-resolution regime. In fact, there is an optimal correlation order beyond which there is practically no improvement for objects of three and more point sources. However, for objects of just two sources, one still has an intuitively expected resolution increase with the cumulant order.
  • End-to-End Hyperspectral-Depth Imaging with Learned Diffractive Optics

    Baek, Seung-Hwan; Ikoma, Hayato; Jeon, Daniel S.; Li, Yuqi; Heidrich, Wolfgang; Wetzstein, Gordon; Kim, Min H. (arXiv, 2020-09-01) [Preprint]
    To extend the capabilities of spectral imaging, hyperspectral and depth imaging have been combined to capture the higher-dimensional visual information. However, the form factor of the combined imaging systems increases, limiting the applicability of this new technology. In this work, we propose a monocular imaging system for simultaneously capturing hyperspectral-depth (HS-D) scene information with an optimized diffractive optical element (DOE). In the training phase, this DOE is optimized jointly with a convolutional neural network to estimate HS-D data from a snapshot input. To study natural image statistics of this high-dimensional visual data and to enable such a machine learning-based DOE training procedure, we record two HS-D datasets. One is used for end-to-end optimization in deep optical HS-D imaging, and the other is used for enhancing reconstruction performance with a real-DOE prototype. The optimized DOE is fabricated with a grayscale lithography process and inserted into a portable HS-D camera prototype, which is shown to robustly capture HS-D information. In extensive evaluations, we demonstrate that our deep optical imaging system achieves state-of-the-art results for HS-D imaging and that the optimized DOE outperforms alternative optical designs.
  • Computational Design of Cold Bent Glass Façades

    Gavriil, Konstantinos; Guseinov, Ruslan; PÉREZ, JESÚS; PELLIS, DAVIDE; Henderson, Paul; Rist, Florian; Pottmann, Helmut; Bickel, Bernd (ACM Transactions on Graphics, ACM, 2020-09) [Article]
    Cold bent glass is a promising and cost-efficient method for realizing doubly curved glass facades. They are produced by attaching planar glass sheets to curved frames and require keeping the occurring stress within safe limits. However, it is very challenging to navigate the design space of cold bent glass panels due to the fragility of the material, which impedes the form-finding for practically feasible and aesthetically pleasing cold bent glass facades. We propose an interactive, data-driven approach for designing cold bent glass facades that can be seamlessly integrated into a typical architectural design pipeline. Our method allows non-expert users to interactively edit a parametric surface while providing real-time feedback on the deformed shape and maximum stress of cold bent glass panels. Designs are automatically refined to minimize several fairness criteria while maximal stresses are kept within glass limits. We achieve interactive frame rates by using a differentiable Mixture Density Network trained from more than a million simulations. Given a curved boundary, our regression model is capable of handling multistable configurations and accurately predicting the equilibrium shape of the panel and its corresponding maximal stress. We show predictions are highly accurate and validate our results with a physical realization of a cold bent glass surface.
  • Rise of nations: Why do empires expand and fall?

    Vakulenko, S.; Lyakhov, Dmitry; Weber, A. G.; Lukichev, D.; Michels, Dominik L. (Chaos, AIP Publishing, 2020-09-01) [Article]
    We consider centralized networks composed of multiple satellites arranged around a few dominating super-egoistic centers. These so-called empires are organized using a divide and rule framework enforcing strong center-satellite interactions while keeping the pairwise interactions between the satellites sufficiently weak. We present a stochastic stability analysis, in which we consider these dynamical systems as stable if the centers have sufficient resources while the satellites have no value. Our model is based on a Hopfield type network that proved its significance in the field of artificial intelligence. Using this model, it is shown that the divide and rule framework provides important advantages: it allows for completely controlling the dynamics in a straight-forward way by adjusting center-satellite interactions. Moreover, it is shown that such empires should only have a single ruling center to provide sufficient stability. To survive, empires should have switching mechanisms implementing adequate behavior models by choosing appropriate local attractors in order to correctly respond to internal and external challenges. By an analogy with Bose-Einstein condensation, we show that if the noise correlations are negative for each pair of nodes, then the most stable structure with respect to noise is a globally connected network. For social systems, we show that controllability by their centers is only possible if the centers evolve slowly. Except for short periods when the state approaches a certain stable state, the development of such structures is very slow and negatively correlated with the size of the system's structure. Hence, increasing size ventually ends up in the "control trap."
  • Method of surface energy investigation by lateral AFM: application to control growth mechanism of nanostructured NiFe films.

    Zubar, T I; Fedosyuk, V M; Trukhanov, S V; Tishkevich, D I; Michels, Dominik L.; Lyakhov, Dmitry; Trukhanov, A V (Scientific reports, Springer Science and Business Media LLC, 2020-09-01) [Article]
    A new method for the specific surface energy investigation based on a combination of the force spectroscopy and the method of nanofriction study using atomic force microscopy was proposed. It was shown that air humidity does not affect the results of investigation by the proposed method as opposed to the previously used methods. Therefore, the method has high accuracy and repeatability in air without use of climate chambers and liquid cells. The proposed method has a high local resolution and is suitable for investigation of the specific surface energy of individual nanograins or fixed nanoparticles. The achievements described in the paper demonstrate one of the method capabilities, which is to control the growth mechanism of thin magnetic films. The conditions for the transition of the growth mechanism of thin Ni80Fe20 films from island to layer-by-layer obtained via electrolyte deposition have been determined using the proposed method and the purpose made probes with Ni coating.
  • On Ambarzumyan-type Inverse Problems of Vibrating String Equations

    Ashrafyan, Yuri; Michels, Dominik L. (arXiv, 2020-08-29) [Preprint]
    We consider the inverse spectral theory of vibrating string equations. In this regard, first eigenvalue Ambarzumyan-type uniqueness theorems are stated and proved subject to separated, self-adjoint boundary conditions. More precisely, it is shown that there is a curve in the boundary parameters' domain on which no analog of it is possible. Necessary conditions of the $n$-th eigenvalue are identified, which allows to state the theorems. In addition, several properties of the first eigenvalue are examined. Lower and upper bounds are identified, and the areas are described in the boundary parameters' domain on which the sign of the first eigenvalue remains unchanged. This paper contributes to inverse spectral theory as well as to direct spectral theory.
  • Robust statistical phase-diversity method for high-accuracy wavefront sensing

    Zhou, Zhisheng; Nie, Yunfeng; Fu, Qiang; Liu, Qiran; Zhang, Jingang (Optics and Lasers in Engineering, Elsevier BV, 2020-08-28) [Article]
    Phase diversity phase retrieval (PDPR) has been a popular technique for quantitatively measuring wavefront errors of optical imaging systems by extracting the phase information from several designated intensity measurements. As the problem is inverse and non-convex in general, the accuracy and robustness of most such algorithms rely greatly on the initial conditions. In this work, we propose a new strategy that combines Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) with the initial points generated by k-means clustering method and three various channels to improve the overall performance. Experimental results show that, for 500 different phase aberrations with root mean square (RMS) value bounded within [0.2λ, 0.3λ], the minimum, the maximum and the mean RMS residual errors reach 0.017λ, 0.066λ and 0.039λ, respectively, and 84.8% of the RMS residual errors are less than 0.05λ. We have further investigated and analyzed the proposed method in details to quantitatively demonstrate its performance: statistical results reveal that our proposed PDPR with k-means clustering enhanced method has excellent robustness in terms of initial points and other influential factors, and the accuracy can outperform its counterpart methods such as classic L-BFGS and modified BFGS.
  • PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization

    Li, Zhize; Bao, Hongyan; Zhang, Xiangliang; Richtarik, Peter (arXiv, 2020-08-25) [Preprint]
    In this paper, we propose a novel stochastic gradient estimator---ProbAbilistic Gradient Estimator (PAGE)---for nonconvex optimization. PAGE is easy to implement as it is designed via a small adjustment to vanilla SGD: in each iteration, PAGE uses the vanilla minibatch SGD update with probability $p$ and reuses the previous gradient with a small adjustment, at a much lower computational cost, with probability $1-p$. We give a simple formula for the optimal choice of $p$. We prove tight lower bounds for nonconvex problems, which are of independent interest. Moreover, we prove matching upper bounds both in the finite-sum and online regimes, which establish that Page is an optimal method. Besides, we show that for nonconvex functions satisfying the Polyak-\L{}ojasiewicz (PL) condition, PAGE can automatically switch to a faster linear convergence rate. Finally, we conduct several deep learning experiments (e.g., LeNet, VGG, ResNet) on real datasets in PyTorch, and the results demonstrate that PAGE converges much faster than SGD in training and also achieves the higher test accuracy, validating our theoretical results and confirming the practical superiority of PAGE.

View more