Visual Computing Center (VCC)
Recent Submissions

Adenita: interactive 3D modelling and visualization of DNA nanostructures.(Nucleic acids research, Oxford University Press (OUP), 20200722) [Article]DNA nanotechnology is a rapidly advancing field, which increasingly attracts interest in many different disciplines, such as medicine, biotechnology, physics and biocomputing. The increasing complexity of novel applications requires significant computational support for the design, modelling and analysis of DNA nanostructures. However, current in silico design tools have not been developed in view of these new applications and their requirements. Here, we present Adenita, a novel software tool for the modelling of DNA nanostructures in a userfriendly environment. A data model supporting different DNA nanostructure concepts (multilayer DNA origami, wireframe DNA origami, DNA tiles etc.) has been developed allowing the creation of new and the import of existing DNA nanostructures. In addition, the nanostructures can be modified and analysed onthefly using an intuitive toolset. The possibility to combine and reuse existing nanostructures as building blocks for the creation of new superstructures, the integration of alternative molecules (e.g. proteins, aptamers) during the design process, and the export option for oxDNA simulations are outstanding features of Adenita, which spearheads a new generation of DNA nanostructure modelling software. We showcase Adenita by reusing a large nanorod to create a new nanostructure through user interactions that employ different editors to modify the original nanorod.

A fully stochastic primaldual algorithm(Optimization Letters, Springer Science and Business Media LLC, 20200714) [Article]A new stochastic primaldual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions / operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d realizations. The proposed algorithm is proven to converge to a saddle point of the Lagrangian function. In the framework of the monotone operator theory, the convergence proof relies on recent results on the stochastic Forward Backward algorithm involving random monotone operators. An example of convex optimization under stochastic linear constraints is considered.

EarlyStage Growth Mechanism and Synthesis ConditionsDependent Morphology of Nanocrystalline Bi Films Electrodeposited from Perchlorate Electrolyte.(Nanomaterials (Basel, Switzerland), MDPI AG, 20200702) [Article]Bi nanocrystalline films were formed from perchlorate electrolyte (PE) on Cu substrate via electrochemical deposition with different duration and current densities. The microstructural, morphological properties, and elemental composition were studied using scanning electron microscopy (SEM), atomic force microscopy (AFM), and energydispersive Xray microanalysis (EDX). The optimal range of current densities for Bi electrodeposition in PE using polarization measurements was demonstrated. For the first time, it was shown and explained why, with a deposition duration of 1 s, codeposition of Pb and Bi occurs. The correlation between synthesis conditions and chemical composition and microstructure for Bi films was discussed. The analysis of the microstructure evolution revealed the changing mechanism of the films' growth from pillarlike (for Pbrich phase) to layered granular form (for Bi) with deposition duration rising. This abnormal behavior is explained by the appearance of a strong Bi growth texture and coalescence effects. The investigations of porosity showed that Bi films have a closelypacked microstructure. The main stages and the growth mechanism of Bi films in the galvanostatic regime in PE with a deposition duration of 130 s are proposed.

Virtual reality framework for editing and exploring medial axis representations of nanometric scale neural structures(Computers and Graphics (Pergamon), Elsevier BV, 20200624) [Article]We present a novel virtual reality (VR) based framework for the exploratory analysis of nanoscale 3D reconstructions of cellular structures acquired from rodent brain samples through serial electron microscopy. The system is specifically targeted on medial axis representations (skeletons) of branched and tubular structures of cellular shapes, and it is designed for providing to domain scientists: i) effective and fast semiautomatic interfaces for tracing skeletons directly on surfacebased representations of cells and structures, ii) fast tools for proofreading, i.e., correcting and editing of semiautomatically constructed skeleton representations, and iii) natural methods for interactive exploration, i.e., measuring, comparing, and analyzing geometric features related to cellular structures based on medial axis representations. Neuroscientists currently use the system for performing morphology studies on sparse reconstructions of glial cells and neurons extracted from a sample of the somatosensory cortex of a juvenile rat. The framework runs in a standard PC and has been tested on two different display and interaction setups: PCtethered stereoscopic headmounted display (HMD) with 3D controllers and tracking sensors, and a large display wall with a standard gamepad controller. We report on a user study that we carried out for analyzing user performance on different tasks using these two setups.

A NonAsymptotic Analysis for Stein Variational Gradient Descent(arXiv, 20200617) [Preprint]We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $\pi\propto e^{V}$ on $\mathbb{R}^d$. In the population limit, SVGD performs gradient descent in the space of probability distributions on the KL divergence with respect to $\pi$, where the gradient is smoothed through a kernel integral operator. In this paper, we provide a novel finite time analysis for the SVGD algorithm. We obtain a descent lemma establishing that the algorithm decreases the objective at each iteration, and provably converges, with less restrictive assumptions on the step size than required in earlier analyses. We further provide a guarantee on the convergence rate in KullbackLeibler divergence, assuming $\pi$ satisfies a Stein logSobolev inequality as in Duncan et al. (2019), which takes into account the geometry induced by the smoothed KL gradient.

DeeperGCN: All You Need to Train Deeper GCNs(arXiv, 20200613) [Preprint]Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, oversmoothing and overfitting issues when going deeper. These challenges limit the representation power of GCNs on largescale graphs. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs. We define differentiable generalized aggregation functions to unify different message aggregation operations (e.g. mean, max). We also propose a novel normalization layer namely MsgNorm and a preactivation version of residual connections for GCNs. Extensive experiments on Open Graph Benchmark (OGB) show DeeperGCN significantly boosts performance over the stateoftheart on the large scale graph learning tasks of node property prediction and graph property prediction. Please visit https://www.deepgcns.org for more information.

A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization(arXiv, 20200612) [Preprint]In this paper, we study the performance of a large family of SGD variants in the smooth nonconvex regime. To this end, we propose a generic and flexible assumption capable of accurate modeling of the second moment of the stochastic gradient. Our assumption is satisfied by a large number of specific variants of SGD in the literature, including SGD with arbitrary sampling, SGD with compressed gradients, and a wide variety of variancereduced SGD methods such as SVRG and SAGA. We provide a single convergence analysis for all methods that satisfy the proposed unified assumption, thereby offering a unified understanding of SGD variants in the nonconvex regime instead of relying on dedicated analyses of each variant. Moreover, our unified analysis is accurate enough to recover or improve upon the bestknown convergence results of several classical methods, and also gives new convergence results for many new methods which arise as special cases. In the more general distributed/federated nonconvex optimization setup, we propose two new general algorithmic frameworks differing in whether direct gradient compression (DC) or compression of gradient differences (DIANA) is used. We show that all methods captured by these two frameworks also satisfy our unified assumption. Thus, our unified convergence analysis also captures a large variety of distributed methods utilizing compressed communication. Finally, we also provide a unified analysis for obtaining faster linear convergence rates in this nonconvex regime under the PL condition.

Accurately Solving Physical Systems with Graph Learning(arXiv, 20200606) [Preprint]Iterative solvers are widely used to accurately simulate physical systems. These solvers require initial guesses to generate a sequence of improving approximate solutions. In this contribution, we introduce a novel method to accelerate iterative solvers for physical systems with graph networks (GNs) by predicting the initial guesses to reduce the number of iterations. Unlike existing methods that aim to learn physical systems in an endtoend manner, our approach guarantees longterm stability and therefore leads to more accurate solutions. Furthermore, our method improves the run time performance of traditional iterative solvers. To explore our method we make use of positionbased dynamics (PBD) as a common solver for physical systems and evaluate it by simulating the dynamics of elastic rods. Our approach is able to generalize across different initial conditions, discretizations, and realistic material properties. Finally, we demonstrate that our method also performs well when taking discontinuous effects into account such as collisions between individual rods. A video showing dynamic results of our graph learning assisted simulations of elastic rods can be found on the project website available at http://computationalsciences.org/publications/shao2020physicalsystemsgraphlearning.html .

Modeling in the Time of COVID19: Statistical and Rulebased Mesoscale Models(arXiv, 20200501) [Preprint]We present a new technique for rapid modeling and construction of scientifically accurate mesoscale biological models. Resulting 3D models are based on few 2D microscopy scans and the latest knowledge about the biological entity represented as a set of geometric relationships. Our new technique is based on statistical and rulebased modeling approaches that are rapid to author, fast to construct, and easy to revise. From a few 2D microscopy scans, we learn statistical properties of various structural aspects, such as the outer membrane shape, spatial properties and distribution characteristics of the macromolecular elements on the membrane. This information is utilized in 3D model construction. Once all imaging evidence is incorporated in the model, additional information can be incorporated by interactively defining rules that spatially characterize the rest of the biological entity, such as mutual interactions among macromolecules, their distances and orientations to other structures. These rules are defined through an intuitive 3D interactive visualization and modeling feedback loop. We demonstrate the utility of our approach on a use case of the modeling procedure of the SARSCoV2 virus particle ultrastructure. Its first complete atomistic model, which we present here, can steer biological research to new promising directions in fighting spread of the virus.

Disentangled Image Generation Through Structured Noise Injection(arXiv, 20200426) [Preprint]We explore different design choices for injecting noise into generative adversarial networks (GANs) with the goal of disentangling the latent space. Instead of traditional approaches, we propose feeding multiple noise codes through separate fullyconnected layers respectively. The aim is restricting the influence of each noise code to specific parts of the generated image. We show that disentanglement in the first layer of the generator network leads to disentanglement in the generated image. Through a gridbased structure, we achieve several aspects of disentanglement without complicating the network architecture and without requiring labels. We achieve spatial disentanglement, scalespace disentanglement, and disentanglement of the foreground object from the background style allowing finegrained control over the generated images. Examples include changing facial expressions in face images, changing beak length in bird images, and changing car dimensions in car images. This empirically leads to better disentanglement scores than stateoftheart methods on the FFHQ dataset.

Unambiguous scattering matrix for nonHermitian systems(Physical Review A, American Physical Society (APS), 20200423) [Article]PT symmetry is a unique platform for light manipulation and versatile use in unidirectional invisibility, lasing, sensing, etc. Broken and unbroken PTsymmetric states in nonHermitian open systems are described by scattering matrices. A multilayer structure, as a simplest example of the open system, has no certain definition of the scattering matrix, since the output ports can be permuted. The uncertainty in definition of the exceptional points bordering PTsymmetric and PTsymmetrybroken states poses an important problem, because the exceptional points are indispensable in applications such as sensing and mode discrimination. Here we derive the proper scattering matrix from the unambiguous relation between the PTsymmetric Hamiltonian and scattering matrix. We reveal that the exceptional points of the scattering matrix with permuted output ports are not related to the PT symmetry breaking. Nevertheless, they can be employed for finding a lasing onset as demonstrated in our timedomain calculations and scatteringmatrix pole analysis. Our results are important for various applications of the nonHermitian systems including encircling exceptional points, coherent perfect absorption, PTsymmetric plasmonics, etc.

Method of Surface Energy Investigation for Nanostructured Materials: Application to Control NiFe Films Growth Mechanism(SSRN Electronic Journal, Elsevier BV, 20200416) [Article]A new method for the specific surface energy investigation based on a combination of the force spectroscopy and the method of nanofriction study using atomic force microscopy was proposed. It was shown that air humidity does not affect the results of investigation by the proposed method as opposed to the previously used methods. Therefore, the method has high accuracy and repeatability in air without use of climate chambers and liquid cells. The proposed method has a high local resolution and is suitable for investigation of the specific surface energy of individual nanoparticles. The achievements described in the paper demonstrate one of the method capabilities, which is to control the growth mechanism of thin magnetic films. The conditions for the transition of the growth mechanism of thin Ni80Fe20 films from island to layerbylayer obtained via electrolyte deposition have been determined using the proposed method and the purpose made probes with Ni coating.

Phase Consistent Ecological Domain Adaptation(arXiv, 20200410) [Preprint]We introduce two criteria to regularize the optimization involved in learning a classifier in a domain where no annotated data are available, leveraging annotated data in a different domain, a problem known as unsupervised domain adaptation. We focus on the task of semantic segmentation, where annotated synthetic data are aplenty, but annotating real data is laborious. The first criterion, inspired by visual psychophysics, is that the map between the two image domains be phasepreserving. This restricts the set of possible learned maps, while enabling enough flexibility to transfer semantic information. The second criterion aims to leverage ecological statistics, or regularities in the scene which are manifest in any image of it, regardless of the characteristics of the illuminant or the imaging sensor. It is implemented using a deep neural network that scores the likelihood of each possible segmentation given a single unannotated image. Incorporating these two priors in a standard domain adaptation framework improves performance across the board in the most common unsupervised domain adaptation benchmarks for semantic segmentation.

Learning a controller fusion network by online trajectory filtering for visionbased UAV racing(IEEE, 20200410) [Conference Paper]Autonomous UAV racing has recently emerged as an interesting research problem. The dream is to beat humans in this new fastpaced sport. A common approach is to learn an endtoend policy that directly predicts controls from raw images by imitating an expert. However, such a policy is limited by the expert it imitates and scaling to other environments and vehicle dynamics is difficult. One approach to overcome the drawbacks of an endtoend policy is to train a network only on the perception task and handle control with a PID or MPC controller. However, a single controller must be extensively tuned and cannot usually cover the whole state space. In this paper, we propose learning an optimized controller using a DNN that fuses multiple controllers. The network learns a robust controller with online trajectory filtering, which suppresses noisy trajectories and imperfections of individual controllers. The result is a network that is able to learn a good fusion of filtered trajectories from different controllers leading to significant improvements in overall performance. We compare our trained network to controllers it has learned from, endtoend baselines and human pilots in a realistic simulation; our network beats all baselines in extensive experiments and approaches the performance of a professional human pilot.

Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms(arXiv, 20200403) [Preprint]We introduce a new primaldual algorithm for minimizing the sum of three convex functions, each of which has its own oracle. Namely, the first one is differentiable, smooth and possibly stochastic, the second is proximable, and the last one is a composition of a proximable function with a linear map. Our theory covers several settings that are not tackled by any existing algorithm; we illustrate their importance with realworld applications. By leveraging variance reduction, we obtain convergence with linear rates under strong convexity and fast sublinear convergence under convexity assumptions. The proposed theory is simple and unified by the umbrella of stochastic DavisYin splitting, which we design in this work. Finally, we illustrate the efficiency of our method through numerical experiments.

Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization(arXiv, 20200226) [Preprint]Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While in other contexts the best performing gradienttype methods invariably rely on some form of acceleration/momentum to reduce the number of iterations, there are no methods which combine the benefits of both gradient compression and acceleration. In this paper, we remedy this situation and propose the first accelerated compressed gradient descent (ACGD) methods. In the single machine regime, we prove that ACGD enjoys the rate $O\left((1+\omega)\sqrt{\frac{L}{\mu}}\log \frac{1}{\epsilon}\right)$ for $\mu$strongly convex problems and $O\left((1+\omega)\sqrt{\frac{L}{\epsilon}}\right)$ for convex problems, respectively, where $L$ is the smoothness constant and $\omega$ is the compression parameter. Our results improve upon the existing nonaccelerated rates $O\left((1+\omega)\frac{L}{\mu}\log \frac{1}{\epsilon}\right)$ and $O\left((1+\omega)\frac{L}{\epsilon}\right)$, respectively, and recover the optimal rates of accelerated gradient descent as a special case when no compression ($\omega=0$) is applied. We further propose a distributed variant of ACGD (called ADIANA) and prove the convergence rate $\widetilde{O}\left(\omega+\sqrt{\frac{L}{\mu}} +\sqrt{\left(\frac{\omega}{n}+\sqrt{\frac{\omega}{n}}\right)\frac{\omega L}{\mu}}\right)$, where $n$ is the number of devices/workers and $\widetilde{O}$ hides the logarithmic factor $\log \frac{1}{\epsilon}$. This improves upon the previous best result $\widetilde{O}\left(\omega + \frac{L}{\mu}+\frac{\omega L}{n\mu} \right)$ achieved by the DIANA method of Mishchenko et al (2019). Finally, we conduct several experiments on realworld datasets which corroborate our theoretical results and confirm the practical superiority of our methods.

HyperLabels: Browsing of Dense and Hierarchical Molecular 3D Models(IEEE Transactions on Visualization and Computer Graphics, IEEE, 20200224) [Article]We present a method for the browsing of hierarchical 3D models in which we combine the typical navigation of hierarchical structures in a 2D environmentusing clicks on nodes, links, or iconswith a 3D spatial data visualization. Our approach is motivated by large molecular models, for which the traditional singlescale navigational metaphors are not suitable. Multiscale phenomena, e.g., in astronomy or geography, are complex to navigate due to their large data spaces and multilevel organization. Models from structural biology are in addition also densely crowded in space and scale. Cutaways are needed to show individual model subparts. The camera has to support exploration on the level of a whole virus, as well as on the level of a small molecule. We address these challenges by employing HyperLabels: active labels thatin addition to their annotational rolealso support user interaction. Clicks on HyperLabels select the next structure to be explored. Then, we adjust the visualization to showcase the inner composition of the selected subpart and enable further exploration. Finally, we use a breadcrumbs panel for orientation and as a mechanism to traverse upwards in the model hierarchy. We demonstrate our concept of hierarchical 3D model browsing using two exemplary models from mesoscale biology.

Percolation and Transport Properties in The Mechanically Deformed Composites Filled with Carbon Nanotubes(Applied Sciences, MDPI AG, 20200218) [Article]The conductivity and percolation concentration of the composite material filled with randomly distributed carbon nanotubes were simulated as a function of the mechanical deformation. Nanotubes were modelled as the hardcore ellipsoids of revolution with high aspect ratio. The evident anisotropy was observed in the percolation threshold and conductivity. The minimal mean values of the percolation of 4.6 vol. % and maximal conductivity of 0.74 S/m were found for the isotropic composite. Being slightly aligned, the composite demonstrates lower percolation concentration and conductivity along the orientation of the nanotubes compared to the perpendicular arrangement.

Wasserstein Proximal Gradient(arXiv, 20200207) [Preprint]We consider the task of sampling from a logconcave probability distribution. This target distribution can be seen as a minimizer of the relative entropy functional defined on the space of probability distributions. The relative entropy can be decomposed as the sum of a functional called the potential energy, assumed to be smooth, and a nonsmooth functional called the entropy. We adopt a Forward Backward (FB) Euler scheme for the discretization of the gradient flow of the relative entropy. This FB algorithm can be seen as a proximal gradient algorithm to minimize the relative entropy over the space of probability measures. Using techniques from convex optimization and optimal transport, we provide a nonasymptotic analysis of the FB algorithm. The convergence rate of the FB algorithm matches the convergence rate of the classical proximal gradient algorithm in Euclidean spaces. The practical implementation of the FB algorithm can be challenging. In practice, the user may choose to discretize the space and work with empirical measures. In this case, we provide a closed form formula for the proximity operator of the entropy.

Modeling classical wavefront sensors(Optics Express, The Optical Society, 20200202) [Article]We present an image formation model for deterministic phase retrieval in propagationbased wavefront sensing, unifying analysis for classical wavefront sensors such as ShackHartmann (slopes tracking) and curvature sensors (based on TransportofIntensity Equation). We show how this model generalizes commonly seen formulas, including TransportofIntensity Equation, from small distances and beyond. Using this model, we analyze theoretically achievable lateral wavefront resolution in propagationbased deterministic wavefront sensing. Finally, via a prototype masked wavefront sensor, we show simultaneous bright field and phase imaging numerically recovered in realtime from a singleshot measurement.