Recent Submissions

  • A Method for 3D Reconstruction and Virtual Reality Analysis of Glial and Neuronal Cells.

    Cali, Corrado; Kare, Kalpana; Agus, Marco; Veloz Castillo, Maria Fernanda; Boges, Daniya; Hadwiger, Markus; Magistretti, Pierre J. (Journal of visualized experiments : JoVE, MyJove Corporation, 2019-10-15) [Article]
    Serial sectioning and subsequent high-resolution imaging of biological tissue using electron microscopy (EM) allow for the segmentation and reconstruction of high-resolution imaged stacks to reveal ultrastructural patterns that could not be resolved using 2D images. Indeed, the latter might lead to a misinterpretation of morphologies, like in the case of mitochondria; the use of 3D models is, therefore, more and more common and applied to the formulation of morphology-based functional hypotheses. To date, the use of 3D models generated from light or electron image stacks makes qualitative, visual assessments, as well as quantification, more convenient to be performed directly in 3D. As these models are often extremely complex, a virtual reality environment is also important to be set up to overcome occlusion and to take full advantage of the 3D structure. Here, a step-by-step guide from image segmentation to reconstruction and analysis is described in detail.
  • 3D cellular reconstruction of cortical glia and parenchymal morphometric analysis from Serial Block-Face Electron Microscopy of juvenile rat.

    Cali, Corrado; Agus, Marco; Kare, Kalpana; Boges, Daniya J; Lehväslaiho, Heikki; Hadwiger, Markus; Magistretti, Pierre J. (Progress in neurobiology, Elsevier BV, 2019-09-25) [Article]
    With the rapid evolution in the automation of serial electron microscopy in life sciences, the acquisition of terabyte-sized datasets is becoming increasingly common. High resolution serial block-face imaging (SBEM) of biological tissues offers the opportunity to segment and reconstruct nanoscale structures to reveal spatial features previously inaccessible with simple, single section, two-dimensional images, with a particular focus on glial cells, whose reconstruction efforts in literature are still limited, compared to neurons. Here, we imaged a 750000 cubic micron volume of the somatosensory cortex from a juvenile P14 rat, with 20 nm accuracy. We recognized a total of 186 cells using their nuclei, and classified them as neuronal or glial based on features of the soma and the processes. We reconstructed for the first time 4 almost complete astrocytes and neurons, 4 complete microglia and 4 complete pericytes, including their intracellular mitochondria, 186 nuclei and 213 myelinated axons. We then performed quantitative analysis on the three-dimensional models. Out of the data that we generated, we observed that neurons have larger nuclei, which correlated with their lesser density, and that astrocytes and pericytes have a higher surface to volume ratio, compared to other cell types. All reconstructed morphologies represent an important resource for computational neuroscientists, as morphological quantitative information can be inferred, to tune simulations that take into account the spatial compartmentalization of the different cell types.
  • Quantitative Phase and Intensity Microscopy Using Snapshot White Light Wavefront Sensing

    Wang, Congli; Fu, Qiang; Dun, Xiong; Heidrich, Wolfgang (Scientific Reports, Springer Science and Business Media LLC, 2019-09-24) [Article]
    Phase imaging techniques are an invaluable tool in microscopy for quickly examining thin transparent specimens. Existing methods are limited to either simple and inexpensive methods that produce only qualitative phase information (e.g. phase contrast microscopy, DIC), or significantly more elaborate and expensive quantitative methods. Here we demonstrate a low-cost, easy to implement microscopy setup for quantitative imaging of phase and bright field amplitude using collimated white light illumination.
  • A Lagrangian Method for Extracting Eddy Boundaries in the Red Sea and the Gulf of Aden

    Friederici, Anke; Mahamadou Kele, Habib Toye; Hoteit, Ibrahim; Weinkauf, Tino; Theisel, Holger; Hadwiger, Markus (IEEE, 2019-09-05) [Conference Paper]
    Mesoscale ocean eddies play a major role for both the intermixing of water and the transport of biological mass. This makes the identification and tracking of their shape, location and deformation over time highly important for a number of applications. While eddies maintain a roughly circular shape in the free ocean, the narrow basins of the Red Sea and Gulf of Aden lead to the formation of irregular eddy shapes that existing methods struggle to identify. We propose the following model: Inside an eddy, particles rotate around a common core and thereby remain at a constant distance under a certain parametrization. The transition to the more unpredictable flow on the outside can thus be identified as the eddy boundary. We apply this algorithm on a realistic simulation of the Red Sea circulation, where we are able to identify the shape of irregular eddies robustly and more coherently than previous methods. We visualize the eddies as tubes in space-time to enable the analysis of their movement and deformation over several weeks.
  • ScaleTrotter: Illustrative Visual Travels Across Negative Scales

    Halladjian, Sarkis; Miao, Haichao; Kouril, David; Groller, M. Eduard; Viola, Ivan; Isenberg, Tobias (IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers (IEEE), 2019-08-22) [Article]
    We present ScaleTrotter, a conceptual framework for an interactive, multi-scale visualization of biological mesoscale data and, specifically, genome data. ScaleTrotter allows viewers to smoothly transition from the nucleus of a cell to the atomistic composition of the DNA, while bridging several orders of magnitude in scale. The challenges in creating an interactive visualization of genome data are fundamentally different in several ways from those in other domains like astronomy that require a multi-scale representation as well. First, genome data has intertwined scale levels-the DNA is an extremely long, connected molecule that manifests itself at all scale levels. Second, elements of the DNA do not disappear as one zooms out-instead the scale levels at which they are observed group these elements differently. Third, we have detailed information and thus geometry for the entire dataset and for all scale levels, posing a challenge for interactive visual exploration. Finally, the conceptual scale levels for genome data are close in scale space, requiring us to find ways to visually embed a smaller scale into a coarser one. We address these challenges by creating a new multi-scale visualization concept. We use a scale-dependent camera model that controls the visual embedding of the scales into their respective parents, the rendering of a subset of the scale hierarchy, and the location, size, and scope of the view. In traversing the scales, ScaleTrotter is roaming between 2D and 3D visual representations that are depicted in integrated visuals. We discuss, specifically, how this form of multi-scale visualization follows from the specific characteristics of the genome data and describe its implementation. Finally, we discuss the implications of our work to the general illustrative depiction of multi-scale data.
  • Multi-Scale Procedural Animations of Microtubule Dynamics Based on Measured Data

    Klein, Tobias; Viola, Ivan; Groller, Eduard; Mindek, Peter (IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers (IEEE), 2019-08-22) [Article]
    Biologists often use computer graphics to visualize structures, which due to physical limitations are not possible to image with a microscope. One example for such structures are microtubules, which are present in every eukaryotic cell. They are part of the cytoskeleton maintaining the shape of the cell and playing a key role in the cell division. In this paper, we propose a scientificallyaccurate multi-scale procedural model of microtubule dynamics as a novel application scenario for procedural animation, which can generate visualizations of their overall shape, molecular structure, as well as animations of the dynamic behaviour of their growth and disassembly. The model is spanning from tens of micrometers down to atomic resolution. All the aspects of the model are driven by scientific data. The advantage over a traditional, manual animation approach is that when the underlying data change, for instance due to new evidence, the model can be recreated immediately. The procedural animation concept is presented in its generic form, with several novel extensions, facilitating an easy translation to other domains with emergent multi-scale behavior.
  • Structured Regularization of Functional Map Computations

    Ren, Jing; Panine, Mikhail; Wonka, Peter; Ovsjanikov, Maks (Computer Graphics Forum, Wiley, 2019-08-12) [Article]
    We consider the problem of non-rigid shape matching using the functional map framework. Specifically, we analyze a commonly used approach for regularizing functional maps, which consists in penalizing the failure of the unknown map to commute with the Laplace-Beltrami operators on the source and target shapes. We show that this approach has certain undesirable fundamental theoretical limitations, and can be undefined even for trivial maps in the smooth setting. Instead we propose a novel, theoretically well-justified approach for regularizing functional maps, by using the notion of the resolvent of the Laplacian operator. In addition, we provide a natural one-parameter family of regularizers, that can be easily tuned depending on the expected approximate isometry of the input shape pair. We show on a wide range of shape correspondence scenarios that our novel regularization leads to an improvement in the quality of the estimated functional, and ultimately pointwise correspondences before and after commonly-used refinement techniques.
  • Refractive telescope design with digital correction of residual chromatic aberrations

    Zhang, Jingang; Nie, Yunfeng; Fu, Qiang; Peng, Yifan; Wang, Shuzhen (SPIE-Intl Soc Optical Eng, 2019-06-21) [Conference Paper]
    In general, optical designers employ combinations of multiple lenses with extraordinary dispersion materials to correct chromatic aberrations, which usually leads to considerable volume and weight. In this paper, a tailored design scheme that exploits state-of-the-art digital aberration correction algorithms in addition to traditional optics design is investigated. In particular, the proposed method is applied to the design of refractive telescopes by shifting the burden of correcting chromatic aberrations to software. By tailoring the point spread function in primary optical design for one specified wavelength and then enforcing multi-wavelength information transfer in a post-processing step, the uncorrected chromatic aberrations are well mitigated. Accordingly, a telescope of f-8, 1,400mm focal length, and 0.14 ◦ field of view is designed with only two lens elements. The image quality of the designed telescope is evaluated by comparing it to the equivalent designs with multiple lenses in a traditional optical design manner, which validates the effectiveness of our design scheme.
  • Local Editing of Procedural Models

    Lipp, M.; Specht, M.; Lau, C.; Wonka, Peter; Müller, P. (Computer Graphics Forum, Wiley, 2019-06-07) [Article]
    Procedural modeling is used across many industries for rapid 3D content creation. However, professional procedural tools often lack artistic control, requiring manual edits on baked results, diminishing the advantages of a procedural modeling pipeline. Previous approaches to enable local artistic control require special annotations of the procedural system and manual exploration of potential edit locations. Therefore, we propose a novel approach to discover meaningful and non-redundant good edit locations (GELs). We introduce a bottom-up algorithm for finding GELs directly from the attributes in procedural models, without special annotations. To make attribute edits at GELs persistent, we analyze their local spatial context and construct a meta-locator to uniquely specify their structure. Meta-locators are calculated independently per attribute, making them robust against changes in the procedural system. Functions on meta-locators enable intuitive and robust multi-selections. Finally, we introduce an algorithm to transfer meta-locators to a different procedural model. We show that our approach greatly simplifies the exploration of the local edit space, and we demonstrate its usefulness in a user study and multiple real-world examples.
  • Shape analysis of 3D nanoscale reconstructions of brain cell nuclear envelopes by implicit and explicit parametric representations

    Agus, Marco; Veloz Castillo, Maria; Garnica Molina, Javier F.; Gobbetti, Enrico; Lehväslaiho, Heikki; Morales Tapia, Alex; Magistretti, Pierre J.; Hadwiger, Markus; Cali, Corrado (Computers & Graphics: X, Elsevier BV, 2019-06-06) [Article]
    Shape analysis of cell nuclei is becoming increasingly important in biology and medicine. Recent results have identified that large variability in shape and size of nuclei has an important impact on many biological processes. Current analysis techniques involve automatic methods for detection and segmentation of histology and microscopy images, but are mostly performed in 2D. Methods for 3D shape analysis, made possible by emerging acquisition methods capable to provide nanometric-scale 3D reconstructions, are still at an early stage, and often assume a simple spherical shape. We introduce here a framework for analyzing 3D nanoscale reconstructions of nuclei of brain cells (mostly neurons), obtained by semiautomatic segmentation of electron micrographs. Our method considers two parametric representations: the first one customizes the implicit hyperquadricsformulation and it is particularly suited for convex shapes, while the latter considers a spherical harmonics decomposition of the explicit radial representation. Point clouds of nuclear envelopes, extracted from image data, are fitted to the parameterized models which are then used for performing statistical analysis and shape comparisons. We report on the analysis of a collection of 121 nuclei of brain cells obtained from the somatosensory cortex of a juvenile rat.
  • Detecting Small Faces in the Wild Based on Generative Adversarial Network and Contextual Information

    Zhang, Yongqiang; Ding, Mingli; Bai, Yancheng; Ghanem, Bernard (Pattern Recognition, Elsevier BV, 2019-05-15) [Article]
    Face detection techniques have been developed for decades, and one of the remaining open challenges is detecting small faces in unconstrained conditions. The reason is that tiny faces are often lacking detailed information and blurry. In this paper, we proposed an algorithm to directly generate a clear high-resolution face from a small blurry one by adopting a generative adversarial network (GAN). Toward this end, the basic GAN formulation achieves it by super-resolving and refining sequentially (e.g. SR-GAN and Cycle-GAN). However, we design a novel network to address the problem of super-resolving and refining jointly. Moreover, we also introduce new training losses (i.e. classification loss and regression loss) to promote the generator network to recover fine details of the small faces and to guide the discriminator network to distinguish face vs. non-face and to refine location simultaneously. Additionally, considering the importance of contextual information when detecting tiny faces in crowded cases, the context around face regions is combined to train the proposed GAN-based network for mining those very small faces from unconstrained scenarios. Extensive experiments on the challenging datasets WIDER FACE and FDDB demonstrate the effectiveness of the proposed method in restoring a clear high-resolution face from a small blurry one, and show that the achieved performance outperforms previous state-of-the-art methods by a large margin.
  • Discretizations of Surfaces with Constant Ratio of Principal Curvatures

    Jimenez, Michael R.; Müller, Christian; Pottmann, Helmut (Discrete & Computational Geometry, Springer Nature, 2019-05-10) [Article]
    Motivated by applications in architecture, we study surfaces with a constant ratio of principal curvatures. These surfaces are a natural generalization of minimal surfaces, and can be constructed by applying a Christoffel-type transformation to appropriate spherical curvature line parametrizations, both in the smooth setting and in a discretization with principal nets. We link this Christoffel-type transformation to the discrete curvature theory for parallel meshes and characterize nets that admit these transformations. In the case of negative curvature, we also present a discretization of asymptotic nets. This case is suitable for design and computation, and forms the basis for a special type of architectural support structures, which can be built by bending flat rectangular strips of inextensible material, such as sheet metal.
  • Metabopolis: scalable network layout for biological pathway diagrams in urban map style

    Wu, Hsiang-Yun; Nöllenburg, Martin; Sousa, Filipa L.; Viola, Ivan (BMC Bioinformatics, Springer Nature, 2019-04-16) [Article]
    Background Biological pathways represent chains of molecular interactions in biological systems that jointly form complex dynamic networks. The network structure changes from the significance of biological experiments and layout algorithms often sacrifice low-level details to maintain high-level information, which complicates the entire image to large biochemical systems such as human metabolic pathways. Results Our work is inspired by concepts from urban planning since we create a visual hierarchy of biological pathways, which is analogous to city blocks and grid-like road networks in an urban area. We automatize the manual drawing process of biologists by first partitioning the map domain into multiple sub-blocks, and then building the corresponding pathways by routing edges schematically, to maintain the global and local context simultaneously. Our system incorporates constrained floor-planning and network-flow algorithms to optimize the layout of sub-blocks and to distribute the edge density along the map domain. We have developed the approach in close collaboration with domain experts and present their feedback on the pathway diagrams based on selected use cases. Conclusions We present a new approach for computing biological pathway maps that untangles visual clutter by decomposing large networks into semantic sub-networks and bundling long edges to create space for presenting relationships systematically.
  • Cuttlefish: Color Mapping for Dynamic Multi-Scale Visualizations

    Waldin, N.; Waldner, M.; Le Muzic, M.; Gröller, E.; Goodsell, D. S.; Autin, L.; Olson, A. J.; Viola, Ivan (Computer Graphics Forum, Wiley, 2019-03-26) [Article]
    Visualizations of hierarchical data can often be explored interactively. For example, in geographic visualization, there are continents, which can be subdivided into countries, states, counties and cities. Similarly, in models of viruses or bacteria at the highest level are the compartments, and below that are macromolecules, secondary structures (such as α-helices), amino-acids, and on the finest level atoms. Distinguishing between items can be assisted through the use of color at all levels. However, currently, there are no hierarchical and adaptive color mapping techniques for very large multi-scale visualizations that can be explored interactively. We present a novel, multi-scale, color-mapping technique for adaptively adjusting the color scheme to the current view and scale. Color is treated as a resource and is smoothly redistributed. The distribution adjusts to the scale of the currently observed detail and maximizes the color range utilization given current viewing requirements. Thus, we ensure that the user is able to distinguish items on any level, even if the color is not constant for a particular feature. The coloring technique is demonstrated for a political map and a mesoscale structural model of HIV. The technique has been tested by users with expertise in structural biology and was overall well received.
  • Local Color Mapping Combined with Color Transfer for Underwater Image Enhancement

    Protasiuk, Rafal; Bibi, Adel; Ghanem, Bernard (2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Institute of Electrical and Electronics Engineers (IEEE), 2019-03-08) [Conference Paper]
    Color correction and color transfer methods have gained a lot of attention in the past few years to circumvent color degradation that may occur due to various sources. In this paper, we propose a novel simple yet powerful strategy to profoundly enhance color distorted underwater images. The proposed approach combines both local and global information through a simple yet powerful affine transform model. Local and global information are carried through local color mapping and color covariance mapping between an input and some reference source, respectively. Several experiments on degraded underwater images demonstrate that the proposed method performs favourably to all other methods including ones that are tailored to correcting underwater images by explicit noise modelling.
  • Computational Mechanical Modelling of Wood—From Microstructural Characteristics Over Wood-Based Products to Advanced Timber Structures

    Füssl, Josef; Lukacevic, Markus; Pillwein, Stefan; Pottmann, Helmut (LebensErfolg, Springer Nature, 2019-02-24) [Book Chapter]
    Wood as structural bearing material is often encountered with skepticism and, therefore, it is not used as extensively as its very good material properties would suggest. Beside building physics and construction reasons, the main cause of this skepticism is its quite complex material behavior, which is the reason that design concepts for wood have so far not achieved a desirable prediction accuracy. Thus, for the prediction of effective mechanical properties of wood, advanced computational tools are required, which are able to predict as well as consider multidimensional strength information at different scales of observation. Within this chapter, three computational methods are presented: an extended finite element approach able to describe strong strain-softening and, thus, reproduce brittle failure modes accurately; a numerical limit analysis approach, exclusively describing ductile failure; and an elastic limit approach based on continuum micromechanics. Based on illustrative results, the performance of these methods is shown and discussed. Furthermore, a finite-element-based design procedure for an elastically-deformed wooden structure is outlined, showing how advanced mechanical information of the base material could be exploited within digital design of complex timber structures in future. Finally, geometric design concepts applicable within digital wood design are discussed, giving insights into possible future developments.
  • Cake layer characterization in Activated Sludge Membrane Bioreactors: Real-time analysis

    Fortunato, Luca; Li, Muxingzi; Cheng, Tuoyuan; Rehman, Zahid Ur; Heidrich, Wolfgang; Leiknes, TorOve (Journal of Membrane Science, Elsevier BV, 2019-02-21) [Article]
    Activated Sludge Membrane Bioreactors (AS-MBR) are recognized as a commercially competitive alternative to conventional wastewater treatments. However, membrane fouling remains one of the main challenges and disadvantages of the process. This study evaluates the suitability of Optical Coherence Tomography (OCT) in monitoring the cake layer development in-situ in AS-MBR under continuous operation. Real-time direct imaging of the cake layer was feasible when limiting the continuous movement of the AS flocs in the reactor by turning aeration off for few minutes prior to scanning a given membrane area. The cake layer morphology was evaluated using both 2D and 3D image analysis. The 3D analysis respect to 2D analysis provided a more representative characterization of the fouling formed in the system. The non-invasive nature of OCT imaging enabled monitoring fouling development over time, where an increase in thickness and a decrease in roughness was observed in the first 200 h of operation. The 3D OCT image analyses were also compared with the 3D confocal laser scanning microscopy (CLSM) image analyses performed at the end of the study. Results demonstrate that OCT imaging can be applied for online, real-time monitoring and analysis of fouling behavior in AS-MBR systems.
  • Optimizing B-spline surfaces for developability and paneling architectural freeform surfaces

    Gavriil, Konstantinos; Schiftner, Alexander; Pottmann, Helmut (Computer-Aided Design, Elsevier BV, 2019-02-10) [Article]
    Motivated by applications in architecture and design, we present a novel method for increasing the developability of a B-spline surface. We use the property that the Gauss image of a developable surface is 1-dimensional and can be locally well approximated by circles. This is cast into an algorithm for thinning the Gauss image by increasing the planarity of the Gauss images of appropriate neighborhoods. A variation of the main method allows us to tackle the problem of paneling a freeform architectural surface with developable panels, in particular enforcing rotational cylindrical, rotational conical and planar panels, which are the main preferred types of developable panels in architecture due to the reduced cost of manufacturing.
  • Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation

    Müller, Matthias; Casser, Vincent; Smith, Neil; Michels, Dominik L.; Ghanem, Bernard (Physics of Solid Surfaces, Springer Nature, 2019-01-28) [Conference Paper]
    Automating the navigation of unmanned aerial vehicles (UAVs) in diverse scenarios has gained much attention in recent years. However, teaching UAVs to fly in challenging environments remains an unsolved problem, mainly due to the lack of training data. In this paper, we train a deep neural network to predict UAV controls from raw image data for the task of autonomous UAV racing in a photo-realistic simulation. Training is done through imitation learning with data augmentation to allow for the correction of navigation mistakes. Extensive experiments demonstrate that our trained network (when sufficient data augmentation is used) outperforms state-of-the-art methods and flies more consistently than many human pilots. Additionally, we show that our optimized network architecture can run in real-time on embedded hardware, allowing for efficient on-board processing critical for real-world deployment. From a broader perspective, our results underline the importance of extensive data augmentation techniques to improve robustness in end-to-end learning setups.
  • SGD: General Analysis and Improved Rates

    Gower, Robert Mansel; Loizou, Nicolas; Qian, Xun; Sailanbayev, Alibek; Shulgin, Egor; Richtarik, Peter (arXiv, 2019-01-27) [Preprint]
    We propose a general yet simple theorem describing the convergence of SGDunder the arbitrary sampling paradigm. Our theorem describes the convergence ofan infinite array of variants of SGD, each of which is associated with aspecific probability law governing the data selection rule used to formmini-batches. This is the first time such an analysis is performed, and most ofour variants of SGD were never explicitly considered in the literature before.Our analysis relies on the recently introduced notion of expected smoothnessand does not rely on a uniform bound on the variance of the stochasticgradients. By specializing our theorem to different mini-batching strategies,such as sampling with replacement and independent sampling, we derive exactexpressions for the stepsize as a function of the mini-batch size. With this wecan also determine the mini-batch size that optimizes the total complexity, andshow explicitly that as the variance of the stochastic gradient evaluated atthe minimum grows, so does the optimal mini-batch size. For zero variance, theoptimal mini-batch size is one. Moreover, we prove insightfulstepsize-switching rules which describe when one should switch from a constantto a decreasing stepsize regime.

View more