• Tagging like Humans: Diverse and Distinct Image Annotation

      Wu, Baoyuan; Chen, Weidong; Sun, Peng; Liu, Wei; Ghanem, Bernard; Lyu, Siwei (arXiv, 2018-03-31)
      In this work we propose a new automatic image annotation model, dubbed {\bf diverse and distinct image annotation} (D2IA). The generative model D2IA is inspired by the ensemble of human annotations, which create semantically relevant, yet distinct and diverse tags. In D2IA, we generate a relevant and distinct tag subset, in which the tags are relevant to the image contents and semantically distinct to each other, using sequential sampling from a determinantal point process (DPP) model. Multiple such tag subsets that cover diverse semantic aspects or diverse semantic levels of the image contents are generated by randomly perturbing the DPP sampling process. We leverage a generative adversarial network (GAN) model to train D2IA. Extensive experiments including quantitative and qualitative comparisons, as well as human subject studies, on two benchmark datasets demonstrate that the proposed model can produce more diverse and distinct tags than the state-of-the-arts.
    • Teaching UAVs to Race Using UE4Sim

      Mueller, Matthias; Casser, Vincent; Smith, Neil; Michels, Dominik L.; Ghanem, Bernard (arXiv, 2017-08-19)
      Automating the navigation of unmanned aerial vehicles (UAVs) in diverse scenarios has gained much attention in the recent years. However, teaching UAVs to fly in challenging environments remains an unsolved problem, mainly due to the lack of data for training. In this paper, we develop a photo-realistic simulator that can afford the generation of large amounts of training data (both images rendered from the UAV camera and its controls) to teach a UAV to autonomously race through challenging tracks. We train a deep neural network to predict UAV controls from raw image data for the task of autonomous UAV racing. Training is done through imitation learning enabled by data augmentation to allow for the correction of navigation mistakes. Extensive experiments demonstrate that our trained network (when sufficient data augmentation is used) outperforms state-of-the-art methods and flies more consistently than many human pilots.
    • Teaching UAVs to Race With Observational Imitation Learning

      Li, Guohao; Mueller, Matthias; Casser, Vincent; Smith, Neil; Michels, Dominik L.; Ghanem, Bernard (arXiv, 2018-03-03)
      Recent work has tackled the problem of autonomous navigation by imitating a teacher and learning an end-to-end policy, which directly predicts controls from raw images. However, these approaches tend to be sensitive to mistakes by the teacher and do not scale well to other environments or vehicles. To this end, we propose a modular network architecture that decouples perception from control, and is trained using Observational Imitation Learning (OIL), a novel imitation learning variant that supports online training and automatic selection of optimal behavior from observing multiple teachers. We apply our proposed methodology to the challenging problem of unmanned aerial vehicle (UAV) racing. We develop a simulator that enables the generation of large amounts of synthetic training data (both UAV captured images and its controls) and also allows for online learning and evaluation. We train a perception network to predict waypoints from raw image data and a control network to predict UAV controls from these waypoints using OIL. Our modular network is able to autonomously fly a UAV through challenging race tracks at high speeds. Extensive experiments demonstrate that our trained network outperforms its teachers, end-to-end baselines, and even human pilots in simulation. The supplementary video can be viewed at https://youtu.be/PeTXSoriflc
    • Theory of Topological Spin Hall Effect in Antiferromagnetic Skyrmion: Impact on Current-induced Motion

      Akosa, Collins Ashu; Tretiakov, O. A.; Tatara, G.; Manchon, Aurelien (arXiv, 2017-09-09)
      We demonstrate that the nontrivial magnetic texture of antiferromagnetic skyrmions (AFM-Sks) promotes a non-vanishing topological spin Hall effect (TSHE) on the flowing electrons. This results in a substantial enhancement of the non-adiabatic torque and hence improves the skyrmion mobility. This non-adiabatic torque increases when decreasing the skyrmion size, and therefore scaling down results in a much higher torque efficiency. In clean AFM-Sks, we find a significant boost of the TSHE close to van Hove singularity. Interestingly, this effect is enhanced away from the band gap in the presence of non-magnetic interstitial defects. Furthermore, unlike their ferromagnetic counterpart, TSHE in AFM-Sks increases with increase in disorder strength thus opening promising avenues for materials engineering of this effect.
    • Thermodynamically consistent modeling and simulation of multi-component two-phase flow model with partial miscibility

      Kou, Jisheng; Sun, Shuyu (arXiv, 2016-11-25)
      A general diffuse interface model with a realistic equation of state (e.g. Peng-Robinson equation of state) is proposed to describe the multi-component two-phase fluid flow based on the principles of the NVT-based framework which is a latest alternative over the NPT-based framework to model the realistic fluids. The proposed model uses the Helmholtz free energy rather than Gibbs free energy in the NPT-based framework. Different from the classical routines, we combine the first law of thermodynamics and related thermodynamical relations to derive the entropy balance equation, and then we derive a transport equation of the Helmholtz free energy density. Furthermore, by using the second law of thermodynamics, we derive a set of unified equations for both interfaces and bulk phases that can describe the partial miscibility of two fluids. A relation between the pressure gradient and chemical potential gradients is established, and this relation leads to a new formulation of the momentum balance equation, which demonstrates that chemical potential gradients become the primary driving force of fluid motion. Moreover, we prove that the proposed model satisfies the total (free) energy dissipation with time. For numerical simulation of the proposed model, the key difficulties result from the strong nonlinearity of Helmholtz free energy density and tight coupling relations between molar densities and velocity. To resolve these problems, we propose a novel convex-concave splitting of Helmholtz free energy density and deal well with the coupling relations between molar densities and velocity through very careful physical observations with a mathematical rigor. We prove that the proposed numerical scheme can preserve the discrete (free) energy dissipation. Numerical tests are carried out to verify the effectiveness of the proposed method.
    • Thermodynamically consistent simulation of nonisothermal diffuse-interface two-phase flow with Peng-Robinson equation of state

      Kou, Jisheng; Sun, Shuyu (arXiv, 2017-12-06)
      In this paper, we consider a diffuse-interface gas-liquid two-phase flow model with inhomogeneous temperatures, in which we employ the Peng-Robinson equation of state and the temperature-dependent influence parameter instead of the van der Waals equation of state and the constant influence parameter used in the existing models. As a result, our model can characterize accurately the physical behaviors of numerous realistic gas-liquid fluids, especially hydrocarbons. Furthermore, we prove a relation associating the pressure gradient with the gradients of temperature and chemical potential, and thereby derive a new formulation of the momentum balance equation, which shows that gradients of the chemical potential and temperature become the primary driving force of the fluid motion. It is rigorously proved that the new formulations of the model obey the first and second laws of thermodynamics. To design efficient numerical methods, we prove that Helmholtz free energy density is a concave function with respect to the temperature under certain physical conditions. Based on the proposed modeling formulations and the convex-concave splitting of Helmholtz free energy density, we propose a novel thermodynamically stable numerical scheme. We rigorously prove that the proposed method satisfies the first and second laws of thermodynamics. Finally, numerical tests are carried out to verify the effectiveness of the proposed simulation method.
    • Total Variation Depth for Functional Data

      Huang, Huang; Sun, Ying (arXiv, 2016-11-15)
      There has been extensive work on data depth-based methods for robust multivariate data analysis. Recent developments have moved to infinite-dimensional objects such as functional data. In this work, we propose a new notion of depth, the total variation depth, for functional data. As a measure of depth, its properties are studied theoretically, and the associated outlier detection performance is investigated through simulations. Compared to magnitude outliers, shape outliers are often masked among the rest of samples and harder to identify. We show that the proposed total variation depth has many desirable features and is well suited for outlier detection. In particular, we propose to decompose the total variation depth into two components that are associated with shape and magnitude outlyingness, respectively. This decomposition allows us to develop an effective procedure for outlier detection and useful visualization tools, while naturally accounting for the correlation in functional data. Finally, the proposed methodology is demonstrated using real datasets of curves, images, and video frames.
    • TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild

      Müller, Matthias; Bibi, Adel Aamer; Giancola, Silvio; Al-Subaihi, Salman; Ghanem, Bernard (arXiv, 2018-03-28)
      Despite the numerous developments in object tracking, further development of current tracking algorithms is limited by small and mostly saturated datasets. As a matter of fact, data-hungry trackers based on deep-learning currently rely on object detection datasets due to the scarcity of dedicated large-scale tracking datasets. In this work, we present TrackingNet, the first large-scale dataset and benchmark for object tracking in the wild. We provide more than 30K videos with more than 14 million dense bounding box annotations. Our dataset covers a wide selection of object classes in broad and diverse context. By releasing such a large-scale dataset, we expect deep trackers to further improve and generalize. In addition, we introduce a new benchmark composed of 500 novel videos, modeled with a distribution similar to our training dataset. By sequestering the annotation of the test set and providing an online evaluation server, we provide a fair benchmark for future development of object trackers. Deep trackers fine-tuned on a fraction of our dataset improve their performance by up to 1.6% on OTB100 and up to 1.7% on TrackingNet Test. We provide an extensive benchmark on TrackingNet by evaluating more than 20 trackers. Our results suggest that object tracking in the wild is far from being solved.
    • UE4Sim: A Photo-Realistic Simulator for Computer Vision Applications

      Mueller, Matthias; Casser, Vincent; Lahoud, Jean; Smith, Neil; Ghanem, Bernard (arXiv, 2017-08-19)
      We present a photo-realistic training and evaluation simulator (UE4Sim) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network (DNN) architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.
    • The ultra-sensitive Nodewalk technique identifies stochastic from virtual, population-based enhancer hubs regulating MYC in 3D: Implications for the fitness of cancer cells

      Sumida, Noriyuki; Sifakis, Emmanouil; Scholz, Barbara A; Fernandez Woodbridge, Alejandro; Kiani, Narsis A.; Gomez-Cabrero, David; Svensson, J Peter; Tegner, Jesper; Gondor, Anita; Ohlsson, Rolf (Cold Spring Harbor Laboratory, 2018-03-27)
      The relationship between stochastic transcriptional bursts and dynamic 3D chromatin states is not well understood due to poor sensitivity and/or resolution of current chromatin structure-based assays. Consequently, it is not well established if enhancers operate individually and/or in clusters to coordinate gene transcription. In the current study, we introduce Nodewalk, which uniquely combines high sensitivity with high resolution to enable the analysis of chromatin networks in minute input material. The >10,000-fold increase in sensitivity over other many-to-all competing methods uncovered that active chromatin hubs identified in large input material, corresponding to 10 000 cells, flanking the MYC locus are primarily virtual. Thus, the close agreement between chromatin interactomes generated from aliquots corresponding to less than 10 cells with randomly re-sampled interactomes, we find that numerous distal enhancers positioned within flanking topologically associating domains (TADs) converge on MYC in largely mutually exclusive manners. Moreover, when comparing with several enhancer baits, the assignment of the MYC locus as the node with the highest dynamic importance index, indicates that it is MYC targeting its enhancers, rather than vice versa. Dynamic changes in the configuration of the boundary between TADs flanking MYC underlie numerous stochastic encounters with a diverse set of enhancers to depict the plasticity of its transcriptional regulation. Such an arrangement might increase the fitness of the cancer cell by increasing the probability of MYC transcription in response to a wide range of environmental cues encountered by the cell during the neoplastic process.
    • Underwater Optical Wireless Communications, Networking, and Localization: A Survey

      Saeed, Nasir; Celik, Abdulkadir; Al-Naffouri, Tareq Y.; Alouini, Mohamed-Slim (arXiv, 2018-02-28)
      Underwater wireless communications can be carried out through acoustic, radio frequency (RF), and optical waves. Compared to its bandwidth limited acoustic and RF counterparts, underwater optical wireless communications (UOWCs) can support higher data rates at low latency levels. However, severe aquatic channel conditions (e.g., absorption, scattering, turbulence, etc.) pose great challenges for UOWCs and significantly reduce the attainable communication ranges, which necessitates efficient networking and localization solutions. Therefore, we provide a comprehensive survey on the challenges, advances, and prospects of underwater optical wireless networks (UOWNs) from a layer by layer perspective which includes: 1) Potential network architectures; 2) Physical layer issues including propagation characteristics, channel modeling, and modulation techniques 3) Data link layer problems covering link configurations, link budgets, performance metrics, and multiple access schemes; 4) Network layer topics containing relaying techniques and potential routing algorithms; 5) Transport layer subjects such as connectivity, reliability, flow and congestion control; 6) Application layer goals and state-of-the-art UOWN applications, and 7) Localization and its impacts on UOWN layers. Finally, we outline the open research challenges and point out the future directions for underwater optical wireless communications, networking, and localization research.
    • Unemployment estimation: Spatial point referenced methods and models

      Pereira, Soraia; Turkman, Kamil Feridun; Correia, Luis; Rue, Haavard (arXiv, 2017-06-26)
      Portuguese Labor force survey, from 4th quarter of 2014 onwards, started geo-referencing the sampling units, namely the dwellings in which the surveys are carried. This opens new possibilities in analysing and estimating unemployment and its spatial distribution across any region. The labor force survey choose, according to an preestablished sampling criteria, a certain number of dwellings across the nation and survey the number of unemployed in these dwellings. Based on this survey, the National Statistical Institute of Portugal presently uses direct estimation methods to estimate the national unemployment figures. Recently, there has been increased interest in estimating these figures in smaller areas. Direct estimation methods, due to reduced sampling sizes in small areas, tend to produce fairly large sampling variations therefore model based methods, which tend to
    • Unidirectional Magnon-Driven Domain Wall Motion due to Interfacial Dzyaloshinskii-Moriya Interaction

      Lee, Seo-Won; Kim, Kyoung-Whan; Moon, Jung-Hwan; Go, Gyungchoon; Manchon, Aurelien; Lee, Hyun-Woo; Everschor-Sitte, Karin; Lee, Kyung-Jin (arXiv, 2018-03-28)
      We theoretically study magnon-driven motion of a tranverse domain wall in the presence of interfacial Dzyaloshinskii-Moriya interaction (DMI). Contrary to previous studies, the domain wall moves along the same direction regardless of the magnon-flow direction. Our symmetry analysis reveals that the odd order DMI contributions to the domain wall velocity are independent of the magnon-flow direction. Corresponding DMI-induced asymmetric transitions from a spin-wave state to another give rise to a large momentum transfer to the domain wall without nonreciprocity and much reflection. This counterintuitive unidirectional motion occurs not only for a spin wave with a single wavevector but also for thermal magnons with distributed wavevectors.
    • Using Multi-Spectral UAV Imagery to Extract Tree Crop Structural Properties and Assess Pruning Effects

      Johansen, Kasper; Raharjo, Tri; McCabe, Matthew (MDPI AG, 2018-04-18)
      Unmanned aerial vehicles (UAV) provide an unprecedented capacity to monitor the development and dynamics of tree growth and structure through time. It is generally thought that the pruning of tree crops encourages new growth, has a positive effect on fruiting, makes fruit-picking easier, and may increase yield, as it increases light interception and tree crown surface area. To establish the response of pruning in an orchard of lychee trees, an assessment of changes in tree structure, i.e. tree crown perimeter, width, height, area and Plant Projective Cover (PPC), was undertaken using multi-spectral UAV imagery collected before and after a pruning event. While tree crown perimeter, width and area could be derived directly from the delineated tree crowns, height was estimated from a produced canopy height model and PPC was most accurately predicted based on the NIR band. Pre- and post-pruning results showed significant differences in all measured tree structural parameters, including an average decrease in tree crown perimeter of 1.94 m, tree crown width of 0.57 m, tree crown height of 0.62 m, tree crown area of 3.5 m2, and PPC of 14.8%. In order to provide guidance on data collection protocols for orchard management, the impact of flying height variations was also examined, offering some insight into the influence of scale and the scalability of this UAV based approach for larger orchards. The different flying heights (i.e. 30, 50 and 70 m) produced similar measurements of tree crown width and PPC, while tree crown perimeter, area and height measurements decreased with increasing flying height. Overall, these results illustrate that routine collection of multi-spectral UAV imagery can provide a means of assessing pruning effects on changes in tree structure in commercial orchards, and highlight the importance of collecting imagery with consistent flight configurations, as varying flying heights may cause changes to tree structural measurements.
    • VQABQ: Visual Question Answering by Basic Questions

      Huang, Jia-Hong; Alfadly, Modar; Ghanem, Bernard (arXiv, 2017-03-19)
      Taking an image and question as the input of our method, it can output the text-based answer of the query question about the given image, so called Visual Question Answering (VQA). There are two main modules in our algorithm. Given a natural language question about an image, the first module takes the question as input and then outputs the basic questions of the main given question. The second module takes the main question, image and these basic questions as input and then outputs the text-based answer of the main question. We formulate the basic questions generation problem as a LASSO optimization problem, and also propose a criterion about how to exploit these basic questions to help answer main question. Our method is evaluated on the challenging VQA dataset and yields state-of-the-art accuracy, 60.34% in open-ended task.
    • Weakly intrusive low-rank approximation method for nonlinear parameter-dependent equations

      Giraldi, Loic; Nouy, Anthony (arXiv, 2017-06-30)
      This paper presents a weakly intrusive strategy for computing a low-rank approximation of the solution of a system of nonlinear parameter-dependent equations. The proposed strategy relies on a Newton-like iterative solver which only requires evaluations of the residual of the parameter-dependent equation and of a preconditioner (such as the differential of the residual) for instances of the parameters independently. The algorithm provides an approximation of the set of solutions associated with a possibly large number of instances of the parameters, with a computational complexity which can be orders of magnitude lower than when using the same Newton-like solver for all instances of the parameters. The reduction of complexity requires efficient strategies for obtaining low-rank approximations of the residual, of the preconditioner, and of the increment at each iteration of the algorithm. For the approximation of the residual and the preconditioner, weakly intrusive variants of the empirical interpolation method are introduced, which require evaluations of entries of the residual and the preconditioner. Then, an approximation of the increment is obtained by using a greedy algorithm for low-rank approximation, and a low-rank approximation of the iterate is finally obtained by using a truncated singular value decomposition. When the preconditioner is the differential of the residual, the proposed algorithm is interpreted as an inexact Newton solver for which a detailed convergence analysis is provided. Numerical examples illustrate the efficiency of the method.
    • Weighted Low-Rank Approximation of Matrices and Background Modeling

      Dutta, Aritra; Li, Xin; Richtarik, Peter (arXiv, 2018-04-15)
      We primarily study a special a weighted low-rank approximation of matrices and then apply it to solve the background modeling problem. We propose two algorithms for this purpose: one operates in the batch mode on the entire data and the other one operates in the batch-incremental mode on the data and naturally captures more background variations and computationally more effective. Moreover, we propose a robust technique that learns the background frame indices from the data and does not require any training frames. We demonstrate through extensive experiments that by inserting a simple weight in the Frobenius norm, it can be made robust to the outliers similar to the $\ell_1$ norm. Our methods match or outperform several state-of-the-art online and batch background modeling methods in virtually all quantitative and qualitative measures.
    • Wetting of Water on Graphene

      Bera, Bijoyendra; Shahidzadeh, Noushine; Mishra, Himanshu; Bonn, Daniel (arXiv, 2016-11-28)
      The wetting properties of graphene have proven controversial and difficult to assess. The presence of a graphene layer on top of a substrate does not significantly change the wetting properties of the solid substrate, suggesting that a single graphene layer does not affect the adhesion between the wetting phase and the substrate. However, wetting experiments of water on graphene show contact angles that imply a large amount of adhesion. Here, we investigate the wetting of graphene by measuring the mass of water vapor adsorbing to graphene flakes of different thickness at different relative humidities. Our experiments unambiguously show that the thinnest of graphene flakes do not adsorb water, from which it follows that the contact angle of water on these flakes is ~180o. Thicker flakes of graphene nanopowder, on the other hand, do adsorb water. A calculation of the van der Waals (vdW) interactions that dominate the adsorption in this system confirms that the adhesive interactions between a single atomic layer of graphene and water are so weak that graphene is superhydrophobic. The observations are confirmed in an independent experiment on graphene-coated water droplets that shows that it is impossible to make liquid 'marbles' with molecularly thin graphene.