In Situ Visualization of WRF Data Using Universal Data Junction

Type
Conference Paper
Authors
Esposito, Aniello cc
Holst, Glendon
KAUST Department
Biological and Environmental Science and Engineering (BESE) Division
Visualization
Date
2021-11-13
Permanent link to this record
http://hdl.handle.net/10754/673905

Metadata
Show full item record
Abstract
An in situ co-processing visualization pipeline based on the Universal Data Junction (UDJ) library and Inshimtu is presented and used for processing data from Weather Research and Forecasting (WRF) simulations. For the common case of analyzing just a number of fields during simulation, UDJ transfers and redistributes the data in approximately 6 % of the time needed by WRF for a MPI-IO output of all variables upon which a previous method with Inshimtu is based. The relative cost of transport and redistribution compared to IO remains approximately constant up to the highest considered node count without obvious impediments to scale further.
Citation
Esposito, A., & Holst, G. (2021). In Situ Visualization of WRF Data Using Universal Data Junction. High Performance Computing, 475–483. doi:10.1007/978-3-030-90539-2_32
Sponsors
This work is part of the HPE/Cray center of excellence collaboration at KAUST. UDJ development has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 773897. We want to thank Hari Dasari colleagues for helping with the test case as well as Tim Dykes and Utz Uwe Haus from the HPE EMEA research lab for support with UDJ.
Publisher
Springer International Publishing
Conference/Event name
International Conference on High Performance Computing, ISC High Performance 2021
ISBN
9783030905385
DOI
10.1007/978-3-030-90539-2_32
Additional Links
https://link.springer.com/10.1007/978-3-030-90539-2_32
Collections
Conference Papers; Biological and Environmental Science and Engineering (BESE) Division