Scientific Applications Performance Evaluation on Burst Buffer

Handle URI:
http://hdl.handle.net/10754/625936
Title:
Scientific Applications Performance Evaluation on Burst Buffer
Authors:
Markomanolis, George S.; Hadri, Bilel; Khurram, Rooh Ul Amin; Feki, Saber
Abstract:
Parallel I/O is an integral component of modern high performance computing, especially in storing and processing very large datasets, such as the case of seismic imaging, CFD, combustion and weather modeling. The storage hierarchy includes nowadays additional layers, the latest being the usage of SSD-based storage as a Burst Buffer for I/O acceleration. We present an in-depth analysis on how to use Burst Buffer for specific cases and how the internal MPI I/O aggregators operate according to the options that the user provides during his job submission. We analyze the performance of a range of I/O intensive scientific applications, at various scales on a large installation of Lustre parallel file system compared to an SSD-based Burst Buffer. Our results show a performance improvement over Lustre when using Burst Buffer. Moreover, we show results from a data hierarchy library which indicate that the standard I/O approaches are not enough to get the expected performance from this technology. The performance gain on the total execution time of the studied applications is between 1.16 and 3 times compared to Lustre. One of the test cases achieved an impressive I/O throughput of 900 GB/s on Burst Buffer.
KAUST Department:
Supercomputing Core Laboratory
Citation:
Markomanolis GS, Hadri B, Khurram R, Feki S (2017) Scientific Applications Performance Evaluation on Burst Buffer. High Performance Computing: 701–711. Available: http://dx.doi.org/10.1007/978-3-319-67630-2_50.
Publisher:
Springer International Publishing
Journal:
Lecture Notes in Computer Science
Issue Date:
19-Oct-2017
DOI:
10.1007/978-3-319-67630-2_50
Type:
Book Chapter
ISSN:
0302-9743; 1611-3349
Sponsors:
For computer time, this research used the resources of the Supercomputing Core Laboratory at King Abdullah University of Science & Technology (KAUST) in Thuwal, Saudi Arabia.
Additional Links:
https://link.springer.com/chapter/10.1007%2F978-3-319-67630-2_50
Appears in Collections:
Book Chapters

Full metadata record

DC FieldValue Language
dc.contributor.authorMarkomanolis, George S.en
dc.contributor.authorHadri, Bilelen
dc.contributor.authorKhurram, Rooh Ul Aminen
dc.contributor.authorFeki, Saberen
dc.date.accessioned2017-10-24T10:54:56Z-
dc.date.available2017-10-24T10:54:56Z-
dc.date.issued2017-10-19en
dc.identifier.citationMarkomanolis GS, Hadri B, Khurram R, Feki S (2017) Scientific Applications Performance Evaluation on Burst Buffer. High Performance Computing: 701–711. Available: http://dx.doi.org/10.1007/978-3-319-67630-2_50.en
dc.identifier.issn0302-9743en
dc.identifier.issn1611-3349en
dc.identifier.doi10.1007/978-3-319-67630-2_50en
dc.identifier.urihttp://hdl.handle.net/10754/625936-
dc.description.abstractParallel I/O is an integral component of modern high performance computing, especially in storing and processing very large datasets, such as the case of seismic imaging, CFD, combustion and weather modeling. The storage hierarchy includes nowadays additional layers, the latest being the usage of SSD-based storage as a Burst Buffer for I/O acceleration. We present an in-depth analysis on how to use Burst Buffer for specific cases and how the internal MPI I/O aggregators operate according to the options that the user provides during his job submission. We analyze the performance of a range of I/O intensive scientific applications, at various scales on a large installation of Lustre parallel file system compared to an SSD-based Burst Buffer. Our results show a performance improvement over Lustre when using Burst Buffer. Moreover, we show results from a data hierarchy library which indicate that the standard I/O approaches are not enough to get the expected performance from this technology. The performance gain on the total execution time of the studied applications is between 1.16 and 3 times compared to Lustre. One of the test cases achieved an impressive I/O throughput of 900 GB/s on Burst Buffer.en
dc.description.sponsorshipFor computer time, this research used the resources of the Supercomputing Core Laboratory at King Abdullah University of Science & Technology (KAUST) in Thuwal, Saudi Arabia.en
dc.publisherSpringer International Publishingen
dc.relation.urlhttps://link.springer.com/chapter/10.1007%2F978-3-319-67630-2_50en
dc.rightsThe final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-67630-2_50en
dc.subjectDataWarpen
dc.subjectI/Oen
dc.subjectBurst Bufferen
dc.titleScientific Applications Performance Evaluation on Burst Bufferen
dc.typeBook Chapteren
dc.contributor.departmentSupercomputing Core Laboratoryen
dc.identifier.journalLecture Notes in Computer Scienceen
dc.eprint.versionPost-printen
kaust.authorMarkomanolis, George S.en
kaust.authorHadri, Bilelen
kaust.authorKhurram, Rooh Ul Aminen
kaust.authorFeki, Saberen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.