Towards automatic parameter tuning of stream processing systems

Handle URI:
http://hdl.handle.net/10754/626125
Title:
Towards automatic parameter tuning of stream processing systems
Authors:
Bilal, Muhammad; Canini, Marco ( 0000-0002-5051-4283 )
Abstract:
Optimizing the performance of big-data streaming applications has become a daunting and time-consuming task: parameters may be tuned from a space of hundreds or even thousands of possible configurations. In this paper, we present a framework for automating parameter tuning for stream-processing systems. Our framework supports standard black-box optimization algorithms as well as a novel gray-box optimization algorithm. We demonstrate the multiple benefits of automated parameter tuning in optimizing three benchmark applications in Apache Storm. Our results show that a hill-climbing algorithm that uses a new heuristic sampling approach based on Latin Hypercube provides the best results. Our gray-box algorithm provides comparable results while being two to five times faster.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Computer Science Program
Citation:
Bilal M, Canini M (2017) Towards automatic parameter tuning of stream processing systems. Proceedings of the 2017 Symposium on Cloud Computing - SoCC ’17. Available: http://dx.doi.org/10.1145/3127479.3127492.
Publisher:
ACM Press
Journal:
Proceedings of the 2017 Symposium on Cloud Computing - SoCC '17
Conference/Event name:
2017 Symposium on Cloud Computing, SoCC 2017
Issue Date:
27-Sep-2017
DOI:
10.1145/3127479.3127492
Type:
Conference Paper
Sponsors:
Muhammad Bilal was supported by a fellowship from the Erasmus Mundus Joint Doctorate in Distributed Computing (EMJD-DC) program funded by the European Commission (EACEA) (FPA 2012-0030).
Additional Links:
https://dl.acm.org/citation.cfm?doid=3127479.3127492
Appears in Collections:
Conference Papers; Computer Science Program; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorBilal, Muhammaden
dc.contributor.authorCanini, Marcoen
dc.date.accessioned2017-11-06T07:09:05Z-
dc.date.available2017-11-06T07:09:05Z-
dc.date.issued2017-09-27en
dc.identifier.citationBilal M, Canini M (2017) Towards automatic parameter tuning of stream processing systems. Proceedings of the 2017 Symposium on Cloud Computing - SoCC ’17. Available: http://dx.doi.org/10.1145/3127479.3127492.en
dc.identifier.doi10.1145/3127479.3127492en
dc.identifier.urihttp://hdl.handle.net/10754/626125-
dc.description.abstractOptimizing the performance of big-data streaming applications has become a daunting and time-consuming task: parameters may be tuned from a space of hundreds or even thousands of possible configurations. In this paper, we present a framework for automating parameter tuning for stream-processing systems. Our framework supports standard black-box optimization algorithms as well as a novel gray-box optimization algorithm. We demonstrate the multiple benefits of automated parameter tuning in optimizing three benchmark applications in Apache Storm. Our results show that a hill-climbing algorithm that uses a new heuristic sampling approach based on Latin Hypercube provides the best results. Our gray-box algorithm provides comparable results while being two to five times faster.en
dc.description.sponsorshipMuhammad Bilal was supported by a fellowship from the Erasmus Mundus Joint Doctorate in Distributed Computing (EMJD-DC) program funded by the European Commission (EACEA) (FPA 2012-0030).en
dc.publisherACM Pressen
dc.relation.urlhttps://dl.acm.org/citation.cfm?doid=3127479.3127492en
dc.rightsArchived with thanks to Proceedings of the 2017 Symposium on Cloud Computing - SoCC '17en
dc.titleTowards automatic parameter tuning of stream processing systemsen
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentComputer Science Programen
dc.identifier.journalProceedings of the 2017 Symposium on Cloud Computing - SoCC '17en
dc.conference.date2017-09-24 to 2017-09-27en
dc.conference.name2017 Symposium on Cloud Computing, SoCC 2017en
dc.conference.locationSanta Clara, CA, USAen
dc.eprint.versionPublisher's Version/PDFen
dc.contributor.institutionUniversité catholique de Louvainen
kaust.authorCanini, Marcoen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.