Handle URI:
http://hdl.handle.net/10754/599105
Title:
Optimizing graph algorithms on pregel-like systems
Authors:
Salihoglu, Semih; Widom, Jennifer
Abstract:
We study the problem of implementing graph algorithms efficiently on Pregel-like systems, which can be surprisingly challenging. Standard graph algorithms in this setting can incur unnecessary inefficiencies such as slow convergence or high communication or computation cost, typically due to structural properties of the input graphs such as large diameters or skew in component sizes. We describe several optimization techniques to address these inefficiencies. Our most general technique is based on the idea of performing some serial computation on a tiny fraction of the input graph, complementing Pregel's vertex-centric parallelism. We base our study on thorough implementations of several fundamental graph algorithms, some of which have, to the best of our knowledge, not been implemented on Pregel-like systems before. The algorithms and optimizations we describe are fully implemented in our open-source Pregel implementation. We present detailed experiments showing that our optimization techniques improve runtime significantly on a variety of very large graph datasets.
Citation:
Salihoglu S, Widom J (2014) Optimizing graph algorithms on pregel-like systems. Proceedings of the VLDB Endowment 7: 577–588. Available: http://dx.doi.org/10.14778/2732286.2732294.
Publisher:
VLDB Endowment
Journal:
Proceedings of the VLDB Endowment
Issue Date:
1-Mar-2014
DOI:
10.14778/2732286.2732294
Type:
Article
ISSN:
2150-8097
Sponsors:
This work was supported by the National Science Foundation(IIS-0904497), the Boeing Corporation, a KAUST research grant,and a research grant from Amazon Web Services.
Appears in Collections:
Publications Acknowledging KAUST Support

Full metadata record

DC FieldValue Language
dc.contributor.authorSalihoglu, Semihen
dc.contributor.authorWidom, Jenniferen
dc.date.accessioned2016-02-25T13:52:58Zen
dc.date.available2016-02-25T13:52:58Zen
dc.date.issued2014-03-01en
dc.identifier.citationSalihoglu S, Widom J (2014) Optimizing graph algorithms on pregel-like systems. Proceedings of the VLDB Endowment 7: 577–588. Available: http://dx.doi.org/10.14778/2732286.2732294.en
dc.identifier.issn2150-8097en
dc.identifier.doi10.14778/2732286.2732294en
dc.identifier.urihttp://hdl.handle.net/10754/599105en
dc.description.abstractWe study the problem of implementing graph algorithms efficiently on Pregel-like systems, which can be surprisingly challenging. Standard graph algorithms in this setting can incur unnecessary inefficiencies such as slow convergence or high communication or computation cost, typically due to structural properties of the input graphs such as large diameters or skew in component sizes. We describe several optimization techniques to address these inefficiencies. Our most general technique is based on the idea of performing some serial computation on a tiny fraction of the input graph, complementing Pregel's vertex-centric parallelism. We base our study on thorough implementations of several fundamental graph algorithms, some of which have, to the best of our knowledge, not been implemented on Pregel-like systems before. The algorithms and optimizations we describe are fully implemented in our open-source Pregel implementation. We present detailed experiments showing that our optimization techniques improve runtime significantly on a variety of very large graph datasets.en
dc.description.sponsorshipThis work was supported by the National Science Foundation(IIS-0904497), the Boeing Corporation, a KAUST research grant,and a research grant from Amazon Web Services.en
dc.publisherVLDB Endowmenten
dc.titleOptimizing graph algorithms on pregel-like systemsen
dc.typeArticleen
dc.identifier.journalProceedings of the VLDB Endowmenten
dc.contributor.institutionStanford Universityen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.