Mizan: A system for dynamic load balancing in large-scale graph processing

Handle URI:
http://hdl.handle.net/10754/564648
Title:
Mizan: A system for dynamic load balancing in large-scale graph processing
Authors:
Khayyat, Zuhair ( 0000-0003-3650-6997 ) ; Awara, Karim; AlOnazi, Amani A.; Jamjoom, Hani T.; Williams, Daniel W.; Kalnis, Panos ( 0000-0002-5060-1360 )
Abstract:
Pregel [23] was recently introduced as a scalable graph mining system that can provide significant performance improvements over traditional MapReduce implementations. Existing implementations focus primarily on graph partitioning as a preprocessing step to balance computation across compute nodes. In this paper, we examine the runtime characteristics of a Pregel system. We show that graph partitioning alone is insufficient for minimizing end-to-end computation. Especially where data is very large or the runtime behavior of the algorithm is unknown, an adaptive approach is needed. To this end, we introduce Mizan, a Pregel system that achieves efficient load balancing to better adapt to changes in computing needs. Unlike known implementations of Pregel, Mizan does not assume any a priori knowledge of the structure of the graph or behavior of the algorithm. Instead, it monitors the runtime characteristics of the system. Mizan then performs efficient fine-grained vertex migration to balance computation and communication. We have fully implemented Mizan; using extensive evaluation we show that - especially for highly-dynamic workloads - Mizan provides up to 84% improvement over techniques leveraging static graph pre-partitioning. © 2013 ACM.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Computer Science Program
Publisher:
Association for Computing Machinery (ACM)
Journal:
Proceedings of the 8th ACM European Conference on Computer Systems - EuroSys '13
Conference/Event name:
8th ACM European Conference on Computer Systems, EuroSys 2013
Issue Date:
2013
DOI:
10.1145/2465351.2465369
Type:
Conference Paper
ISBN:
9781450319942
Appears in Collections:
Conference Papers; Computer Science Program; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorKhayyat, Zuhairen
dc.contributor.authorAwara, Karimen
dc.contributor.authorAlOnazi, Amani A.en
dc.contributor.authorJamjoom, Hani T.en
dc.contributor.authorWilliams, Daniel W.en
dc.contributor.authorKalnis, Panosen
dc.date.accessioned2015-08-04T07:10:51Zen
dc.date.available2015-08-04T07:10:51Zen
dc.date.issued2013en
dc.identifier.isbn9781450319942en
dc.identifier.doi10.1145/2465351.2465369en
dc.identifier.urihttp://hdl.handle.net/10754/564648en
dc.description.abstractPregel [23] was recently introduced as a scalable graph mining system that can provide significant performance improvements over traditional MapReduce implementations. Existing implementations focus primarily on graph partitioning as a preprocessing step to balance computation across compute nodes. In this paper, we examine the runtime characteristics of a Pregel system. We show that graph partitioning alone is insufficient for minimizing end-to-end computation. Especially where data is very large or the runtime behavior of the algorithm is unknown, an adaptive approach is needed. To this end, we introduce Mizan, a Pregel system that achieves efficient load balancing to better adapt to changes in computing needs. Unlike known implementations of Pregel, Mizan does not assume any a priori knowledge of the structure of the graph or behavior of the algorithm. Instead, it monitors the runtime characteristics of the system. Mizan then performs efficient fine-grained vertex migration to balance computation and communication. We have fully implemented Mizan; using extensive evaluation we show that - especially for highly-dynamic workloads - Mizan provides up to 84% improvement over techniques leveraging static graph pre-partitioning. © 2013 ACM.en
dc.publisherAssociation for Computing Machinery (ACM)en
dc.titleMizan: A system for dynamic load balancing in large-scale graph processingen
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentComputer Science Programen
dc.identifier.journalProceedings of the 8th ACM European Conference on Computer Systems - EuroSys '13en
dc.conference.date15 April 2013 through 17 April 2013en
dc.conference.name8th ACM European Conference on Computer Systems, EuroSys 2013en
dc.conference.locationPragueen
dc.contributor.institutionIBM T. J. Watson Research Center, Yorktown Heights, NY, United Statesen
kaust.authorAlOnazi, Amani A.en
kaust.authorKalnis, Panosen
kaust.authorKhayyat, Zuhairen
kaust.authorAwara, Karimen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.