Simplifying Scalable Graph Processing with a Domain-Specific Language
Permanent link to this recordhttp://hdl.handle.net/10754/599620
MetadataShow full item record
AbstractLarge-scale graph processing, with its massive data sets, requires distributed processing. However, conventional frameworks for distributed graph processing, such as Pregel, use non-traditional programming models that are well-suited for parallelism and scalability but inconvenient for implementing non-trivial graph algorithms. In this paper, we use Green-Marl, a Domain-Specific Language for graph analysis, to intuitively describe graph algorithms and extend its compiler to generate equivalent Pregel implementations. Using the semantic information captured by Green-Marl, the compiler applies a set of transformation rules that convert imperative graph algorithms into Pregel's programming model. Our experiments show that the Pregel programs generated by the Green-Marl compiler perform similarly to manually coded Pregel implementations of the same algorithms. The compiler is even able to generate a Pregel implementation of a complicated graph algorithm for which a manual Pregel implementation is very challenging.
CitationHong S, Salihoglu S, Widom J, Olukotun K (2014) Simplifying Scalable Graph Processing with a Domain-Specific Language. Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization - CGO ’14. Available: http://dx.doi.org/10.1145/2581122.2544162.
SponsorsThis work was funded by DARPA Contract, Xgraphs; Languageand Algorithms for Heterogeneous Graph Streams, FA8750-12-2-0335; Army contract AHPCRC W911NF-07-2-0027-1; the Na-tional Science Foundation (IIS-0904497) and a KAUST researchgrant; Stanford PPL affiliates program, Pervasive Parallelism Lab:Oracle, AMD, Intel, NVIDIA, and Huawei. Authors also acknowl-edge additional support from Oracle.
JournalProceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization - CGO '14