DDHH: A decentralized deep learning framework for large-scale heterogeneous networks
KAUST DepartmentComputer Science Program
Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division
Machine Intelligence & kNowledge Engineering Lab
Permanent link to this recordhttp://hdl.handle.net/10754/670753
MetadataShow full item record
AbstractLearning vector representations (i.e., embeddings) of nodes for graph-structured information network has attracted vast interest from both industry and academia. Most real-world networks exhibit a complex and heterogeneous format, enclosing high-order relationships and rich semantic information among nodes. However, existing heterogeneous network embedding (HNE) frameworks are commonly designed in a centralized fashion, i.e., all the data storage and learning process take place on a single machine. Hence, those HNE methods show severe performance bottlenecks when handling large-scale networks due to high consumption on memory, storage, and running time. In light of this, to cope with large-scale HNE tasks with strong efficiency and effectiveness guarantee, we propose Decentralized Deep Heterogeneous Hypergraph (DDHH) embedding framework in this paper. In DDHH, we innovatively formulate a large heterogeneous network as a hypergraph, where its hyperedges can connect a set of semantically similar nodes. Our framework then intelligently partitions the heterogeneous network using the identified hyperedges. Then, each resulted subnetwork is assigned to a distributed worker, which employs the deep information maximization theorem to locally learn node embeddings from the partition received. We further devise a novel embedding alignment scheme to precisely project independently learned node embeddings from all subnetworks onto a public vector space, thus allowing for downstream tasks. As shown from our experimental results, DDHH significantly improves the efficiency and accuracy of existing HNE models, and can easily scale up to large-scale heterogeneous networks.
CitationImran, M., Yin, H., Chen, T., Huang, Z., Zhang, X., & Zheng, K. (2021). DDHH: A Decentralized Deep Learning Framework for Large-scale Heterogeneous Networks. 2021 IEEE 37th International Conference on Data Engineering (ICDE). doi:10.1109/icde51399.2021.00196
SponsorsThis work was supported by ARC Discovery Project (GrantNo.DP190101985, DP170103954).
Conference/Event name37th IEEE International Conference on Data Engineering, ICDE 2021