Show simple item record

dc.contributor.authorGasanov, Elnur
dc.contributor.authorKhaled, Ahmed
dc.contributor.authorHorvath, Samuel
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2021-11-29T12:10:27Z
dc.date.available2021-11-29T12:10:27Z
dc.date.issued2021-11-22
dc.identifier.urihttp://hdl.handle.net/10754/673826
dc.description.abstractFederated Learning (FL) is an increasingly popular machine learning paradigm in which multiple nodes try to collaboratively learn under privacy, communication and multiple heterogeneity constraints. A persistent problem in federated learning is that it is not clear what the optimization objective should be: the standard average risk minimization of supervised learning is inadequate in handling several major constraints specific to federated learning, such as communication adaptivity and personalization control. We identify several key desiderata in frameworks for federated learning and introduce a new framework, FLIX, that takes into account the unique challenges brought by federated learning. FLIX has a standard finite-sum form, which enables practitioners to tap into the immense wealth of existing (potentially non-local) methods for distributed optimization. Through a smart initialization that does not require any communication, FLIX does not require the use of local steps but is still provably capable of performing dissimilarity regularization on par with local methods. We give several algorithms for solving the FLIX formulation efficiently under communication constraints. Finally, we corroborate our theoretical results with extensive experimentation.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/2111.11556.pdf
dc.rightsArchived with thanks to arXiv
dc.titleFLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
dc.typePreprint
dc.contributor.departmentComputer Science
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Science and Engineering (CEMSE) Division
dc.contributor.departmentStatistics
dc.contributor.departmentStatistics Program
dc.eprint.versionPre-print
dc.identifier.arxivid2111.11556
kaust.personGasanov, Elnur
kaust.personKhaled, Ahmed
kaust.personHorvath, Samuel
kaust.personRichtarik, Peter
refterms.dateFOA2021-11-29T12:12:19Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
12.78Mb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record