Show simple item record

dc.contributor.authorHanzely, Filip
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2020-11-18T12:43:11Z
dc.date.available2020-11-18T12:43:11Z
dc.date.issued2020-02-10
dc.identifier.urihttp://hdl.handle.net/10754/666018
dc.description.abstractWe propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees. Notably, our methods are similar but not identical to federated averaging / local SGD, thus shedding some light on the essence of the elusive method. In particular, our methods do not perform full averaging steps and instead merely take steps towards averaging. We argue for the benefits of this new paradigm for federated learning.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/2002.05516
dc.rightsArchived with thanks to arXiv
dc.titleFederated Learning of a Mixture of Global and Local Models
dc.typePreprint
dc.contributor.departmentApplied Mathematics and Computational Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.contributor.departmentComputer Science Program
dc.eprint.versionPre-print
dc.identifier.arxivid2002.05516
kaust.personHanzely, Filip
kaust.personRichtarik, Peter
refterms.dateFOA2020-11-18T12:43:51Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
1.259Mb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record