Show simple item record

dc.contributor.authorKhaled, Ahmed
dc.contributor.authorMishchenko, Konstantin
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2021-09-01T13:24:31Z
dc.date.available2021-09-01T13:24:31Z
dc.date.issued2020
dc.identifier.issn2640-3498
dc.identifier.urihttp://hdl.handle.net/10754/670896
dc.description.abstractWe provide a new analysis of local SGD, removing unnecessary assumptions and elaborating on the difference between two data regimes: identical and heterogeneous. In both cases, we improve the existing theory and provide values of the optimal stepsize and optimal number of local iterations. Our bounds are based on a new notion of variance that is specific to local SGD methods with different data. The tightness of our results is guaranteed by recovering known statements when we plug H “ 1, where H is the number of local steps. The empirical evidence further validates the severe impact of data heterogeneity on the performance of local SGD.
dc.relation.urlhttps://arxiv.org/pdf/1909.04746.pdf
dc.titleTighter Theory for Local SGD on Identical and Heterogeneous Data
dc.typeConference Paper
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Science and Engineering (CEMSE) Division
dc.conference.nameNeurIPS 2019 Federated Learning Workshop
dc.identifier.wosutWOS:000559931300027
dc.eprint.versionPost-print
dc.contributor.institutionCairo University
dc.identifier.volume108
dc.identifier.pages4519-4528
kaust.personKhaled, Ahmed
kaust.personMishchenko, Konstantin
kaust.personRichtarik, Peter


This item appears in the following Collection(s)

Show simple item record