Linearly convergent stochastic heavy ball method for minimizing generalization error

Handle URI:
http://hdl.handle.net/10754/626510
Title:
Linearly convergent stochastic heavy ball method for minimizing generalization error
Authors:
Loizou, Nicolas; Richtarik, Peter
Abstract:
In this work we establish the first linear convergence result for the stochastic heavy ball method. The method performs SGD steps with a fixed stepsize, amended by a heavy ball momentum term. In the analysis, we focus on minimizing the expected loss and not on finite-sum minimization, which is typically a much harder problem. While in the analysis we constrain ourselves to quadratic loss, the overall objective is not necessarily strongly convex.
KAUST Department:
KAUST, Kingdom of Saudi Arabia
Publisher:
arXiv
Issue Date:
30-Oct-2017
ARXIV:
arXiv:1710.10737
Type:
Preprint
Additional Links:
http://arxiv.org/abs/1710.10737v1; http://arxiv.org/pdf/1710.10737v1
Appears in Collections:
Other/General Submission; Other/General Submission

Full metadata record

DC FieldValue Language
dc.contributor.authorLoizou, Nicolasen
dc.contributor.authorRichtarik, Peteren
dc.date.accessioned2017-12-28T07:32:14Z-
dc.date.available2017-12-28T07:32:14Z-
dc.date.issued2017-10-30en
dc.identifier.urihttp://hdl.handle.net/10754/626510-
dc.description.abstractIn this work we establish the first linear convergence result for the stochastic heavy ball method. The method performs SGD steps with a fixed stepsize, amended by a heavy ball momentum term. In the analysis, we focus on minimizing the expected loss and not on finite-sum minimization, which is typically a much harder problem. While in the analysis we constrain ourselves to quadratic loss, the overall objective is not necessarily strongly convex.en
dc.publisherarXiven
dc.relation.urlhttp://arxiv.org/abs/1710.10737v1en
dc.relation.urlhttp://arxiv.org/pdf/1710.10737v1en
dc.rightsArchived with thanks to arXiven
dc.titleLinearly convergent stochastic heavy ball method for minimizing generalization erroren
dc.typePreprinten
dc.contributor.departmentKAUST, Kingdom of Saudi Arabiaen
dc.eprint.versionPre-printen
dc.contributor.institutionUniversity of Edinburgh, United Kingdomen
dc.identifier.arxividarXiv:1710.10737en
kaust.authorRichtarik, Peteren
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.