Show simple item record

dc.contributor.authorQian, Xu
dc.contributor.authorQu, Zheng
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2019-05-29T11:42:34Z
dc.date.available2019-05-29T11:42:34Z
dc.date.issued2019-01-24
dc.identifier.urihttp://hdl.handle.net/10754/653121
dc.description.abstractWe study the problem of minimizing the average of a very large number ofsmooth functions, which is of key importance in training supervised learningmodels. One of the most celebrated methods in this context is the SAGAalgorithm. Despite years of research on the topic, a general-purpose version ofSAGA---one that would include arbitrary importance sampling and minibatchingschemes---does not exist. We remedy this situation and propose a general andflexible variant of SAGA following the {\em arbitrary sampling} paradigm. Weperform an iteration complexity analysis of the method, largely possible due tothe construction of new stochastic Lyapunov functions. We establish linearconvergence rates in the smooth and strongly convex regime, and under aquadratic functional growth condition (i.e., in a regime not assuming strongconvexity). Our rates match those of the primal-dual method Quartz for which anarbitrary sampling analysis is available, which makes a significant steptowards closing the gap in our understanding of complexity of primal and dualmethods for finite sum problems.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/1901.08669
dc.rightsArchived with thanks to arXiv
dc.titleSAGA with Arbitrary Sampling
dc.typePreprint
dc.contributor.departmentComputer Science
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.contributor.institutionThe University of Hong Kong, Hong Kong
dc.contributor.institutionUniversity of Edinburgh, United Kingdom
dc.contributor.institutionMoscow Institute of Physics and Technology, Russian Federation.
dc.identifier.arxivid1901.08669
kaust.personQian, Xu
kaust.personRichtarik, Peter
dc.versionv1
refterms.dateFOA2019-05-29T11:42:46Z


Files in this item

Thumbnail
Name:
1901.08669.pdf
Size:
410.6Kb
Format:
PDF
Description:
Preprint

This item appears in the following Collection(s)

Show simple item record