Show simple item record

dc.contributor.authorHanzely, Filip
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2021-06-20T10:35:32Z
dc.date.available2018-04-04T12:38:14Z
dc.date.available2021-06-20T10:35:32Z
dc.date.issued2021-06-09
dc.date.submitted2019-10-09
dc.identifier.citationHanzely, F., & Richtárik, P. (2021). Fastest rates for stochastic mirror descent methods. Computational Optimization and Applications, 79(3), 717–766. doi:10.1007/s10589-021-00284-5
dc.identifier.issn1573-2894
dc.identifier.issn0926-6003
dc.identifier.doi10.1007/s10589-021-00284-5
dc.identifier.urihttp://hdl.handle.net/10754/627407
dc.description.abstractRelative smoothness—a notion introduced in Birnbaum et al. (Proceedings of the 12th ACM conference on electronic commerce, ACM, pp 127–136, 2011) and recently rediscovered in Bauschke et al. (Math Oper Res 330–348, 2016) and Lu et al. (Relatively-smooth convex optimization by first-order methods, and applications, arXiv:1610.05708, 2016)—generalizes the standard notion of smoothness typically used in the analysis of gradient type methods. In this work we are taking ideas from well studied field of stochastic convex optimization and using them in order to obtain faster algorithms for minimizing relatively smooth functions. We propose and analyze two new algorithms: Relative Randomized Coordinate Descent (relRCD) and Relative Stochastic Gradient Descent (relSGD), both generalizing famous algorithms in the standard smooth setting. The methods we propose can be in fact seen as particular instances of stochastic mirror descent algorithms, which has been usually analyzed under stronger assumptions: Lipschitzness of the objective and strong convexity of the reference function. As a consequence, one of the proposed methods, relRCD corresponds to the first stochastic variant of mirror descent algorithm with linear convergence rate.
dc.publisherSpringer Science and Business Media LLC
dc.relation.urlhttps://link.springer.com/10.1007/s10589-021-00284-5
dc.rightsArchived with thanks to Computational Optimization and Applications
dc.titleFastest rates for stochastic mirror descent methods
dc.typeArticle
dc.contributor.departmentApplied Mathematics and Computational Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.contributor.departmentComputer Science Program
dc.identifier.journalComputational Optimization and Applications
dc.rights.embargodate2022-06-09
dc.eprint.versionPost-print
dc.contributor.institutionMoscow Institute of Physics and Technology (MIPT), Dolgoprudny, Russia
dc.identifier.volume79
dc.identifier.issue3
dc.identifier.pages717-766
dc.identifier.arxivid1803.07374
kaust.personHanzely, Filip
kaust.personRichtarik, Peter
dc.date.accepted2021-05-19
dc.versionv1
dc.identifier.eid2-s2.0-85107729803
refterms.dateFOA2018-06-13T17:32:27Z


Files in this item

Thumbnail
Name:
1803.07374v1.pdf
Size:
620.2Kb
Format:
PDF
Description:
Preprint

This item appears in the following Collection(s)

Show simple item record

VersionItemEditorDateSummary

*Selected version