Show simple item record

dc.contributor.authorKovalev, Dmitry
dc.contributor.authorMishchenko, Konstantin
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2019-12-22T12:22:55Z
dc.date.available2019-12-22T12:22:55Z
dc.date.issued2019-12-03
dc.identifier.urihttp://hdl.handle.net/10754/660727
dc.description.abstractWe present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton's method (SN), and the second is a stochastic variant of cubically regularized Newton's method (SCN). We establish local linear-quadratic convergence results. Unlike existing stochastic variants of second order methods, which require the evaluation of a large number of gradients and/or Hessians in each iteration to guarantee convergence, our methods do not have this shortcoming. For instance, the simplest variants of our methods in each iteration need to compute the gradient and Hessian of a {\em single} randomly selected function only. In contrast to most existing stochastic Newton and quasi-Newton methods, our approach guarantees local convergence faster than with first-order oracle and adapts to the problem's curvature. Interestingly, our method is not unbiased, so our theory provides new intuition for designing new stochastic methods.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/1912.01597
dc.rightsArchived with thanks to arXiv
dc.titleStochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates
dc.typePreprint
dc.contributor.departmentKing Abdullah University of Science and Technology, Thuwal, Saudi Arabia
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.identifier.arxivid1912.01597
kaust.personKovalev, Dmitry
kaust.personMishchenko, Konstantin
kaust.personRichtarik, Peter
refterms.dateFOA2019-12-22T12:23:23Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
509.5Kb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record