Show simple item record

dc.contributor.authorDoikov, Nikita
dc.contributor.authorRichtarik, Peter
dc.date.accessioned2018-02-22T10:34:42Z
dc.date.available2018-02-22T10:34:42Z
dc.date.issued2018-02-12
dc.identifier.urihttp://hdl.handle.net/10754/627181
dc.description.abstractWe study the problem of minimizing the sum of three convex functions: a differentiable, twice-differentiable and a non-smooth term in a high dimensional setting. To this effect we propose and analyze a randomized block cubic Newton (RBCN) method, which in each iteration builds a model of the objective function formed as the sum of the natural models of its three components: a linear model with a quadratic regularizer for the differentiable term, a quadratic model with a cubic regularizer for the twice differentiable term, and perfect (proximal) model for the nonsmooth term. Our method in each iteration minimizes the model over a random subset of blocks of the search variable. RBCN is the first algorithm with these properties, generalizing several existing methods, matching the best known bounds in all special cases. We establish ${\cal O}(1/\epsilon)$, ${\cal O}(1/\sqrt{\epsilon})$ and ${\cal O}(\log (1/\epsilon))$ rates under different assumptions on the component functions. Lastly, we show numerically that our method outperforms the state-of-the-art on a variety of machine learning problems, including cubically regularized least-squares, logistic regression with constraints, and Poisson regression.
dc.publisherProceedings of Machine Learning Research
dc.relation.urlhttp://proceedings.mlr.press/v80/doikov18a.html
dc.rightsArchived with thanks to Proceedings of Machine Learning Research
dc.titleRandomized Block Cubic Newton Method
dc.typeConference Paper
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.conference.nameProceedings of the 35th International Conference on Machine Learning
dc.conference.locationStockholm, Sweden
dc.eprint.versionPre-print
dc.contributor.institutionNational Research University Higher School of Economics, Moscow, Russia
dc.contributor.institutionMoscow Institute of Physics and Technology, Dolgoprudny, Russia.
dc.contributor.institutionUniversity of Edinburgh,Edinburgh, United Kingdom
dc.identifier.arxividarXiv:1802.04084
kaust.personRichtarik, Peter
refterms.dateFOA2018-06-14T05:49:01Z


Files in this item

Thumbnail
Name:
1802.04084v1.pdf
Size:
733.3Kb
Format:
PDF
Description:
Preprint

This item appears in the following Collection(s)

Show simple item record