dc.contributor.author Doikov, Nikita dc.contributor.author Richtarik, Peter dc.date.accessioned 2018-02-22T10:34:42Z dc.date.available 2018-02-22T10:34:42Z dc.date.issued 2018-02-12 dc.identifier.uri http://hdl.handle.net/10754/627181 dc.description.abstract We study the problem of minimizing the sum of three convex functions: a differentiable, twice-differentiable and a non-smooth term in a high dimensional setting. To this effect we propose and analyze a randomized block cubic Newton (RBCN) method, which in each iteration builds a model of the objective function formed as the sum of the natural models of its three components: a linear model with a quadratic regularizer for the differentiable term, a quadratic model with a cubic regularizer for the twice differentiable term, and perfect (proximal) model for the nonsmooth term. Our method in each iteration minimizes the model over a random subset of blocks of the search variable. RBCN is the first algorithm with these properties, generalizing several existing methods, matching the best known bounds in all special cases. We establish ${\cal O}(1/\epsilon)$, ${\cal O}(1/\sqrt{\epsilon})$ and ${\cal O}(\log (1/\epsilon))$ rates under different assumptions on the component functions. Lastly, we show numerically that our method outperforms the state-of-the-art on a variety of machine learning problems, including cubically regularized least-squares, logistic regression with constraints, and Poisson regression. dc.publisher Proceedings of Machine Learning Research dc.relation.url http://proceedings.mlr.press/v80/doikov18a.html dc.rights Archived with thanks to Proceedings of Machine Learning Research dc.title Randomized Block Cubic Newton Method dc.type Conference Paper dc.contributor.department Computer Science Program dc.contributor.department Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division dc.conference.name Proceedings of the 35th International Conference on Machine Learning dc.conference.location Stockholm, Sweden dc.eprint.version Pre-print dc.contributor.institution National Research University Higher School of Economics, Moscow, Russia dc.contributor.institution Moscow Institute of Physics and Technology, Dolgoprudny, Russia. dc.contributor.institution University of Edinburgh,Edinburgh, United Kingdom dc.identifier.arxivid arXiv:1802.04084 kaust.person Richtarik, Peter refterms.dateFOA 2018-06-14T05:49:01Z
﻿

Name:
1802.04084v1.pdf
Size:
733.3Kb
Format:
PDF
Description:
Preprint