Show simple item record

dc.contributor.advisorRichtarik, Peter
dc.contributor.authorHanzely, Filip
dc.date.accessioned2020-08-24T08:39:27Z
dc.date.available2020-08-24T08:39:27Z
dc.date.issued2020-08-20
dc.identifier.citationHanzely, F. (2020). Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters. KAUST Research Repository. https://doi.org/10.25781/KAUST-4F2DH
dc.identifier.doi10.25781/KAUST-4F2DH
dc.identifier.urihttp://hdl.handle.net/10754/664789
dc.description.abstractMany key problems in machine learning and data science are routinely modeled as optimization problems and solved via optimization algorithms. With the increase of the volume of data and the size and complexity of the statistical models used to formulate these often ill-conditioned optimization tasks, there is a need for new efficient algorithms able to cope with these challenges. In this thesis, we deal with each of these sources of difficulty in a different way. To efficiently address the big data issue, we develop new methods which in each iteration examine a small random subset of the training data only. To handle the big model issue, we develop methods which in each iteration update a random subset of the model parameters only. Finally, to deal with ill-conditioned problems, we devise methods that incorporate either higher-order information or Nesterov’s acceleration/momentum. In all cases, randomness is viewed as a powerful algorithmic tool that we tune, both in theory and in experiments, to achieve the best results. Our algorithms have their primary application in training supervised machine learning models via regularized empirical risk minimization, which is the dominant paradigm for training such models. However, due to their generality, our methods can be applied in many other fields, including but not limited to data science, engineering, scientific computing, and statistics.
dc.language.isoen
dc.subjectoptimization
dc.subjectmachine learning
dc.subjectstochastic gradient
dc.subjectvariance reduction
dc.subjectcoordinate descent
dc.titleOptimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters
dc.typeDissertation
dc.contributor.departmentComputer, Electrical and Mathematical Science and Engineering (CEMSE) Division
thesis.degree.grantorKing Abdullah University of Science and Technology
dc.contributor.committeememberTempone, Raul
dc.contributor.committeememberGhanem, Bernard
dc.contributor.committeememberWright, Stephen
dc.contributor.committeememberZhang, Tong
thesis.degree.disciplineComputer Science
thesis.degree.nameDoctor of Philosophy
dc.identifier.arxivid2008.11824
refterms.dateFOA2020-08-24T08:39:28Z
kaust.request.doiyes


Files in this item

Thumbnail
Name:
Thesis.pdf
Size:
15.59Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record