Show simple item record

dc.contributor.authorDeng, Zeyu
dc.contributor.authorKammoun, Abla
dc.contributor.authorThrampoulidis, Christos
dc.date.accessioned2019-12-23T12:10:20Z
dc.date.available2019-12-23T12:10:20Z
dc.date.issued2019-11-13
dc.identifier.urihttp://hdl.handle.net/10754/660756
dc.description.abstractWe consider a model for logistic regression where only a subset of features of size $p$ is used for training a linear classifier over $n$ training samples. The classifier is obtained by running gradient-descent (GD) on the logistic-loss. For this model, we investigate the dependence of the generalization error on the overparameterization ratio $\kappa=p/n$. First, building on known deterministic results on convergence properties of the GD, we uncover a phase-transition phenomenon for the case of Gaussian regressors: the generalization error of GD is the same as that of the maximum-likelihood (ML) solution when $\kappa<\kappa_\star$, and that of the max-margin (SVM) solution when $\kappa>\kappa_\star$. Next, using the convex Gaussian min-max theorem (CGMT), we sharply characterize the performance of both the ML and SVM solutions. Combining these results, we obtain curves that explicitly characterize the generalization error of GD for varying values of $\kappa$. The numerical results validate the theoretical predictions and unveil double-descent phenomena that complement similar recent observations in linear regression settings.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/1911.05822
dc.rightsArchived with thanks to arXiv
dc.titleA Model of Double Descent for High-dimensional Binary Linear Classification
dc.typePreprint
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.contributor.institutionElectrical and Computer Engineering Department at the University of California, Santa Barbara, USA
dc.identifier.arxivid1911.05822
kaust.personKammoun, Abla
refterms.dateFOA2019-12-23T12:10:51Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
452.7Kb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record