On differentially private stochastic convex optimization with heavy-Tailed data
dc.contributor.author | Wang, Di | |
dc.contributor.author | Xiao, Hanshen | |
dc.contributor.author | Devadas, Srini | |
dc.contributor.author | Xu, Jinhui | |
dc.date.accessioned | 2021-05-24T07:53:30Z | |
dc.date.available | 2021-05-24T07:53:30Z | |
dc.date.issued | 2020-01-01 | |
dc.identifier.isbn | 9781713821120 | |
dc.identifier.uri | http://hdl.handle.net/10754/669227 | |
dc.description.abstract | In this paper, we consider the problem of de-signing Differentially Private (DP) algorithms for Stochastic Convex Optimization (SCO) on heavy-Tailed data. The irregularity of such data violates some key assumptions used in almost all existing DP-SCO and DP-ERM methods, re-sulting in failure to provide the DP guarantees. To better understand this type of challenges, we provide in this paper a comprehensive study of DP-SCO under various settings. First, we con-sider the case where the loss function is strongly convex and smooth. For this case, we propose a method based on the sample-And-Aggregate framework, which has an excess population risk of O( d3 n4 ) (after omitting other factors), where n is the sample size and d is the dimensional-ity of the data. Then, we show that with some additional assumptions on the loss functions, it is possible to reduce the expected excess popula-tion risk to O ( d2 n2 ). To lift these additional condi-tions, we also provide a gradient smoothing and trimming based scheme to achieve excess popula-tion risks of O ( d2 n2 ) and O( d 2 3 (n2) 13 ) for strongly convex and general convex loss functions, respec-tively, with high probability. Experiments sug-gest that our algorithms can effectively deal with the challenges caused by data irregularity. | |
dc.description.sponsorship | Di Wang and Jinhui Xu were supported in part by the National Science Foundation (NSF) under Grant No. CCF1716400 and IIS-1919492. | |
dc.publisher | International Machine Learning Society (IMLS) | |
dc.rights | Archived with thanks to International Machine Learning Society (IMLS) | |
dc.title | On differentially private stochastic convex optimization with heavy-Tailed data | |
dc.type | Conference Paper | |
dc.contributor.department | Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division | |
dc.conference.date | 2020-07-13 to 2020-07-18 | |
dc.conference.name | 37th International Conference on Machine Learning, ICML 2020 | |
dc.conference.location | Virtual, Online | |
dc.eprint.version | Pre-print | |
dc.contributor.institution | Department of Computer Science and Engineering, State University of New York at Buffalo, Buffalo, NY | |
dc.contributor.institution | CSAIL, MIT, Cambridge, MA | |
dc.identifier.volume | PartF168147-13 | |
dc.identifier.pages | 10023-10033 | |
kaust.person | Wang, Di | |
dc.identifier.eid | 2-s2.0-85105337629 | |
refterms.dateFOA | 2021-05-24T08:09:19Z |