Show simple item record

dc.contributor.authorWang, Di
dc.contributor.authorXiao, Hanshen
dc.contributor.authorDevadas, Srini
dc.contributor.authorXu, Jinhui
dc.date.accessioned2021-05-24T07:53:30Z
dc.date.available2021-05-24T07:53:30Z
dc.date.issued2020-01-01
dc.identifier.isbn9781713821120
dc.identifier.urihttp://hdl.handle.net/10754/669227
dc.description.abstractIn this paper, we consider the problem of de-signing Differentially Private (DP) algorithms for Stochastic Convex Optimization (SCO) on heavy-Tailed data. The irregularity of such data violates some key assumptions used in almost all existing DP-SCO and DP-ERM methods, re-sulting in failure to provide the DP guarantees. To better understand this type of challenges, we provide in this paper a comprehensive study of DP-SCO under various settings. First, we con-sider the case where the loss function is strongly convex and smooth. For this case, we propose a method based on the sample-And-Aggregate framework, which has an excess population risk of O( d3 n4 ) (after omitting other factors), where n is the sample size and d is the dimensional-ity of the data. Then, we show that with some additional assumptions on the loss functions, it is possible to reduce the expected excess popula-tion risk to O ( d2 n2 ). To lift these additional condi-tions, we also provide a gradient smoothing and trimming based scheme to achieve excess popula-tion risks of O ( d2 n2 ) and O( d 2 3 (n2) 13 ) for strongly convex and general convex loss functions, respec-tively, with high probability. Experiments sug-gest that our algorithms can effectively deal with the challenges caused by data irregularity.
dc.description.sponsorshipDi Wang and Jinhui Xu were supported in part by the National Science Foundation (NSF) under Grant No. CCF1716400 and IIS-1919492.
dc.publisherInternational Machine Learning Society (IMLS)
dc.rightsArchived with thanks to International Machine Learning Society (IMLS)
dc.titleOn differentially private stochastic convex optimization with heavy-Tailed data
dc.typeConference Paper
dc.contributor.departmentKing Abdullah University of Science and Technology, Thuwal, Saudi Arabia
dc.conference.date2020-07-13 to 2020-07-18
dc.conference.name37th International Conference on Machine Learning, ICML 2020
dc.conference.locationVirtual, Online
dc.eprint.versionPre-print
dc.contributor.institutionDepartment of Computer Science and Engineering, State University of New York at Buffalo, Buffalo, NY
dc.contributor.institutionCSAIL, MIT, Cambridge, MA
dc.identifier.volumePartF168147-13
dc.identifier.pages10023-10033
kaust.personWang, Di
dc.identifier.eid2-s2.0-85105337629
refterms.dateFOA2021-05-24T08:09:19Z


Files in this item

Thumbnail
Name:
wang20y.pdf
Size:
360.7Kb
Format:
PDF
Description:
publisher's version

This item appears in the following Collection(s)

Show simple item record