Show simple item record

dc.contributor.authorAl-Subaihi, Salman
dc.contributor.authorBibi, Adel
dc.contributor.authorAlfadly, Modar
dc.contributor.authorGhanem, Bernard
dc.contributor.authorHamdi, Abdullah
dc.date.accessioned2019-12-18T08:23:47Z
dc.date.available2019-12-18T08:23:47Z
dc.date.issued2019-05-28
dc.identifier.urihttp://hdl.handle.net/10754/660655
dc.description.abstractTraining Deep Neural Networks (DNNs) that are robust to norm bounded adversarial attacks remains an elusive problem. While verification based methods are generally too expensive to robustly train large networks, it was demonstrated in Gowal et al that bounded input intervals can be inexpensively propagated per layer through large networks. This interval bound propagation (IBP) approach led to high robustness and was the first to be employed on large networks. However, due to the very loose nature of the IBP bounds, particularly for large networks, the required training procedure is complex and involved. In this paper, we closely examine the bounds of a block of layers composed of an affine layer followed by a ReLU nonlinearity followed by another affine layer. To this end, we propose expected bounds, true bounds in expectation, that are provably tighter than IBP bounds in expectation. We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP. With such tight bounds, we demonstrate that a simple standard training procedure can achieve the best robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/1905.12418
dc.rightsArchived with thanks to arXiv
dc.titleExpected Tight Bounds for Robust Deep Neural Network Training
dc.typePreprint
dc.contributor.departmentElectrical Engineering Program
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.identifier.arxivid1905.12418
kaust.personAl-Subaihi, Salman
kaust.personBibi, Adel
kaust.personAlfadly, Modar
kaust.personGhanem, Bernard
kaust.personHamdi, Abdullah
refterms.dateFOA2019-12-18T08:24:22Z
kaust.acknowledged.supportUnitOffice of Sponsored Research


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
3.233Mb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record