Show simple item record

dc.contributor.authorAlfarra, Motasem
dc.contributor.authorBibi, Adel
dc.contributor.authorHammoud, Hasan
dc.contributor.authorGaafar, Mohamed
dc.contributor.authorGhanem, Bernard
dc.date.accessioned2020-02-27T13:20:16Z
dc.date.available2020-02-27T13:20:16Z
dc.date.issued2020-02-20
dc.identifier.urihttp://hdl.handle.net/10754/661768
dc.description.abstractThis work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine). Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of these zonotopes are functions of the neural network parameters. This geometric characterization provides new perspective to three tasks. Specifically, we propose a new tropical perspective to the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of a network's decision boundaries. Moreover, we use this characterization to propose a new set of tropical regularizers, which directly deal with the decision boundaries of a network. We investigate the use of these regularizers in neural network pruning (by removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (by producing input perturbations that explicitly perturb the decision boundaries' geometry and ultimately change the network's prediction).
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/2002.08838
dc.rightsArchived with thanks to arXiv
dc.titleOn the Decision Boundaries of Deep Neural Networks: A Tropical Geometry Perspective
dc.typePreprint
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.contributor.departmentElectrical Engineering
dc.contributor.departmentElectrical Engineering Program
dc.contributor.departmentVCC Analytics Research Group
dc.eprint.versionPre-print
dc.contributor.institutionFraunhofer Heinrich Hertz Institute, Berlin, Germany
dc.identifier.arxivid2002.08838
kaust.personAlfarra, Motasem
kaust.personBibi, Adel
kaust.personHammoud, Hasan
kaust.personGhanem, Bernard
refterms.dateFOA2020-02-27T13:20:59Z
kaust.acknowledged.supportUnitOffice of Sponsored Research


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
5.512Mb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record