Analytic Expressions for Probabilistic Moments of PL-DNN with Gaussian Input
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Computer Science Program
Visual Computing Center (VCC)
Online Publication Date2018-12-18
Print Publication Date2018-06
Permanent link to this recordhttp://hdl.handle.net/10754/631804
MetadataShow full item record
AbstractThe outstanding performance of deep neural networks (DNNs), for the visual recognition task in particular, has been demonstrated on several large-scale benchmarks. This performance has immensely strengthened the line of research that aims to understand and analyze the driving reasons behind the effectiveness of these networks. One important aspect of this analysis has recently gained much attention, namely the reaction of a DNN to noisy input. This has spawned research on developing adversarial input attacks as well as training strategies that make DNNs more robust against these attacks. To this end, we derive in this paper exact analytic expressions for the first and second moments (mean and variance) of a small piecewise linear (PL) network (Affine, ReLU, Affine) subject to general Gaussian input. We experimentally show that these expressions are tight under simple linearizations of deeper PL-DNNs, especially popular architectures in the literature (e.g. LeNet and AlexNet). Extensive experiments on image classification show that these expressions can be used to study the behaviour of the output mean of the logits for each class, the interclass confusion and the pixel-level spatial noise sensitivity of the network. Moreover, we show how these expressions can be used to systematically construct targeted and non-targeted adversarial attacks.
CitationBibi A, Alfadly M, Ghanem B (2018) Analytic Expressions for Probabilistic Moments of PL-DNN with Gaussian Input. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Available: http://dx.doi.org/10.1109/CVPR.2018.00948.
SponsorsThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.
Conference/Event name31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018