dc.contributor.author Mohamed, Abduallah dc.contributor.author Sadiq, Muhammed Mohaimin dc.contributor.author AlBadawy, Ehab dc.contributor.author Elhoseiny, Mohamed dc.contributor.author Claudel, Christian dc.date.accessioned 2020-11-18T12:38:20Z dc.date.available 2020-11-18T12:38:20Z dc.date.issued 2020-06-15 dc.identifier.uri http://hdl.handle.net/10754/666017 dc.description.abstract We introduce Inner Ensemble Networks (IENs) which reduce the variance within the neural network itself without an increase in the model complexity. IENs utilize ensemble parameters during the training phase to reduce the network variance. While in the testing phase, these parameters are removed without a change in the enhanced performance. IENs reduce the variance of an ordinary deep model by a factor of $1/m^{L-1}$, where $m$ is the number of inner ensembles and $L$ is the depth of the model. Also, we show empirically and theoretically that IENs lead to a greater variance reduction in comparison with other similar approaches such as dropout and maxout. Our results show a decrease of error rates between 1.7\% and 17.3\% in comparison with an ordinary deep model. We also show that IEN was preferred by Neural Architecture Search (NAS) methods over prior approaches. Code is available at https://github.com/abduallahmohamed/inner_ensemble_nets. dc.publisher arXiv dc.relation.url https://arxiv.org/pdf/2006.08305 dc.rights Archived with thanks to arXiv dc.title Inner Ensemble Networks: Average Ensemble as an Effective Regularizer dc.type Preprint dc.contributor.department Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division dc.eprint.version Pre-print dc.contributor.institution UT Austin. dc.contributor.institution 2UAlbany. dc.contributor.institution Stanford. dc.contributor.institution Equal Advising. dc.identifier.arxivid 2006.08305 kaust.person Elhoseiny, Mohamed dc.relation.issupplementedby github:abduallahmohamed/inner_ensemble_nets refterms.dateFOA 2020-11-18T12:38:53Z display.relations Is Supplemented By:
﻿

### Files in this item

Name:
Preprintfile1.pdf
Size:
425.7Kb
Format:
PDF
Description:
Pre-print