Towards Efficient Neuromorphic Hardware: Unsupervised Adaptive Neuron Pruning
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Permanent link to this recordhttp://hdl.handle.net/10754/663934
MetadataShow full item record
AbstractTo solve real-time challenges, neuromorphic systems generally require deep and complex network structures. Thus, it is crucial to search for effective solutions that can reduce network complexity, improve energy efficiency, and maintain high accuracy. To this end, we propose unsupervised pruning strategies that are focused on pruning neurons while training in spiking neural networks (SNNs) by utilizing network dynamics. The importance of neurons is determined by the fact that neurons that fire more spikes contribute more to network performance. Based on these criteria, we demonstrate that pruning with an adaptive spike count threshold provides a simple and effective approach that can reduce network size significantly and maintain high classification accuracy. The online adaptive pruning shows potential for developing energy-efficient training techniques due to less memory access and less weight-update computation. Furthermore, a parallel digital implementation scheme is proposed to implement spiking neural networks (SNNs) on field programmable gate array (FPGA). Notably, our proposed pruning strategies preserve the dense format of weight matrices, so the implementation architecture remains the same after network compression. The adaptive pruning strategy enables 2.3× reduction in memory size and 2.8× improvement on energy efficiency when 400 neurons are pruned from an 800-neuron network, while the loss of classification accuracy is 1.69%. And the best choice of pruning percentage depends on the trade-off among accuracy, memory, and energy. Therefore, this work offers a promising solution for effective network compression and energy-efficient hardware implementation of neuromorphic systems in real-time applications.
CitationGuo, W., Yantır, H. E., Fouda, M. E., Eltawil, A. M., & Salama, K. N. (2020). Towards Efficient Neuromorphic Hardware: Unsupervised Adaptive Neuron Pruning. Electronics, 9(7), 1059. doi:10.3390/electronics9071059
SponsorsWe acknowledge the financial support from King Abdullah University of Science and Technology (KAUST), Saudi Arabia.
Except where otherwise noted, this item's license is described as This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.