Show simple item record

dc.contributor.advisorBajic, Vladimir B.
dc.contributor.authorAlShahrani, Mona
dc.date.accessioned2015-05-27T08:34:20Z
dc.date.available2015-05-27T08:34:20Z
dc.date.issued2015-05-24
dc.identifier.urihttp://hdl.handle.net/10754/555862
dc.description.abstractArtificial Neural Networks (ANNs) are known to be among the most effective and expressive machine learning models. Their impressive abilities to learn have been reflected in many broad application domains such as image recognition, medical diagnosis, online banking, robotics, dynamic systems, and many others. ANNs with multiple layers of complex non-linear transformations (a.k.a Deep ANNs) have shown recently successful results in the area of computer vision and speech recognition. ANNs are parametric models that approximate unknown functions in which parameter values (weights) are adapted during training. ANN’s weights can be large in number and thus render the trained model more complex with chances for “overfitting” training data. In this study, we explore the effects of network pruning on performance of ANNs and ranking of features that describe the data. Simplified ANN model results in fewer parameters, less computation and faster training. We investigate the use of Hessian-based pruning algorithms as well as simpler ones (i.e. non Hessian-based) on nine datasets with varying number of input features and ANN parameters. The Hessian-based Optimal Brain Surgeon algorithm (OBS) is robust but slow. Therefore a faster parallel Hessian- approximation is provided. An additional speedup is provided using a variant we name ‘Simple n Optimal Brain Surgeon’ (SNOBS), which represents a good compromise between robustness and time efficiency. For some of the datasets, the ANN pruning experiments show on average 91% reduction in the number of ANN parameters and about 60% - 90% in the number of ANN input features, while maintaining comparable or better accuracy to the case when no pruning is applied. Finally, we show through a comprehensive comparison with seven state-of-the art feature filtering methods that the feature selection and ranking obtained as a byproduct of the ANN pruning is comparable in accuracy to these methods.
dc.language.isoen
dc.subjectArtificial
dc.subjectPrunning
dc.subjectNeural
dc.subjectNetwork
dc.subjectFeature
dc.subjectRanking
dc.titleTowards an Efficient Artificial Neural Network Pruning and Feature Ranking Tool
dc.typeThesis
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
thesis.degree.grantorKing Abdullah University of Science and Technology
dc.contributor.committeememberHoehndorf, Robert
dc.contributor.committeememberSolovyev, Victor
thesis.degree.disciplineComputer Science
thesis.degree.nameMaster of Science
refterms.dateFOA2016-05-24T00:00:00Z


Files in this item

Thumbnail
Name:
Mona Alsharani_MS_Thesis_final copy.pdf
Size:
15.42Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record