Large margin classification with indefinite similarities

Handle URI:
http://hdl.handle.net/10754/621497
Title:
Large margin classification with indefinite similarities
Authors:
Alabdulmohsin, Ibrahim ( 0000-0002-9387-5820 ) ; Cisse, Moustapha; Gao, Xin ( 0000-0002-7108-3574 ) ; Zhang, Xiangliang ( 0000-0002-3574-5665 )
Abstract:
Classification with indefinite similarities has attracted attention in the machine learning community. This is partly due to the fact that many similarity functions that arise in practice are not symmetric positive semidefinite, i.e. the Mercer condition is not satisfied, or the Mercer condition is difficult to verify. Examples of such indefinite similarities in machine learning applications are ample including, for instance, the BLAST similarity score between protein sequences, human-judged similarities between concepts and words, and the tangent distance or the shape matching distance in computer vision. Nevertheless, previous works on classification with indefinite similarities are not fully satisfactory. They have either introduced sources of inconsistency in handling past and future examples using kernel approximation, settled for local-minimum solutions using non-convex optimization, or produced non-sparse solutions by learning in Krein spaces. Despite the large volume of research devoted to this subject lately, we demonstrate in this paper how an old idea, namely the 1-norm support vector machine (SVM) proposed more than 15 years ago, has several advantages over more recent work. In particular, the 1-norm SVM method is conceptually simpler, which makes it easier to implement and maintain. It is competitive, if not superior to, all other methods in terms of predictive accuracy. Moreover, it produces solutions that are often sparser than more recent methods by several orders of magnitude. In addition, we provide various theoretical justifications by relating 1-norm SVM to well-established learning algorithms such as neural networks, SVM, and nearest neighbor classifiers. Finally, we conduct a thorough experimental evaluation, which reveals that the evidence in favor of 1-norm SVM is statistically significant.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Alabdulmohsin I, Cisse M, Gao X, Zhang X (2016) Large margin classification with indefinite similarities. Machine Learning 103: 215–237. Available: http://dx.doi.org/10.1007/s10994-015-5542-8.
Publisher:
Springer Science + Business Media
Journal:
Machine Learning
Issue Date:
7-Jan-2016
DOI:
10.1007/s10994-015-5542-8
Type:
Article
ISSN:
0885-6125; 1573-0565
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorAlabdulmohsin, Ibrahimen
dc.contributor.authorCisse, Moustaphaen
dc.contributor.authorGao, Xinen
dc.contributor.authorZhang, Xiangliangen
dc.date.accessioned2016-11-03T08:30:47Z-
dc.date.available2016-11-03T08:30:47Z-
dc.date.issued2016-01-07en
dc.identifier.citationAlabdulmohsin I, Cisse M, Gao X, Zhang X (2016) Large margin classification with indefinite similarities. Machine Learning 103: 215–237. Available: http://dx.doi.org/10.1007/s10994-015-5542-8.en
dc.identifier.issn0885-6125en
dc.identifier.issn1573-0565en
dc.identifier.doi10.1007/s10994-015-5542-8en
dc.identifier.urihttp://hdl.handle.net/10754/621497-
dc.description.abstractClassification with indefinite similarities has attracted attention in the machine learning community. This is partly due to the fact that many similarity functions that arise in practice are not symmetric positive semidefinite, i.e. the Mercer condition is not satisfied, or the Mercer condition is difficult to verify. Examples of such indefinite similarities in machine learning applications are ample including, for instance, the BLAST similarity score between protein sequences, human-judged similarities between concepts and words, and the tangent distance or the shape matching distance in computer vision. Nevertheless, previous works on classification with indefinite similarities are not fully satisfactory. They have either introduced sources of inconsistency in handling past and future examples using kernel approximation, settled for local-minimum solutions using non-convex optimization, or produced non-sparse solutions by learning in Krein spaces. Despite the large volume of research devoted to this subject lately, we demonstrate in this paper how an old idea, namely the 1-norm support vector machine (SVM) proposed more than 15 years ago, has several advantages over more recent work. In particular, the 1-norm SVM method is conceptually simpler, which makes it easier to implement and maintain. It is competitive, if not superior to, all other methods in terms of predictive accuracy. Moreover, it produces solutions that are often sparser than more recent methods by several orders of magnitude. In addition, we provide various theoretical justifications by relating 1-norm SVM to well-established learning algorithms such as neural networks, SVM, and nearest neighbor classifiers. Finally, we conduct a thorough experimental evaluation, which reveals that the evidence in favor of 1-norm SVM is statistically significant.en
dc.publisherSpringer Science + Business Mediaen
dc.subjectIndefinite kernelsen
dc.subjectLinear programmingen
dc.subjectSimilarity-based classificationen
dc.subjectSupervised learningen
dc.subjectSupport vector machineen
dc.titleLarge margin classification with indefinite similaritiesen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalMachine Learningen
kaust.authorAlabdulmohsin, Ibrahimen
kaust.authorCisse, Moustaphaen
kaust.authorGao, Xinen
kaust.authorZhang, Xiangliangen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.