Maximum mutual information regularized classification

Handle URI:
http://hdl.handle.net/10754/556641
Title:
Maximum mutual information regularized classification
Authors:
Wang, Jim Jing-Yan; Wang, Yi; Zhao, Shiguang; Gao, Xin ( 0000-0002-7108-3574 )
Abstract:
In this paper, a novel pattern classification approach is proposed by regularizing the classifier learning to maximize mutual information between the classification response and the true class label. We argue that, with the learned classifier, the uncertainty of the true class label of a data sample should be reduced by knowing its classification response as much as possible. The reduced uncertainty is measured by the mutual information between the classification response and the true class label. To this end, when learning a linear classifier, we propose to maximize the mutual information between classification responses and true class labels of training samples, besides minimizing the classification error and reducing the classifier complexity. An objective function is constructed by modeling mutual information with entropy estimation, and it is optimized by a gradient descend method in an iterative algorithm. Experiments on two real world pattern classification problems show the significant improvements achieved by maximum mutual information regularization.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Maximum mutual information regularized classification 2015, 37:1 Engineering Applications of Artificial Intelligence
Publisher:
Elsevier BV
Journal:
Engineering Applications of Artificial Intelligence
Issue Date:
7-Sep-2014
DOI:
10.1016/j.engappai.2014.08.009
ARXIV:
arXiv:1409.7780
Type:
Article
ISSN:
09521976
Additional Links:
http://linkinghub.elsevier.com/retrieve/pii/S0952197614002085; http://arxiv.org/abs/1409.7780
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorWang, Jim Jing-Yanen
dc.contributor.authorWang, Yien
dc.contributor.authorZhao, Shiguangen
dc.contributor.authorGao, Xinen
dc.date.accessioned2015-06-10T11:13:39Zen
dc.date.available2015-06-10T11:13:39Zen
dc.date.issued2014-09-07en
dc.identifier.citationMaximum mutual information regularized classification 2015, 37:1 Engineering Applications of Artificial Intelligenceen
dc.identifier.issn09521976en
dc.identifier.doi10.1016/j.engappai.2014.08.009en
dc.identifier.urihttp://hdl.handle.net/10754/556641en
dc.description.abstractIn this paper, a novel pattern classification approach is proposed by regularizing the classifier learning to maximize mutual information between the classification response and the true class label. We argue that, with the learned classifier, the uncertainty of the true class label of a data sample should be reduced by knowing its classification response as much as possible. The reduced uncertainty is measured by the mutual information between the classification response and the true class label. To this end, when learning a linear classifier, we propose to maximize the mutual information between classification responses and true class labels of training samples, besides minimizing the classification error and reducing the classifier complexity. An objective function is constructed by modeling mutual information with entropy estimation, and it is optimized by a gradient descend method in an iterative algorithm. Experiments on two real world pattern classification problems show the significant improvements achieved by maximum mutual information regularization.en
dc.publisherElsevier BVen
dc.relation.urlhttp://linkinghub.elsevier.com/retrieve/pii/S0952197614002085en
dc.relation.urlhttp://arxiv.org/abs/1409.7780en
dc.rightsNOTICE: this is the author’s version of a work that was accepted for publication in Engineering Applications of Artificial Intelligence. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Engineering Applications of Artificial Intelligence, 7 September 2014. DOI: 10.1016/j.engappai.2014.08.009en
dc.subjectPattern classificationen
dc.subjectMaximum mutual informationen
dc.subjectEntropyen
dc.subjectGradient descenden
dc.titleMaximum mutual information regularized classificationen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalEngineering Applications of Artificial Intelligenceen
dc.eprint.versionPost-printen
dc.contributor.institutionDepartment of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USAen
dc.contributor.institutionDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang 150001, PR Chinaen
dc.identifier.arxividarXiv:1409.7780en
kaust.authorWang, Jim Jing-Yanen
kaust.authorGao, Xinen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.