A Matrix Splitting Method for Composite Function Minimization

Handle URI:
http://hdl.handle.net/10754/626454
Title:
A Matrix Splitting Method for Composite Function Minimization
Authors:
Yuan, Ganzhao; Zheng, Wei-Shi; Ghanem, Bernard ( 0000-0002-5534-587X )
Abstract:
Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization and cardinality regularized optimization as special cases. This paper proposes and analyzes a new Matrix Splitting Method (MSM) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Incorporating a new Gaussian elimination procedure, the matrix splitting method achieves state-of-the-art performance. For convex problems, we establish the global convergence, convergence rate, and iteration complexity of MSM, while for non-convex problems, we prove its global convergence. Finally, we validate the performance of our matrix splitting method on two particular applications: nonnegative matrix factorization and cardinality regularized sparse coding. Extensive experiments show that our method outperforms existing composite function minimization techniques in term of both efficiency and efficacy.
KAUST Department:
Visual Computing Center (VCC); Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Electrical Engineering Program
Publisher:
arXiv
Issue Date:
7-Dec-2016
ARXIV:
arXiv:1612.02317
Type:
Preprint
Additional Links:
http://arxiv.org/abs/1612.02317v1; http://arxiv.org/pdf/1612.02317v1
Appears in Collections:
Other/General Submission; Other/General Submission; Electrical Engineering Program; Visual Computing Center (VCC); Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorYuan, Ganzhaoen
dc.contributor.authorZheng, Wei-Shien
dc.contributor.authorGhanem, Bernarden
dc.date.accessioned2017-12-28T07:32:10Z-
dc.date.available2017-12-28T07:32:10Z-
dc.date.issued2016-12-07en
dc.identifier.urihttp://hdl.handle.net/10754/626454-
dc.description.abstractComposite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization and cardinality regularized optimization as special cases. This paper proposes and analyzes a new Matrix Splitting Method (MSM) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Incorporating a new Gaussian elimination procedure, the matrix splitting method achieves state-of-the-art performance. For convex problems, we establish the global convergence, convergence rate, and iteration complexity of MSM, while for non-convex problems, we prove its global convergence. Finally, we validate the performance of our matrix splitting method on two particular applications: nonnegative matrix factorization and cardinality regularized sparse coding. Extensive experiments show that our method outperforms existing composite function minimization techniques in term of both efficiency and efficacy.en
dc.publisherarXiven
dc.relation.urlhttp://arxiv.org/abs/1612.02317v1en
dc.relation.urlhttp://arxiv.org/pdf/1612.02317v1en
dc.rightsArchived with thanks to arXiven
dc.titleA Matrix Splitting Method for Composite Function Minimizationen
dc.typePreprinten
dc.contributor.departmentVisual Computing Center (VCC)en
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentElectrical Engineering Programen
dc.eprint.versionPre-printen
dc.contributor.institutionSun Yat-sen University (SYSU)en
dc.identifier.arxividarXiv:1612.02317en
kaust.authorYuan, Ganzhaoen
kaust.authorGhanem, Bernarden
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.