dc.contributor.author Yuan, Ganzhao dc.contributor.author Zheng, Wei-Shi dc.contributor.author Shen, Li dc.contributor.author Ghanem, Bernard dc.date.accessioned 2019-04-28T13:15:22Z dc.date.available 2019-04-28T13:15:22Z dc.date.issued 2018-06-07 dc.identifier.uri http://hdl.handle.net/10754/632539 dc.description.abstract Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, $\ell_1$ norm regularized optimization, and $\ell_0$ norm regularized optimization as special cases. This paper proposes and analyzes a new Generalized Matrix Splitting Algorithm (GMSA) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Our algorithm is derived from a novel triangle operator mapping, which can be computed exactly using a new generalized Gaussian elimination procedure. We establish the global convergence, convergence rate, and iteration complexity of GMSA for convex problems. In addition, we also discuss several important extensions of GMSA. Finally, we validate the performance of our proposed method on three particular applications: nonnegative matrix factorization, $\ell_0$ norm regularized sparse coding, and $\ell_1$ norm regularized Dantzig selector problem. Extensive experiments show that our method achieves state-of-the-art performance in term of both efficiency and efficacy. dc.description.abstract Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, $\ell_1$ norm regularized optimization, and $\ell_0$ norm regularized optimization as special cases. This paper proposes and analyzes a new Generalized Matrix Splitting Algorithm (GMSA) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Our algorithm is derived from a novel triangle operator mapping, which can be computed exactly using a new generalized Gaussian elimination procedure. We establish the global convergence, convergence rate, and iteration complexity of GMSA for convex problems. In addition, we also discuss several important extensions of GMSA. Finally, we validate the performance of our proposed method on three particular applications: nonnegative matrix factorization, $\ell_0$ norm regularized sparse coding, and $\ell_1$ norm regularized Dantzig selector problem. Extensive experiments show that our method achieves state-of-the-art performance in term of both efficiency and efficacy. dc.description.abstract Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, $\ell_1$ norm regularized optimization, and $\ell_0$ norm regularized optimization as special cases. This paper proposes and analyzes a new Generalized Matrix Splitting Algorithm (GMSA) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Our algorithm is derived from a novel triangle operator mapping, which can be computed exactly using a new generalized Gaussian elimination procedure. We establish the global convergence, convergence rate, and iteration complexity of GMSA for convex problems. In addition, we also discuss several important extensions of GMSA. Finally, we validate the performance of our proposed method on three particular applications: nonnegative matrix factorization, $\ell_0$ norm regularized sparse coding, and $\ell_1$ norm regularized Dantzig selector problem. Extensive experiments show that our method achieves state-of-the-art performance in term of both efficiency and efficacy. dc.publisher arXiv dc.relation.url http://arxiv.org/abs/1806.03165v1 dc.relation.url http://arxiv.org/pdf/1806.03165v1 dc.rights Archived with thanks to arXiv dc.title A Generalized Matrix Splitting Algorithm dc.type Preprint dc.contributor.department Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division dc.contributor.department Electrical Engineering Program dc.contributor.department Visual Computing Center (VCC) dc.eprint.version Pre-print dc.contributor.institution Key Laboratory of Machine Intelligence and Advanced Computing, \nMinistry of Education, Beijing 221143, China. dc.contributor.institution School of Data and Computer Science, Sun Yat-sen University (SYSU), Guangzhou, Guangdong 510275, China dc.contributor.institution Tencent AI Lab, Shenzhen, China. dc.identifier.arxivid 1806.03165 kaust.person Ghanem, Bernard dc.version v1 refterms.dateFOA 2019-04-28T14:03:58Z
﻿

Name:
1806.03165v1.pdf
Size:
1.362Mb
Format:
PDF
Description:
Preprint