Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
Type
ArticleKAUST Department
Computer Science ProgramComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Date
2020-09-22Online Publication Date
2020-09-22Print Publication Date
2021-09Embargo End Date
2021-09-22Submitted Date
2019-10-03Permanent link to this record
http://hdl.handle.net/10754/665355
Metadata
Show full item recordAbstract
Given a multivariate data set, sparse principal component analysis (SPCA) aims to extract several linear combinations of the variables that together explain the variance in the data as much as possible, while controlling the number of nonzero loadings in these combinations. In this paper we consider 8 different optimization formulations for computing a single sparse loading vector: we employ two norms for measuring variance (L2, L1) and two sparsity-inducing norms (L0, L1), which are used in two ways (constraint, penalty). Three of our formulations, notably the one with L0 constraint and L1 variance, have not been considered in the literature. We give a unifying reformulation which we propose to solve via the alternating maximization (AM) method. We show that AM is equivalent to GPower for all formulations. Besides this, we provide 24 efficient parallel SPCA implementations: 3 codes (multi-core, GPU and cluster) for each of the 8 problems. Parallelism in the methods is aimed at (1) speeding up computations (our GPU code can be 100 times faster than an efficient serial code written in C++), (2) obtaining solutions explaining more variance and (3) dealing with big data problems (our cluster code can solve a 357 GB problem in a minute).Citation
Richtárik, P., Jahani, M., Ahipaşaoğlu, S. D., & Takáč, M. (2020). Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes. Optimization and Engineering. doi:10.1007/s11081-020-09562-3Publisher
Springer NatureJournal
Optimization and EngineeringarXiv
1212.4137ae974a485f413a2113503eed53cd6c53
10.1007/s11081-020-09562-3