FFTLasso: Large-Scale LASSO in the Fourier Domain

Handle URI:
http://hdl.handle.net/10754/626825
Title:
FFTLasso: Large-Scale LASSO in the Fourier Domain
Authors:
Bibi, Adel Aamer ( 0000-0002-6169-3918 ) ; Itani, Hani; Ghanem, Bernard ( 0000-0002-5534-587X )
Abstract:
In this paper, we revisit the LASSO sparse representation problem, which has been studied and used in a variety of different areas, ranging from signal processing and information theory to computer vision and machine learning. In the vision community, it found its way into many important applications, including face recognition, tracking, super resolution, image denoising, to name a few. Despite advances in efficient sparse algorithms, solving large-scale LASSO problems remains a challenge. To circumvent this difficulty, people tend to downsample and subsample the problem (e.g. via dimensionality reduction) to maintain a manageable sized LASSO, which usually comes at the cost of losing solution accuracy. This paper proposes a novel circulant reformulation of the LASSO that lifts the problem to a higher dimension, where ADMM can be efficiently applied to its dual form. Because of this lifting, all optimization variables are updated using only basic element-wise operations, the most computationally expensive of which is a 1D FFT. In this way, there is no need for a linear system solver nor matrix-vector multiplication. Since all operations in our FFTLasso method are element-wise, the subproblems are completely independent and can be trivially parallelized (e.g. on a GPU). The attractive computational properties of FFTLasso are verified by extensive experiments on synthetic and real data and on the face recognition task. They demonstrate that FFTLasso scales much more effectively than a state-of-the-art solver.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Electrical Engineering Program; Visual Computing Center (VCC)
Citation:
Bibi A, Itani H, Ghanem B (2017) FFTLasso: Large-Scale LASSO in the Fourier Domain. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Available: http://dx.doi.org/10.1109/cvpr.2017.465.
Publisher:
IEEE
Journal:
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Conference/Event name:
30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Issue Date:
9-Nov-2017
DOI:
10.1109/cvpr.2017.465
Type:
Conference Paper
Sponsors:
This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research. We thank Congli Wang for his help in the CUDA implementation.
Additional Links:
http://ieeexplore.ieee.org/document/8099948/
Appears in Collections:
Conference Papers; Electrical Engineering Program; Visual Computing Center (VCC); Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorBibi, Adel Aameren
dc.contributor.authorItani, Hanien
dc.contributor.authorGhanem, Bernarden
dc.date.accessioned2018-01-16T11:46:50Z-
dc.date.available2018-01-16T11:46:50Z-
dc.date.issued2017-11-09en
dc.identifier.citationBibi A, Itani H, Ghanem B (2017) FFTLasso: Large-Scale LASSO in the Fourier Domain. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Available: http://dx.doi.org/10.1109/cvpr.2017.465.en
dc.identifier.doi10.1109/cvpr.2017.465en
dc.identifier.urihttp://hdl.handle.net/10754/626825-
dc.description.abstractIn this paper, we revisit the LASSO sparse representation problem, which has been studied and used in a variety of different areas, ranging from signal processing and information theory to computer vision and machine learning. In the vision community, it found its way into many important applications, including face recognition, tracking, super resolution, image denoising, to name a few. Despite advances in efficient sparse algorithms, solving large-scale LASSO problems remains a challenge. To circumvent this difficulty, people tend to downsample and subsample the problem (e.g. via dimensionality reduction) to maintain a manageable sized LASSO, which usually comes at the cost of losing solution accuracy. This paper proposes a novel circulant reformulation of the LASSO that lifts the problem to a higher dimension, where ADMM can be efficiently applied to its dual form. Because of this lifting, all optimization variables are updated using only basic element-wise operations, the most computationally expensive of which is a 1D FFT. In this way, there is no need for a linear system solver nor matrix-vector multiplication. Since all operations in our FFTLasso method are element-wise, the subproblems are completely independent and can be trivially parallelized (e.g. on a GPU). The attractive computational properties of FFTLasso are verified by extensive experiments on synthetic and real data and on the face recognition task. They demonstrate that FFTLasso scales much more effectively than a state-of-the-art solver.en
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research. We thank Congli Wang for his help in the CUDA implementation.en
dc.publisherIEEEen
dc.relation.urlhttp://ieeexplore.ieee.org/document/8099948/en
dc.titleFFTLasso: Large-Scale LASSO in the Fourier Domainen
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentElectrical Engineering Programen
dc.contributor.departmentVisual Computing Center (VCC)en
dc.identifier.journal2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)en
dc.conference.dateJUL 21-26, 2016en
dc.conference.name30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)en
dc.conference.locationHonolulu, HIen
kaust.authorBibi, Adel Aameren
kaust.authorItani, Hanien
kaust.authorGhanem, Bernarden
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.